Interaction between IDEs and AI Assistants in the Process of Writing Automated Tests
Keywords:
IDE, AI assistant, automated tests, GitHub Copilot, Kotlin Multiplatform, Page Object, mobile automation, flakiness, feature-flag, E2E-testingAbstract
This paper looks at how modern, integrated development environments (IDEs) work with AI helpers to make it easier to write automated tests. The paper tries to see how far these kinds of tools take away the boring workload from engineers and better the quality of test thinking. Fast growth in using AI helpers in IDEs justifies this study: 74% of developers plan on still using ChatGPT, and 41% — GitHub Copilot. More than 80% of firms have put Copilot into everyday use, and 90% say it has raised job happiness. Meanwhile, only 31% of suggestions are taken up, and 17% of those stay in the codebase, showing that AI has a small degree of independence on automated tests. This work remains novel in a systematic analysis that intertwines quantitative survey data with Copilot usage telemetry, and crowdsourced info on mobile platform fragmentation, further coupled with an architectural review of AI-plugin integration within Android Studio and IntelliJ IDEA. The support covers Kotlin Multiplatform, Gradle scripts, Page Object, Gherkin, and cross-platform steps generation. Suggestion acceptance rates, flakiness metrics, and CI/CD licensing requirements have been integrated to evaluate the real-world impact of AI assistants. Findings at a high level indicate that helpful AI assistants in IDEs reduce the time it takes to build automated test scaffolds; helpful ones further automate locator choice considering Android/iOS fragmentation; they generate cross-platform scenarios in KMP projects and reduce flaky testing percentages through helpful wait-template recommendations. In return, though, suggestions are accepted by only about one-third of engineers; responsibility for quality and security of tests remains with the expert and therefore requires code review, static analysis, as well as license scanning. This article will help test developers, QA engineers, and automation architects who use AI helpers to make mobile and cross-platform testing better and more trustworthy.
References
[1] “Technology | 2024 Developer Survey,” Stackoverflow. https://survey.stackoverflow.co/2024/technology#worked-with-vs-want-to-work-with (accessed Apr. 22, 2025).
[2] Y. Gao, “Research: Quantifying GitHub Copilot’s impact in the enterprise with Accenture,” The GitHub Blog, May 13, 2024. https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-in-the-enterprise-with-accenture/ (accessed Apr. 23, 2025).
[3] V. Vasudevan, “Github Copilot Adoption Trends: Insights from Real Data,” Opsera, Apr. 29, 2025. https://www.opsera.io/blog/github-copilot-adoption-trends-insights-from-real-data (accessed Apr. 30, 2025).
[4] Jash Unadkat, “Difference between Mobile and Web Application Testing,” Browser Stack, Dec. 13, 2024. https://www.browserstack.com/guide/differences-between-mobile-application-testing-and-web-application-testing (accessed Apr. 24, 2025).
[5] J. Rossignol, “Apple Reveals How Many iPhones Are Running iOS 18,” MacRumors, Jan. 29, 2025. https://www.macrumors.com/2025/01/29/ios-18-adoption-stats/ (accessed Apr. 25, 2025).
[6] M. Ulianov, “How GitHub Copilot Promotes Junior Developers to Mid-Level,” FullStack Labs, Feb. 10, 2025. https://www.fullstack.com/labs/resources/blog/how-github-copilot-promotes-junior-developers-to-mid-level (accessed Apr. 26, 2025).
[7] “State of Developer Ecosystem Report 2024,” JetBrains, 2024. https://www.jetbrains.com/lp/devecosystem-2024/ (accessed Apr. 26, 2025).
[8] “The State of Developer Ecosystem in 2021,” JetBrains. https://www.jetbrains.com/lp/devecosystem-2021/kotlin/ (accessed Apr. 27, 2025).
[9] “Full compatibility and enhanced authentication for GitHub Copilot in JetBrains IDEs 2024.3 - GitHub Changelog,” The GitHub Blog, Dec. 04, 2024. https://github.blog/changelog/2024-12-04-full-compatibility-and-enhanced-authentication-for-github-copilot-in-jetbrains-ides-2024-3/ (accessed Apr. 28, 2025).
[10] J. Clover, “iOS 18 Installed on 76% of iPhones Introduced in the Last Four Years,” MacRumors, Jan. 24, 2025. https://www.macrumors.com/2025/01/24/ios-18-adoption-rate/ (accessed May 01, 2025).
[11] “Android Version Market Share Worldwide,” StatCounter Global Stats, Apr. 2025. https://gs.statcounter.com/android-version-market-share (accessed May 03, 2025).
[12] “AndroidX Test Library,” Mvn repository. https://mvnrepository.com/artifact/androidx.test.espresso/espresso-core (accessed May 07, 2025).
[13] “UI automator,” Mvn repository. https://mvnrepository.com/artifact/androidx.test.uiautomator/uiautomator (accessed May 09, 2025).
[14] L. Cruz and R. Abreu, “On the Energy Footprint of Mobile Testing Frameworks,” IEEE Transactions on Software Engineering (TSE), Oct. 2019, doi: https://doi.org/10.48550/arxiv.1910.08768.
[15] A. Crudu, “Unit Testing vs UI Testing in iOS Development - When to Use Each for Optimal Results,” MoldStud, Apr. 06, 2025. https://moldstud.com/articles/p-unit-testing-vs-ui-testing-in-ios-development-when-to-use-each-for-optimal-results (accessed May 10, 2025).
[16] J. Micco, “Flaky Tests at Google and How We Mitigate Them,” Google Testing Blog, May 27, 2016. https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html (accessed May 11, 2025).
[17] “How Much Testing is Enough?” Google Testing Blog, Jun. 15, 2021. https://testing.googleblog.com/2021/ (accessed May 12, 2025).
[18] “Pricing,” Browser Stack. https://www.browserstack.com/pricing?cycle=annual (accessed May 13, 2025).
[19] A. Anisimov, “Results of the First Kotlin Multiplatform Survey,” JetBrains, Apr. 28, 2021. https://blog.jetbrains.com/kotlin/2021/01/results-of-the-first-kotlin-multiplatform-survey/ (accessed May 15, 2025).
[20] E. López-Mañas, “The State of Developer Ecosystem in 2023,” JetBrains. https://www.jetbrains.com/lp/devecosystem-2023/kotlin/ (accessed May 16, 2025).
[21] “Test your multiplatform app − tutorial,” JetBrains, May 19, 2025. https://www.jetbrains.com/help/kotlin-multiplatform-dev/multiplatform-run-tests.html (accessed May 20, 2025).
[22] “Multiplatform expect / actual module support,” GitHub, Dec. 07, 2024. https://github.com/InsertKoinIO/koin-annotations/issues/105 (accessed May 17, 2025).
[23] “Kotlin Multiplatform’s three levels of testing with Kotest,” Dev Community, Jan. 21, 2023. https://dev.to/bjornvdlaan/kotlin-multiplatforms-three-levels-of-testing-with-kotest-3e24 (accessed May 18, 2025).
[24] R. Lukasik, “Running our Android Espresso tests in Parallel,” Medium, Oct. 27, 2023. https://rlukasik.medium.com/running-our-android-espresso-tests-in-parallel-2c13ef9231f7 (accessed May 19, 2025).
[25] “Copilot Chat now has a 64k context window with OpenAI GPT-4o - GitHub Changelog,” The GitHub Blog, Dec. 06, 2025. https://github.blog/changelog/2024-12-06-copilot-chat-now-has-a-64k-context-window-with-openai-gpt-4o/ (accessed May 19, 2025).
[26] Y. Fu, “Security Weaknesses of Copilot Generated Code in GitHub,” Arxiv.org, 2023. https://arxiv.org/html/2310.02059v2 (accessed May 28, 2025).
Downloads
Published
Issue
Section
License
Copyright (c) 2025 American Scientific Research Journal for Engineering, Technology, and Sciences
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who submit papers with this journal agree to the following terms.
