Chatbot Quality Assurance Using RPA


  • Ganesh Gatla 480 Meadow Hill Drive, Alpharetta, GA,30004


Chatbot, Quality Assurance, RPA


Chatbots are becoming mainstream consumer engagement tools, and well-developed chatbots are already transforming user experience and personalization. Chatbot Quality Assurance (QA) is an essential part of the development and deployment process, regardless of whether it’s conducted by one entity (business) or two (developers and business), to ensure ideal results. Robotic Process Automation (RPA) can be explored as a potential facilitator to improve, augment, streamline, or optimize chatbot QA. RPA is ideally suited for tasks that can be clearly defined (rule-based) and are repeating in nature. This limits its ability to become an all-encompassing technology for chatbot QA testing, but it can still be useful in replacing part of the manual QA testing of chatbots. Chatbot QA is a complex domain in its own right and has its own challenges, including the lack of streamlined/standardized testing protocols and quality measures, though traits like intent recognition, responsiveness, conversational flow, etc., are usually tested, especially at the end-user testing phase. RPA can be useful in certain areas of chatbot QA, including its ability to increase the sample size for training and testing datasets, generating input variations, splitting testing/conversation data sets, testing for typo resiliency, etc. The general rule is that the easier a testing process is to clearly define and set rules for, the better it's a candidate for RPA-based testing. This naturally increases the lean towards technical testing and makes it moderately unfeasible as an end-user testing alternative. It has the potential to optimize chatbot QA in conjunction with AI and ML testing tools.


J. Pereira and O. Diaz, "Chatbot Dimensions that Matter: Lessons from the Trenches," in Springer, 2018.

A. Følstad, C. B. Nordheim and C. A. Bjørkli, "What Makes Users Trust a Chatbot for Customer Service? An Exploratory Interview Study," in Lecture Notes in Computer Science, 2018.

M. Hermès and V. Juan, Towards A Better Understanding Of Service Quality Attributes Of A Chatbot, 2019.

L. Li, K. Y. Lee, E. Emokpae and S.-B. Yang, "What makes you continuously use chatbot services? Evidence from Chinese online travel agencies," Springer, 2021.

P. Hofmann, C. Samp, and N. Urbach, "Robotic process automation," Springer, 2019.

E. Adamopoulou and L. Moussiades, "Chatbots: History, technology, and applications," Machine Learning with Applications, vol. 2, 2020.

J. M. López-Morales, P. C. Cañizares, S. Pérez-Soler, E. Guerra and J. d. Lara, "Asymob: a platform for measuring and clustering chatbots," in ICSE ’22 Companion, Pittsburgh, PA, 2022.

S. Bravo-Santos, E. Guerra and J. d. Lara, "Testing Chatbots with Charm," in 13th International Conference, QUATIC - Springer, Faro, Portugal, 2020.

J. Jiang and N. Ahuja, "Response Quality in Human-Chatbot Collaborative Systems," in SIGIR, Virtual Event, China, 2020.

R. J. Moore and R. Arar, "Conversational UX Design: An Introduction," Studies in Conversational UX Design, p. 1 to 16, 2018.

W. M. P. v. d. Aalst, M. Bichler and A. Heinzl, "Robotic Process Automation," Business & Information Systems Engineering, vol. 60, p. 269–272, 2018.

L. Ivan?i?, D. S. Vugec, and V. B. Vukši?, "Robotic Process Automation: Systematic Literature Review," in International Conference on Business Process Management, Vienna, Austria, 2019.

R. Syed, S. Suriadi, M. Adams, W. Bandara, S. J. L. C. Ouyang, A. H. t. Hofstede, I. v. d. Weerd, M. T. Wynn and H. A. Reijers, "Robotic Process Automation: Contemporary themes and challenges," Computers in Industry, vol. 115, 2020.

K. C. Moffitt, A. M. Rozario, and M. A. Vasarhelyi, "Robotic Process Automation for Auditing," Journal of Emerging Technologies in Accounting, pp. 1-10, 2018.

S. Yatskiv, I. Voytyuk, N. Yatskiv, O. Kushnir, Y. Trufanova and V. Panasyuk, "Improved Method of Software Automation Testing Based on the Robotic Process Automation Technology," in International Conference on Advanced Computer Information Technologies (ACIT), Ceske Budejovice, Czech Republic, 2019.

B. Setiaji and F. W. Wibowo, "Chatbot Using A Knowledge in Database," in 7th International Conference on Intelligent Systems, Modelling, and Simulation, Bangkok, Thailand, 2016.

V. V., J. B. Cooper, and R. L. J., "Algorithm Inspection for Chatbot Performance Evaluation," in Procedia Computer Science, Trivandrum, India, 2020.

J. Bozic, O. A. Tazl and F. Wotawa, "Chatbot Testing Using AI Planning," in IEEE, Newark, CA, USA, 2019.

M. J. Hloušek, M. Smutek and Z. Hloušková, "SOCIAL COUNSELLING CHATBOT - PILOT TESTING," in NORDSCI, 2021.

A. F. Muhammad, D. Susanto, A. Alimudin, F. Adila, M. H. Assidiqi, and S. Nabhan, "Developing English Conversation Chatbot Using," in IEEE - International Electronics Symposium (IES), Surabaya, Indonesia, 2020.

D. Calvaresi, A. Ibrahim, J.-P. Calbimonte, R. Schegg, E. Fragniere and M. Schumacher, "The Evolution of Chatbots in Tourism: A Systematic Literature Review," in Springer, Cham - ENTER 2021 eTourism Conference, 2021.

V. Koumaras, A. Foteas, A. Papaioannou, M. Kapari, C. Sakkas and H. Koumaras, "5G Performance Testing of Mobile Chatbot Applications," in 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), Barcelona, Spain, 2018.

J. Bozic and F. Wotawa, "Security Testing for Chatbots," in Testing Software and Systems: 30th IFIP WG 6.1 International Conference, ICTSS, Cádiz, Spain, 2018.

N. Waheed, M. Ikram, S. S. Hashmi, X. He, and P. Nanda, "An Empirical Assessment of Security and Privacy Risks of Web based-Chatbots," in WISE 2020 Conference, Amsterdam and Leiden, Netherlands, 2020.

W. E. Hefny, Y. Mansy, M. Abdallah and S. Abdennadher, "Jooka: A Bilingual Chatbot for University Admission," World Conference on Information Systems and Technologies, vol. 1367, p. 671–681, 2021.

D. Al-Ghadhban and N. Al-Twairesh, "Nabiha: An Arabic Dialect Chatbot," (IJACSA) International Journal of Advanced Computer Science and Applications, vol. 11, no. 3, 2020.

P. Anki, A. Bustamam and R. A. Buyung, "Comparative Analysis of Performance between Multimodal Implementation of Chatbot Based on News Classification Data Using Categories," MDPI Electronics, 2021.

E. Amer, A. Hazem, O. Farouk, A. Louca, Y. Mohamed and M. Ashraf, "A Proposed Chatbot Framework for COVID-19," in International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), Cairo Egypt, 2021.




How to Cite

Gatla, G. (2022). Chatbot Quality Assurance Using RPA. American Scientific Research Journal for Engineering, Technology, and Sciences, 90(1), 362–370. Retrieved from