Citation:

Muhammad Irfan, Mochamad Yudha Febrianta, Dian Puteri Ramadhani. "Analysis of ChatGPT Application Service Quality Using Text Classification and Topic Modeling" International Research Journal of Economics and Management Studies, Vol. 3, No. 8, pp. 412-418, 2024.

Abstract:

This research aims to understand the sentiments and main topics of users related to the service quality of the ChatGPT Android application by analyzing User-Generated Content (UGC) on the Google Play Store based on user experiences. The research method used in this study is sentiment analysis with the RoBERTa algorithm and topic modeling using LDA. After comparing the Naive Bayes, SVM, and RoBERTa algorithms, RoBERTa was found to have the highest accuracy, with 72% for dimension classification and 94% for sentiment classification. Positive sentiment was 86.17%, and negative sentiment was 13.83%. The dimensions used to measure service quality are Content Quality, Engagement, Reliability, Usability, and Privacy. The results of the study show that the service quality of the ChatGPT Android application is mostly positive in the dimensions of Engagement, Reliability, and Content Quality. However, the dimensions of Privacy and Usability have negative views. Positive sentiment findings include satisfaction with the information provided, the dark mode feature, the user-friendly interface, user engagement, and application reliability. On the other hand, negative sentiment highlights issues with privacy and usability. This knowledge can be used to improve service quality, especially in the dimensions of Privacy and Usability.

References:

[1] Alamsyah, A., & Girawan, N. D. (2023). Improving Clothing Product Quality and Reducing Waste Based on Consumer Review Using RoBERTa and BERTopic Language Model. Big Data and Cognitive Computing, 7(4), 168. https://doi.org/10.3390/bdcc7040168
[2] Ariyanto, A., Bangun, R., Rifqi, M., Indillah, M., Ferlina, A., Trenggana, M., Sholihah, R., Ariyanti, M., Widiati, E., Irawan, P., Ratih, S. D., Suryanti Ismail, R., Putra, S., Mulia Utama, A., Syahputra, J., & Budiman, B. (2023). MANAJEMEN PEMASARAN. www.freepik.com
[3] Boukabous, M., & Azizi, M. (2021). A comparative study of deep learning-based language representation learning models. Indonesian Journal of Electrical Engineering and Computer Science, 22(2), 1032–1040. https://doi.org/10.11591/ijeecs.v22.i2.pp1032-1040
[4] Br.Ginting, D. Y., Suyanto, A., & Mochamad, Y. F. (2022). Analysis Of The Effect Of E-Service Quality On E-Customer Satisfaction And E-Customer Loyalty On Shopee. Asian Journal of Research in Business and Management. https://doi.org/10.55057/ajrbm.2022.4.3.35
[5] Data Boks. (2023, July). Data Pengguna ChatGPT Bocor di “Dark Web”, Indonesia Masuk Daftar Kebocoran 10 Besar. DataBoks.
[6] Garini, J. G., Hidayanto, A. N., & Fina, A. (2023). Using machine learning to improve a telco self-service mobile application in Indonesia. IAES International Journal of Artificial Intelligence (IJ-AI), 12(4), 1947. https://doi.org/10.11591/ijai.v12.i4.pp1947-1959
[7] Gloria, M., & Talavera, V. (2020). Measuring Service Quality in Philippine Banks: An Exploratory Study Using SERVQUAL and Q-Methodology. In Philippine Management Review (Vol. 27).
[8] Kim Ki-Hun, Kim Kwang-Jae, Lee Dae-Ho, & Kim Min-Geun. (2019). Identification of critical quality dimensions for continuance intention in. International Journal of Information Management.
[9] Kotler, P., Keller, K. L., & Chernev Alexander. (2022). Marketing Management (16 Edition) (16th ed.). Pearson Education.
[10] Masrury, R. A., Fannisa, & Alamsyah, A. (2019, July 1). Analyzing tourism mobile applications perceived quality using sentiment analysis and topic modeling. 2019 7th International Conference on Information and Communication Technology, ICoICT 2019. https://doi.org/10.1109/ICoICT.2019.8835255
[11] Okey, O. D., Udo, E. U., Rosa, R. L., Rodríguez, D. Z., & Kleinschmidt, J. H. (2023). Investigating ChatGPT and cybersecurity: A perspective on topic modeling and sentiment analysis. Computers and Security, 135. https://doi.org/10.1016/j.cose.2023.103476
[12] Rio Sasongko, S. (2021). FAKTOR-FAKTOR KEPUASAN PELANGGAN DAN LOYALITAS PELANGGAN (LITERATURE REVIEW MANAJEMEN PEMASARAN). 3(1). https://doi.org/10.31933/jimt.v3i1
[13] Rukmayuninda Ririh, K., Laili, N., Wicaksono, A., & Tsurayya, S. (2020). STUDI KOMPARASI DAN ANALISIS SWOT PADA IMPLEMENTASI KECERDASAN BUATAN (ARTIFICIAL INTELLIGENCE) DI INDONESIA. In Jurnal Teknik Industri (Vol. 15, Issue 2).
[14] Shidqi, F., & Yudha Febrianta, M. (2023). Analisis Kualitas Layanan Internet Service Provider Menggunakan Metode Analisis Sentimen Dan Topic Modelling. SEIKO: Journal of Management & Business, 6(2), 439–450.
[15] Suryawardani, B., Wulandari, A., Satrya, G. B., Marcelino, D., Prawita, F. N., Febrianta, M. Y., & Tantra, T. (2022). Digital Tools for Co-working Space Batununggal Using Digital-Ads, AR-Based Socmed Content, and Enterprise Resource Planning. Jurnal Penyuluhan, 19(01), 12–21. https://doi.org/10.25015/19202341777
[16] Yang, L., Li, Y., Wang, J., & Sherratt, R. S. (2020). Sentiment Analysis for E-Commerce Product Reviews in Chinese Based on Sentiment Lexicon and Deep Learning. IEEE Access, 8, 23522–23530. https://doi.org/10.1109/ACCESS.2020.2969854
[17] Zeithaml, V. A., Bitner, M. J., & Gremler, D. D. (2017). Services marketing: integrating customer focus across the firm.
[18] Zhou, Q., Lim, F. J., Yu, H., Xu, G., Ren, X., Liu, D., Wang, X., Mai, X., & Xu, H. (2021). A study on factors affecting service quality and loyalty intention in mobile banking. Journal of Retailing and Consumer Services, 60. https://doi.org/10.1016/j.jretconser.2020.102424

Keywords:

ChatGPT, Sentiment Analysis, Topic Modeling, Service Quality.