Abstract

A chatbot system enables communication between computer and users through natural language. The university students face many problems for the queries and details regarding their examination and it is also difficult for them to physically visit the examination department frequently. The concerned authorities find it challenging to handle thousands of student queries in parallel, thus, consuming a lot of time. At present, majority of the chatbot system are available in English language. Therefore, people who are not fluent in English language cannot use these systems due to the obvious reason of language barrier. This paper focuses on the viability of a chatbot system for the examination department of Mehran University of Engineering Technology (MUET) Jamshoro. The chatbot discussed in this paper has been designed to support both Urdu and English languages. Three chatbot interfaces have been developed, Urdu typing based chatbot, English typing based chatbot and English voice based Chabot. The typing based chatbots (both English and Urdu) have been developed using AIML, and voice based chatbot is based on dialogflow called as API.ai. This study is not only concerned with the development of three interfaces of chatbot systems, but also covers their evaluation. To find best chatbot interface, a survey was conducted with engineering students of MUET Jamshoro. The results revealed that the among the three chatbot interfaces, English typing-based interface is the best due to the usability, easiness and the general understanding of queries.

Keywords: Chatbot, AIML, Dialogflow, User interface, Urdu.

Received: 16 July 2019 / Revised: 21 August 2019 / Accepted: 23 September 2019/ Published: 29 October 2019

Contribution/ Originality

This paper contribution is twofold, first is the development of a chatbot system that enables communication in Urdu and English languages. Second is the comparison of both chatbots by performing the user testing.

1. INTRODUCTION

Worldwide various chat systems are available that enable communication using natural languages. These chat systems are broadly categorized into two main types namely, Human-Human Dialog System and Human-Computer Dialog System. There is not any involvement of machine learning in Human-Human Dialog System and this work as a negotiator between two humans. The Human-Human Dialog System doesn’t require natural processing abilities through machines. Globally, WhatsApp and skype are the most popular Human-Human Dialog chat system. However, the other available chat system, the Human-Computer Dialog System is termed as chatbot [1].

Basically, chatbot is a computer program that pretends chat with humans through natural language [1] 2]. On any platform like mobile, website and desktop application, this system can interact with the humans [3]. While interacting with the human, chatbot simulates as a human being. Human being only interacts with one human at a time, the chatbot interacts and communicates with hundreds and thousands of persons simultaneously ¹ . It works and responds without considering how many persons are interacting and what time of the day and night it is. However, for natural language processing ability, the development of chatbot is an arduous task. Chatbots are applicable in many fields such as in education, traveling, real estate, internet, gaming, education, ecommerce, hospitality and health, call centers, media, financial bot insurance banks, business, traveling customer service and in shopping [4-6]² .

In universities, for the students and administration departments, there is need to develop Human-Computer Dialog System (Chatbot) for the facilitation of students and easiness of service providers. University students face many problems for getting details and queries concerning their examination related documentation. This may affect students’ academic progress and cause tension, mental agony and distraction from studies. It is laborious for authorities to deal with thousands of student’s queries in parallel and consumes a lot of time. For solving above mentioned problems, the purpose here is to develop a MUET Examination Chatbot that facilitates and serves number of students at the same time by providing answers to their required queries. The system replies using an effective Graphical User Interface, similar to a real person talking to the user. With the help of the proposed system, students immediately get information of no dues certificates, pass certificate, transcript certificate degree certificate, exam form, exam fees, challan submission process, photo size, how to fill up online exam form, penalty on late fee submission and details of documents required for forms and certificates. This system also reduces burden on the employees working in examination department at MUET Jamshoro.

Majority of the available chatbot systems are in English, but, in Pakistan among fifty-seven spoken languages, only five percent peoples can understand and speak English [7] 8]. Due to the obvious reason of language barrier the people who are not fluent in English language can’t use these systems. Due to this reason, the chatbot is developed enabling communication in Urdu language as well. The two main contributions of this paper are:

To develop a chatbot system that enables communication in Urdu and English languages.
To compare all interfaces by testing them with real users and find best interface among them.

2. LITERATURE REVIEW

Chatbot have been developed to meet a variety of applications and can be utilized in several fields. This section presents numerous chatbot related studies in academia.

The study of Mikic, et al. [9] demonstrated the development of “CHARLIE”, a chatterbot based on AIML and can be used as an interface among human and intelligent educational system (INES). CHARLIE enables common chat with students to answer student’s queries. This chatbot shows the course material and responds to queries related with the material. The Bhargava and Nikhil [10] study was concerned with the development of intelligent speech recognition system which is an e-learning system for students. This system is particularly designed for disable persons. This is an AIML based system that utilizes voice recognition for groups of words and phonemes. The objective of this study was to include an English language tutorial software easily accessible by disabled persons especially with blindness and hand paralysis. Jia [11] developed a computer assisted English learning chatbot “CSIEC” based on reasoning and textual knowledge with multiple functions designed for English instruction mainly targeting on supplying a virtual chatting partner, which enables chat in English the users anywhere and anytime. It responds and communicates according to the user input, user and its personality knowledge, inference knowledge, dialogue context, and common-sense knowledge. All knowledge are stored in NLML form for natural language text.

The Michelle [12] designed a Drothy chatbot that is a network management aware system which not only interact with the user but also interact with the managed network through management protocol like SNMP. It works as original network management tool. In McKenzie [13] authors developed a career advisor chat for Leeds University. This chatbot stores knowledge in AIML form. This career advisor chatbot takes generic simple question from students and give appropriate response. A research was conducted by Doshi, et al. [14] in which they developed an android based artificial intelligence chatbot system. They used program-o which is an AIML interpreter that generates response of user input. This chatbot interact with user by using voice and text responses. In Cameron, et al. [15] the researchers demonstrated the design for a medical chatbot that diagnose and provides remedies based on symptoms user provides to the system. It is text-based system in which knowledge is stored in AIML files. This system can determine the seriousness of the diagnoses and if required it may connect the user to the online available doctor.

One of the systems proposed by Thomas [16] is an e-business chatbot. This e-business chatbot system utilizes AIML and Latent Sematic Analysis (LSA) and gives customer care service on e-commerce websites. Kataria, et al. [17] presented an online chatbot known as bot autonomous emotional support (BAES) that helps to uplift the mental condition of depressed people. BAES can help Indian public user and may understand both Hindi and English languages. Authors in Hussain and Athula [18] proposed a chatbot named as Virtual diabetes management system VDMS, that can be used by diabetic patients and general community for diabetes education and management. Another system proposed in Lokman and Zain [19] showed the architectural design of virtual dietitian (ViDi), a system that will help diabetic patient and function as a virtual dietitian. In architectural design of this system Vpath is used to remember the conversation path.

Authors in Kazi, et al. [20] explained the development of a chatbot for medical students that is based on the open source AIML. In this they utilized Unified Medical Language System (UMLS) as a domain knowledge source and produce responses of queries. The study of Satu and Parvez [21] demonstrated the review of applications of chatbot which are designed by deploying the AIML script. They provided detail information regarding the various applications of chatbot. They also described that the chatbot based on AIML are efficient in work, light in weight and easy to implement.

3. DESIGN AND IMPLEMENTATION

This section explains the design and implementation of MUET examination department chatbot system. It comprises of two parts, in first part, we describe the detailed design and implementation of typing based chatbot, whereas, in second part, we explain the design and implementation of voice based chatbot. The typing based chatbot system supports both English and Urdu languages and voice based chatbot system supports only English language.

3.1. Design and Implementation of Typing Based Chatbot

We have designed a web based chatbot that is capable to chat in both Urdu and English languages. The knowledge base of this chatbot has been designed using AIML. The architecture of typing based chatbot is shown in Figure 1. This chatbot functions in the specific domain of examination department of MUET. It is developed and coded using python programing language together with Pyaiml, flask framework, html, cascading style sheets (CSS), jQuery, JavaScript and bootstrap.

Figure-1. Architecture of typing based chatbot.

Typing based chatbot takes the text input from user which is passed to knowledge identification. Pyaiml is used for knowledge identification. It is a python AIML interpreter which is completely written in python instead of utilizing any third-party reliance. Pyaiml constructs a directed pattern tree by reading all pattern of AIML from database. The backtracking depth-first search algorithm is implemented in Pyaiml for pattern matching. In database, the data is stored in AIML object form.

Figure-2. Knowledge of typing based chatbot.

The knowledge of typing based chatbot is shown in Figure 2. When AIML interpreter receives user input, it is converted in AIML pattern and answer is fetched by using backtracking depth-first search algorithm. After matching the pattern, the required answer is return to Pyaiml, which is then displayed to the user. For the next user input, the same process is repeated.

3.2. User Interface of Typing Based Chatbot

The main page or home page of typing based chatbot provides navigation choices for chatbots in two different languages. First option allows the user to start communication with bot using English language. Figure 3 shows user interface of English typing-based Chabot. In second option, the user can communicate with bot using Urdu language. Figure 4 shows user interface of typing based Urdu chatbot. The chatbot user interface is a text box where user can write their quires. After receiving the queries, the chatbot sends this input to the system and after receiving the response, it is displayed on the screens.

Figure-3. User interface of English typing based chatbot.

Figure-4. User interface of Urdu typing based chatbot.

3.3. Design and Implementation of Voice Based Chatbot

To demonstrate the voice based approach, we have designed a web based chatbot that can be used to chat with the bot through voice in English language only. The voice based chatbot has been developed using API.ai which is also called as dialogeflow framework. The architecture of voice based chatbot is shown in Figure 5. The MUET examination voice based chatbot can function in the specific domain of examination department. In technical details, MUET voice based chatbot is being coded using node js, Jason, JavaScript, CSS, html and API.ai (dialogflow).

Figure-5. Architecture of voice based chatbot.

Voice based chatbot takes the voice input from user using the microphone. The Web speech APIs captures user voice from the microphone and conveys the input message to dialogflow for response. The web speech APIs is used for speech recognition and speech synthesis interface, it converts the user’s voice messages to text and text messages to voice. Dialogflow first examines the input message, received input message is matched and categorized to corresponding intent. An intent map what a user says and how he wishes the system to respond. When an intent is got by dialogflow, then required information is send to fulfillments. Further fulfillments produce dynamic responses which are based on information looked up from a database. The database of voice based chatbot is shown in Figure 6. Once it is retrieved, the necessary information is passed back through dialogflow. The dialogflow returns the response text back to the user by using the speech synthesis interface to give it a synthetic voice.

3.4. User Interface of Voice Based Chatbot

Figure 7 shows the home screen of voice based chatbot system; here user can start communication with the bot by speaking in English language after clicking on microphone button. When the input is received in the form of vocal commands from this user interface, it is forwarded to the system for processing. Once the response is ready, it is forwarded to the user in the form of male voice.

Figure-6. Database of voice based chatbot.

Figure-7. Home screen of voice based chatbot.

4. EVALUATION

The evaluation was carried out in order to compare and find the best interface among the three chatbot interfaces. During the evaluation process, 50 students from MUET Jamshoro were asked to test all the three flavors of chatbots by asking three questions. Among the participants who took part in evaluation, 29 were male and 21 were female with ages between 19 and 22. A survey was presented to find the best interface by general understanding of the students regarding chatbot usability and easiness. The survey questions and the participant’s responses are given in Table 1. The results and student’s feedback regarding the use of three interfaces of chatbot system were calculated and presented in graphs.

5. RESULTS AND FINDINGS

In the survey, the participants were given options from ‘strongly agree’ to ’strongly disagree’ to select the option best suited for them. This survey was conducted by all participants of the evaluation process of the three interfaces of chatbot system. Figure 8] 9] 10] 11] 12 shows the result of the conducted survey.

Table-1. Survey questions and responses.

Survey questions	Strongly agree	Agree	Disagree	Strongly disagree
1. It’s easy to use English typing based chatbot?	52%	48%
2. It’s easy to use Urdu typing based chatbot?	50%	50%
3. It’s easy to use English voice based chatbot?	42%	58%
4. I like the design and colors used for the English typing based chatbot.	56%	44%
5. I like the design and colors used for the Urdu typing based chatbot.	48%	46%	6%
6. I like the design and colors used for the English voice based chatbot.	52%	48%
7. I notice any spelling / grammar mistakes in English typing based chatbot.			2%	98%
8. I notice any spelling / grammar mistakes in Urdu typing based chatbot.			4%	96%
9. I notice any spelling / grammar mistakes in English voice based chatbot.			4%	96%
10. The text clear and legible in the interface English typing based chatbot.	64%	36%
11. The text clear and legible in the interface Urdu typing based chatbot.	38%	60%	2%
12. The text clear and legible in the interface English voice based chatbot.	50%	50%
13. It’s beneficial, if we attach English typing based chatbot with examination department?	53%	47%
14. It’s beneficial, if we attach Urdu typing based chatbot with examination department?	48%	52%
15. It’s beneficial, if we attach English voice based chatbot with examination department?	50%	50%
16. I want to use English typing based chatbot in any other place.	50%	47%	4%
17. I want to use Urdu typing based chatbot in any other place.	32%	62%	6%
18. I want to use English voice based chatbot in any other place.	44%	52%	2%	2%
19. Using English typing based chatbot. I believe that I was talking to a real examination centre employee.	68%	30%	2%
20. Using Urdu typing based chatbot. I believe that I was talking to a real examination centre employee.	58%	38%	4%
21. Using English voice based chatbot. I believe that I was talking to a real examination centre employee.	62%	34%	4%
22. I find chatting to the English typing based examination department chatbot helpful.	54%	46%
23. I find chatting to the Urdu typing based examination department chatbot helpful.	48%	52%
24. I find chatting to the English voice based examination department chatbot helpful.	38%	62%

Figure 8 a, b, c is showing the combined comparative analysis of question 1, 2, 3. During testing of English typing based chatbot, out of 50 students, 52% of students replied with “strongly agree” with using this interface of chatbot and remaining 48% replied with “agree”. None of the student has found it difficult while using that English typing based chatbot. In testing Urdu typing based chatbot, 25 students among total of 50 students responded with “strongly agree” with using this Urdu typing based interface of chatbot, while the remaining 25 students replied with “agree”. None of the student has found it difficult while using that interface. In testing English voice based chatbot, among all students, 21 students (42%) responded “strongly agree” to use voice based chatbot and 26 students (58%) responded with “agree” to use this voice based chatbot. After comparative analysis of results, all interfaces were found easy to use and among three English typing based chatbot is found best.

Figure-8. Evaluation result of typing-based and voice based chatbot interfaces.

Figure 9 a, b, c is showing the combined comparative analysis of question 4, 5, 6. These questions are related to graphical user interface of chatbot. For example, design, color. After comparative analysis of results, it is observed that majority of students responded with “strongly agree” and “agree” and only 6% students are disagreed with design and color Urdu typing based chatbot.

Figure-9. Evaluation result of chatbot graphical user interface.

Figure 10a, b, c is depicting the combined comparative analysis of question 7, 8, 9. These questions are related to text or grammatical mistake. After comparative analysis of results, it is perceived that all of students provided positive results.

Figure 11a, b, c is portraying the combined comparative analysis of question 10, 11, 12. These questions are related to text size of chatbot. After comparative analysis of results, it is observed that majority of students provide positive result and only 2% of students are disagreed with text size of Urdu typing based chatbot.

Figure-10. Evaluation result of chatbot text or grammatical mistake.

Figure-11. Evaluation result of chatbot text size.

Figure-12. Evaluation results of chatbot feasibility.

Figure 12a, b, c is depicting the combined comparative analysis of question 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24. These questions are related to feasibility, effectiveness, efficiency, satisfaction and reusability of chatbot. It has been observed that comparatively a smaller number of students were disagreed with the questions related to chatbot usage at other places and questions related to the talking to a real examination centre employee.

After performing user testing from different departments via evaluation form, it has been observed that the developed chatbot produced mostly positive results. All the students have opinion that this MUET examination chatbot is very useful and none of the students said that it’s a useless system.

6. CONCLUSIONS

This paper demonstrates the design and implementation of a chatbot system for examination department of Mehran University of engineering & technology Jamshoro.

This system is capable to provide response to several students queries in parallel. This helps both administrative authorities and students with their busy schedule get response immediately while communicating in both English and Urdu languages. Therefore, we have designed a web based chatbot that is capable to chat in both; Urdu and English. This chatbot can answer for queries in the textual as well as in voice form of user input. This study is not only concerned with the development of three interfaces of chatbot systems, but also covers their evaluation. After conducting a survey from the engineering students of MUET Jamshoro, it is revealed that students are more interested to use the English typing-based interface.

This chatbot can answer only MUET Examination department’s related queries, in future the chatbot will be able to response over all administration side related queries. In present chatbot the voice based chatbot support only English language in future Urdu voice based chatbot can be developed.

Funding: This study received no specific financial support.

Competing Interests: The authors declare that they have no competing interests.

Contributors/Acknowledgement: We thank the faculty and students at Mehran university engineering and technology Jamshoro who helped in the user survey.

REFERENCES

[1] B. Hettige and A. S. Karunananda, "First ainhala chatbot in action," in In Proceedings of the 3rd Annual Sessions of Sri Lanka Association for Artificial Intelligence (SLAAI), University of Moratuwa, 2006.

[2] K. Bala, M. Kumar, S. Hulawale, and S. Pandita, "Chat-bot for college management system using AI," International Research Journal of Engineering and Technology, vol. 4, pp. 2030-2033, 2017.

[3] T. D. Orin, "Implementation of a Bangla chatbot," Doctoral Dissertation, BRAC University, 2017.

[4] M. Pereira and L. Coheur, "Just. Chat-a platform for processing information to be used in chatbots. pp. 1-10. Available: https://fenix.tecnico.ulisboa.pt/downloadFile/395145485809/ExtendedAbstract.pdf," 2013.

[5] A. Patil, K. Marimuthu, and R. Niranchana, "Comparative study of cloud platforms to develop a chatbot," International Journal of Engineering & Technology, vol. 6, pp. 57-61, 2017. Available at: https://doi.org/10.14419/ijet.v6i3.7628.

[6] R. Kulkarni, A. Methwani, N. Pawar, C. Valecha, and S. Pooja, "College chat-bot," International Journal of Advanced Research in Computer Engineering & Technology, vol. 6, pp. 526-529, 2017.

[7] M. Kaleem, J. O'Shea, and K. Crockett, "Development of umair the urdu conversational agent for customer service," in In: World Congress on Engineering, IAENG 2, 2014.

[8] S. Hussain, "Computational linguistics (CL) in Pakistan: Issues and proposals," In EACL, Workshop on Computational Logistics for the Languages of South Asia Expanding Synergies with Europe, 2003.

[9] F. A. Mikic, J. C. Burguillo, M. Llamas, D. A. Rodríguez, and E. Rodríguez, "Charlie: An aiml-based chatterbot which works as an interface among ines and humans," in In IEEE Annual Conference 2009 EAEEIE, 2009, pp. 1-6.

[10] V. Bhargava and M. Nikhil, "An intelligent speech recognition system for education system. Available: https://pdfs.semanticscholar.org/8ea7/725bab4e390e4d8ab8eb ee747d2a5340c3b2.pdf," 2009.

[11] J. Jia, "CSIEC: A computer assisted English learning chatbot based on textual knowledge and reasoning," Knowledge-Based Systems, vol. 22, pp. 249-255, 2009. Available at: https://doi.org/10.1016/j.knosys.2008.09.001.

[12] M. D. Leonhardt, L. Tarouco, R. M. Vicari, E. R. Santos, and M. D. S. da Silva, "Using chatbots for network management training through problem-based oriented education " in In Seventh IEEE International Conference on Advanced Learning Technologies, ICALT, 2007, pp. 845-847.

[13] J. F. McKenzie, "Simulation of a careers advisor Using chabot technologies," Doctoral Dissertation, University of Leeds, School of Computing, 2004.

[14] S. V. Doshi, S. B. Pawar, A. G. Shelar, and S. S. Kulkarni, "Artificial intelligence chatbot in android system using open source program-O," International Journal of Advanced Research in Computer and Communication Engineering, vol. 6, pp. 816-821, 2017.

[15] G. Cameron, D. Cameron, G. Megaw, R. Bond, M. Mulvenna, S. O'Neill, and M. McTear, "Towards a chatbot for digital counselling," in In Proceedings of the 31st British Computer Society Human Computer Interaction Conference, BCS Learning & Development Ltd, 2017, p. 24.

[16] N. T. Thomas, "An e-business chatbot using AIML and LSA," in In Proceedings of IEEE International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2016, pp. 2740-2742.

[17] P. Kataria, K. Rode, A. Jain, P. Dwivedi, S. Bhingarkar, and M.-C. P. India, "User adaptive chatbot for mitigating depression," International Journal of Pure and Applied Mathematics, vol. 118, pp. 349-361, 2018.

[18] S. Hussain and G. Athula, "Extending a conventional chatbot knowledge base to external knowledge source and introducing user based sessions for diabetes education," in In IEEE 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), 2018, pp. 698-703.

[19] A. S. Lokman and J. M. Zain, "An architectural design of Virtual Dietitian (ViDi) for diabetic patients," in In 2nd IEEE International Conference on Computer Science and Information Technology, 2009, pp. 408-411.

[20] H. Kazi, B. Chowdhry, and Z. Memon, "Medchatbot: An umls based chatbot for medical students," International Journal of Computer Applications, vol. 55, pp. 1-5, 2012. Available at: https://doi.org/10.5120/8844-2886.

[21] M. S. Satu and M. H. Parvez, "Review of integrated applications with aiml based chatbot," in In IEEE International Conference on Computer and Information Engineering (ICCIE), 2015, pp. 87-90.

Views and opinions expressed in this article are the views and opinions of the author(s), Review of Information Engineering and Applications shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content.

Footnotes:

1. https://.marutitech.com/benefit-chatbot.

2. https://blog.appliedai.com/chatbot-applications/.

Index