Abstract:

The use of chatbots as an online survey tool is becoming increasingly popular owing to their communicative affordances. A chatbot survey or "conversational survey" is an online survey presented in the form of a conversation. Instead of answering static questions, respondents are invited to have a conversation with a chatbot character. Emotional intelligence (EI) is foundational to students’ success in the university. Interestingly, chatbot’s application of measuring emotional intelligence has rarely been studied. Therefore, we aimed to design a chatbot to measure the emotional intelligence of teachers and students in a higher education environment and verify its effectiveness. To achieve our research objectives, we designed the chatbot using Landbot. Landbot is a code-free chatbot development platform that allows users to create conversation experiences using a simple drag-and-drop interface. In the process of designing, we applied humanization technology and develop a situational judgment test (SJT) embedded in the chatbot to improve the hedonic experience of users while measuring their emotional intelligence. The situational judgment test consists of ten scenarios for the users and presents them with four corresponding choices. The ten situations mainly evaluate the user's ability to be aware of the emotion, evaluate emotion, regulate emotion and express emotion. We used the 14 items Wong and Law Emotional Intelligence Scale (WLEIS) to compare its validity. Our experimental study with 30 teachers and students from a university in Hubei China showed that the overall correlation between the two measures was 0.68 (p < 0.001). However, after converting the SJI scale to a 7-point likert scale, the average emotional intelligence score for SJI chatbot is significantly lower than the WLEIS score (∆M=1.92,p<0.001). This study demonstrates that while it is feasible to measure emotional intelligence by a chatbot using the situational judgment test approach, the response level to emotional intelligent is not the same. Nevertheless, our findings support the development of the conversational approach to measuring emotional intelligence and making the process more interesting and natural. This exploratory study brings implications and insights to future research in deploying SJT in chatbot for assessing emotional intelligence of individual as well as using chatbots to cultivate and improve users’ emotional intelligence.

Keywords:

Chatbot, emotional intelligence, measure, humanization, situational judgment test.