ANSWER ESTIMATION SYSTEM USING RANDOM QUESTION OF CLASS EVALUATION ITEMS BASED ON A HIERARCHICAL ITEM BANK

This paper describes the development of the lesson improvement support system using random question in class evaluation questionnaire. Currently, faculty development (FD) is mandatory in Japanese universities and is defined as systematic teaching improvement activities. To consistently achieve high-quality education, various FD activities have been implemented. Class evaluation questionnaires completed by students are a typical means of FD that nearly 98% of national universities in Japan have implemented. A class evaluation questionnaire is easy to implement in one of the final classes of a class. Another advantage of class evaluation questionnaires is that a large amount of data can be collected on students’ opinions and thoughts.

On the other hand, it has been reported that the results of many class evaluation questionnaires are difficult to use for FD. One reason for this is that the information provided to teachers consists of aggregate results for a small number of questions. Teachers cannot use aggregate results alone to improve their teaching. Further, the questions are abstract, so it is difficult to determine specific areas for improvements and specific improvement methods. One way to solve this problem is to ask many questions. However, a large number of questions increases the burden on the students.

Thus, a new methodology is needed for implementing class evaluation questionnaires used in FD. Accordingly, a lesson improvement support system was developed using a hierarchical item bank that can estimate responses to questions and suggest teaching methods.

The hierarchical item bank is a model that constructs relations among questions (items) based on a Bayesian network.

It is possible to estimate the responses to many items from the responses to a few. This feature can random question for students. Hence, all students answer all differently items (questionnaire). Random questions can help teacher get the accurate answers for all of the items estimated in hierarchical item bank.

First, a methodology is considered for providing improvement methods using estimation based on the hierarchical item bank. Items in the hierarchical item bank vary from abstract to specific. A specific item is a question directly related to teaching behavior (verbal behavior and non-verbal behavior related to teaching methods). In the hierarchy of the item bank, there are parent nodes (abstract items) and child nodes (specific items). By this hierarchical relationship, specific teaching methods are identified for improvement.

Next, the lesson improvement support system is explained. The system provides the teacher with estimation results based on the hierarchical item bank. The teacher can view the results as distributions of responses to specific items. For specific items associated with teaching behavior, teacher can obtain suggestions for improvements in the class.

This enables lesson improvement support using the hierarchical item bank. Future topics for investigation include experiments for evaluating the system.