Abstract:

This paper describes the development of a support system for improving teaching skills through the use of adaptive questions in a course evaluation questionnaire. Faculty development (FD) is mandatory at Japanese universities and is defined as a set of activities for systematically improving teaching skills.

Course evaluation questionnaires surveys have been conducted at nearly 99.3% of all national universities in Japan. The questionnaires are designed to collect information on students’ understanding and satisfaction with classes and are large-scale and easy to use. However, it may be difficult to apply the information collected from questionnaires specifically to the improvement of teaching skills, for example, because teachers receive only the overall results of course evaluation surveys and not the item-specific results.

To solve this problem, we proposed a hierarchical item bank for predicting responses to many different question items based on previously recorded responses to only a few question items in order to identify and characterize relationships among course evaluation items using a Bayesian network. This method should predict responses to abstract questions and help improve specific teaching methods. However, the question items for the hierarchical item bank have not yet been examined. We believe that using adaptive questions based on the hierarchical item bank in course evaluation questionnaires could be an efficient means for improving teaching skills.

Therefore, we proposed a new hierarchical item bank capable of adaptively selecting question items. There are many patterns for selecting question items from the item bank, meaning that selection would be computationally expensive. In this study, we focused on a hierarchical item bank with a non-circular tree structure. Our novel technique uses subgraphs to search for nodes and obtain high-value information.

This method divides the item bank’s structure into highly specific areas and makes calculations for item selection accordingly. We conducted simulation tests using a course evaluation questionnaire and showed that the proposed method significantly reduces computational costs. Thus, this method efficiently calculates and selects appropriate adaptive question items. In addition, the amount of information was weighted using text from the course syllabus. This process led to the selection of question items that were very similar to the lesson content and was able to handle a large amount of information.

We also developed a support system for improving teaching skills by using the proposed method to adaptively select question items. The system predicts responses based on the hierarchical item bank. The teacher can view the distributions of predicted responses as well as the actual responses to question items, such as those associated with teaching behavior, and make improvements to their teaching style accordingly. The system also updates the item bank’s network parameters and question items to predict responses that better match the class and can also be used to review past lessons. We conducted an evaluation experiment with teachers and revealed that the system contributed to increasing teachers’ motivation to improve their teaching skills. In addition, the system can help teachers understand the strengths of teaching style and areas that can be improved.

Keywords:

Course Evaluation, Development of System, Bayesian Network.