Abstract:

Positive Behavioral Interventions and Supports (PBIS) is a framework helping schools define and teach shared behavioral expectations. Implementation varies widely as school teams translate these expectations into behavior matrices and routines, making it difficult to design a single assessment tool that is both psychometrically robust and flexible across contexts. Automatic Item Generation (AIG) and generative AI can address this challenge by creating questionnaires anchored to a common construct model but adapted to local needs. This study examines a methodological shift from traditional scale development to the use of generative AI for item construction and optimization in PBIS. PBIS-RISE (Positive Behavioral Interventions and Supports – Reflecting on Interactions, Standards & Emotions) is an AI-generated digital scale for behavioral monitoring, grounded in three PBIS-based factors (Be Respectful, Be Responsible, Be Safe/Be Orderly). For each factor, items tap seven underlying dimensions: rule knowledge, school practice, generalization to external contexts, personal values and motivations, adherence to heteronomous norms, resistance to negative peer modelling, and emotional reactions to one’s own and others’ behavior. Each indicator is assessed through paired questions on normative evaluations (How appropriate is this behavior?) and self-reported practices (What do you usually do?), distinguishing behavioral knowledge, intentions and enacted conduct. The innovative contribution of PBIS-RISE lies not in online delivery, but in the way a validated factorial model is converted into a generative template. After confirmation of the factor structure and lower-order dimensions, each dimension is translated into a semantic prompt that guides generative AI to create new questionnaires for additional behavioral rules in the PBIS framework while preserving alignment with the original construct model. The validated questionnaire thus becomes a reusable “prompt schema” enabling scalable, theory-consistent assessment across multiple expectations and settings. PBIS-RISE will first be validated to confirm the hypothesized factorial structure and lower-order dimensions. Approximately 400 students aged 9–13 years from public primary and lower-secondary schools will complete the questionnaire online in classroom sessions during the 2025/26 school year. Planned psychometric analyses include confirmatory factor analysis, estimation of internal consistency (Cronbach’s alpha, McDonald’s omega), and item-level analyses of item functioning. Where sample size allows, measurement invariance will be examined across gender and school level, and test–retest reliability over a 4-week interval will be assessed in a subsample. After validation, theoretical specifications and empirical findings from the traditional scale will be used to instruct a generative AI model to produce controlled item variants for new PBIS rules. By imposing constraints on content, clarity, reading level, polarity and alignment with behavioral indicators, the model will generate a repertoire of items to be re-evaluated qualitatively and psychometrically on a comparable sample. Finally, the contribute will clarify whether AI-generated items can maintain rigorous psychometric standards while enabling more adaptive, context-sensitive assessment tools that better support PBIS teams in monitoring student behavior and guiding data-based decision making.

Keywords:

Automatic Item Generation, Positive Behaviour Intervention and Support, Digital assessment, Context-sensitive assessment, PBIS.