Scenario‐Based AI Literacy Scale (SAILS): Evidence for distinct instrumental and critical‐reflective AI skills and their difference from traditional digital skills

Scheibenzuber, Christian ; Chernikova, Olga ; Vejvoda, Johanna ; Sailer, Michael und Fischer, Frank (2026): Scenario‐Based AI Literacy Scale (SAILS): Evidence for distinct instrumental and critical‐reflective AI skills and their difference from traditional digital skills. In: British Journal of Educational Technology [Forthcoming]

Abstract

Given the increasing integration of Artificial Intelligence (AI) into everyday life and professional contexts, it is essential to investigate learners' existing capabilities regarding AI tools to inform possible interventions to equip them with necessary AI skills, but also advance the theoretical frameworks on digital skills measurement and development. In this vein, this study aims to validate a Scenario-Based AI Literacy Scale (SAILS) tailored to vocational learners. In this study, we differentiate between instrumental (e.g., using AI to prepare a presentation) and critical-reflective AI skills (e.g., recognizing AI-generated deepfake content). The scale is operationalized through a scenario-based self-assessment approach, ensuring a context-driven evaluation of AI skills. We validated the AI skills scale consisting of 12 scenarios (5 instrumental and 7 critical-reflective) with a sample of police officers in training (N = 420), investigating reliability and validity of the scale. Additionally, we compared SAILS with a traditional digital skills scale to investigate convergent and discriminant validity. The analysis resulted in excellent reliability of the subscales (Cronbach's α = 0.88 for critical-reflective skills and 0.89 for instrumental skills) and the entire scale as a whole (Cronbach's α = 0.92). A 3-factor-model (including critical-reflective and instrumental AI skills subscales as well as digital skills) shows an acceptable model fit (RMSEA = 0.06, TLI = 0.93) with the standardized factor loadings ranging from 0.56 to 0.81 indicating an acceptable construct validity. AI skills were not only found to be related to more general digital skills, but also exhibiting some unique features, emphasizing that AI use requires skills which were not yet covered by digital literacy. These results provide support for an easy-to-use template, to be used for additional research in different contexts where objective performance measurements cannot be used and with a broader array of learners.

• Item Type: Journal article
• Faculties: Psychology and Education Science > Department Psychology > Education and Educational Psychology
• Subjects: 100 Philosophy and Psychology > 150 Psychology
• ISSN: 0007-1013
• Language: English
• DFG: Gefördert durch die Deutsche Forschungsgemeinschaft (DFG) - 542251251