Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load

Abdelrahman, Yomna; Velloso, Eduardo; Dingler, Tilman; Schmidt, Albrecht and Vetere, Frank (September 2017): Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3: 33:1-33:20

Abstract

Current digital systems are largely blind to users’ cognitive states. Systems that adapt to users’ states show great potential for augmenting cognition and for creating novel user experiences. However, most approaches for sensing cognitive states, and cognitive load specifically, involve obtrusive technologies, such as physiological sensors attached to users’ bodies. This paper present an unobtrusive indicator of the users’ cognitive load based on thermal imaging that is applicable in real-world. We use a commercial thermal camera to monitor a person’s forehead and nose temperature changes to estimate their cognitive load. To assess the effect of different levels of cognitive load on facial temperature we conducted a user study with 12 participants. The study showed that different levels of the Stroop test and the complexity of reading texts affect facial temperature patterns, thereby giving a measure of cognitive load. To validate the feasibility for real-time assessments of cognitive load, we conducted a second study with 24 participants, we analyzed the temporal latency of temperature changes. Our system detected temperature changes with an average latency of 0.7 seconds after users were exposed to a stimulus, outperforming latency in related work that used other thermal imaging techniques. We provide empirical evidence showing how to unobtrusively detect changes in cognitive load in real-time. Our exploration of exposing users to different content types gives rise to thermal-based activity tracking, which facilitates new applications in the field of cognition-aware computing.

• Item Type: Journal article
• EU Funded Grant Agreement Number: 683008
• EU Projects: Horizon 2020 > ERC Grants > ERC Consolidator Grant > ERC Grant 683008: AMPLIFY - Amplifying Human Perception Through Interactive Digital Technologies
• Form of publication: Submitted Version
• Keywords: Thermal Imaging, Thermal latency, cognitive load, Human Computer Interaction
• Faculties: Mathematics, Computer Science and Statistics > Computer Science
• Subjects: 000 Computer science, information and general works > 004 Data processing computer science
• ISSN: 2474-9567
• Place of Publication: New York, NY, USA
• Language: English