Colorimetric Sensor Reading and Illumination Correction via Multi-Task Deep-Learning

Castelblanco, Alejandra; Matzeu, Giusy; Ruggeri, Elisabetta; Omenetto, Fiorenzo G.; Hilgendorff, Anne ; Schnabel, Julia A. und Schubert, Benjamin (2023): Colorimetric Sensor Reading and Illumination Correction via Multi-Task Deep-Learning. 45th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society (EMBC), Sydney, Australia, 24. -27. Juli 2023. In: 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Piscataway: IEEE.

Abstract

Colorimetric sensors represent an accessible and sensitive nanotechnology for rapid and accessible measurement of a substance’s properties (e.g., analyte concentration) via color changes. Although colorimetric sensors are widely used in healthcare and laboratories, interpretation of their output is performed either by visual inspection or using cameras in highly controlled illumination set-ups, limiting their usage in end-user applications, with lower resolutions and altered light conditions. For that purpose, we implement a set of image processing and deep-learning (DL) methods that correct for non-uniform illumination alterations and accurately read the target variable from the color response of the sensor. Methods that perform both tasks independently vs. jointly in a multi-task model are evaluated. Video recordings of colorimetric sensors measuring temperature conditions were collected to build an experimental reference dataset. Sensor images were augmented with non-uniform color alterations. The best-performing DL architecture disentangles the luminance, chrominance, and noise via separate decoders and integrates a regression task in the latent space to predict the sensor readings, achieving a mean squared error (MSE) performance of 0.811±0.074[°C] and r2=0.930±0.007, under strong color perturbations, resulting in an improvement of 1.26[°C] when compared to the MSE of the best performing method with independent denoising and regression tasks.Clinical Relevance— The proposed methodology aims to improve the accuracy of colorimetric sensor reading and their large-scale accessibility as point-of-care diagnostic and continuous health monitoring devices, in altered illumination conditions.

• Item Type: Conference or Workshop Item (Paper)
• Faculties: Medicine > Medical Center of the University of Munich > Pediatric Clinic and Outpatient Clinic in the Dr. von Hauner Children‘s Hospital
• Subjects: 000 Computer science, information and general works > 004 Data processing computer science; 500 Science > 530 Physics; 500 Science > 540 Chemistry; 600 Technology > 600 Technology; 600 Technology > 610 Medicine and health
• ISBN: 979-8-3503-2447-1
• Place of Publication: Piscataway
• Language: English