Measurement equivalence in probability and nonprobability online panels

Einarsson, Hafsteinn; Sakshaug, Joseph W.; Cernat, Alexandru; Cornesse, Carina und Blom, Annelies G. (2022): Measurement equivalence in probability and nonprobability online panels. In: International Journal of Market Research, Bd. 64, Nr. 4: S. 484-505

Abstract

Nonprobability online panels are commonly used in the social sciences as a fast and inexpensive way of collecting data in contrast to more expensive probability-based panels. Given their ubiquitous use in social science research, a great deal of research is being undertaken to assess the properties of nonprobability panels relative to probability ones. Much of this research focuses on selection bias, however, there is considerably less research assessing the comparability (or equivalence) of measurements collected from respondents in nonprobability and probability panels. This article contributes to addressing this research gap by testing whether measurement equivalence holds between multiple probability and nonprobability online panels in Australia and Germany. Using equivalence testing in the Confirmatory Factor Analysis framework, we assessed measurement equivalence in six multi-item scales (three in each country). We found significant measurement differences between probability and nonprobability panels and within them, even after weighting by demographic variables. These results suggest that combining or comparing multi-item scale data from different sources should be done with caution. We conclude with a discussion of the possible causes of these findings, their implications for survey research, and some guidance for data users.

• Dokumententyp: Zeitschriftenartikel
• Fakultät: Mathematik, Informatik und Statistik > Statistik
• Themengebiete: 500 Naturwissenschaften und Mathematik > 510 Mathematik
• ISSN: 1470-7853
• Sprache: Englisch
• DFG: Gefördert durch die Deutsche Forschungsgemeinschaft (DFG) - 139943784