Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology

Hofmann, Valentin; Pierrehumbert, Janet und Schütze, Hinrich (Juli 2022): Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology. 39 th International Conference on Machine Learning, Baltimore, Maryland, USA, July 2022. Chaudhuri, Kamalika; Jegelka, Stefanie; Song, Le; Szepesvari, Csaba; Niu, Gang und Sabato, Sivan (Hrsg.): In: Proceedings of the 39 th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 202, Bd. 162 S. 8796-8810

Abstract

We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.

• Dokumententyp: Konferenzbeitrag (Paper)
• EU Funded Grant Agreement Number: 740516
• EU-Projekte: Horizon 2020 > ERC Grants > ERC Advanced Grant > ERC Grant 740516: NonSequeToR - Non-sequence models for tokenization replacement
• Fakultätsübergreifende Einrichtungen: Centrum für Informations- und Sprachverarbeitung (CIS)
• Themengebiete: 400 Sprache > 400 Sprache ; 400 Sprache > 410 Linguistik
• Sprache: Englisch