Logo Logo
Help
Contact
Switch Language to German
Dufter, Philipp; Schütze, Hinrich (3. November 2019): Analytical Methods for Interpretable Ultradense Word Embeddings. UNSPECIFIED, November 3–7, 2019, Hong Kong, China
[img]
Preview
Creative Commons Attribution 423kB

Abstract

Word embeddings are useful for a wide vari- ety of tasks, but they lack interpretability. By rotating word spaces, interpretable dimensions can be identified while preserving the informa- tion contained in the embeddings without any loss. In this work, we investigate three meth- ods for making word spaces interpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and DensRay, a new method we pro- pose. In contrast to Densifier, DensRay can be computed in closed form, is hyperparameter- free and thus more robust than Densifier. We evaluate the three methods on lexicon induc- tion and set-based word analogy. In addition we provide qualitative insights as to how inter- pretable word spaces can be used for removing gender bias from embeddings.