Center for Information and Language Processing (CIS)

2022

Hofmann, Valentin; Schütze, Hinrich; Pierrehumbert, Janet B. (May 2022): An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers. 60th Annual Meeting of the Association for Computational Linguistics, May 22-27, 2022, Dublin, Ireland. [PDF, 480kB]

Imani, Ayyoob; Senel, Lütfi Kerem; Sabet, Masoud Jalili; Yvon, François; Schütze, Hinrich (May 2022): Graph Neural Networks for Multiparallel Word Alignment. ACL 2022, May 2022, Dublin, Ireland. [PDF, 749kB]

Leiß, Hans ORCID: 0000-0002-4162-2258 (2022): An algebraic representation of the fixed-point closure of *-continuous Kleene algebras – A categorical Chomsky–Schützenberger theorem. In: Mathematical Structures in Computer Science, Vol. 32, No. Special issue 6: pp. 685-728 [PDF, 779kB]

Liang, Sheng; Zhao, Mengjie; Schütze, Hinrich (May 2022): Modular and Parameter-Efficient Multimodal Fusion with Prompting. ACL 2022, May 2022, Dublin, Ireland. [PDF, 858kB]

Liu, Yihong; Jabbar, Haris; Schütze, Hinrich (May 2022): Flow-Adapter Architecture for Unsupervised Machine Translation. 60th Annual Meeting of the Association for Computational Linguistics, May 2022, Dublin, Ireland. [PDF, 618kB]

Maronikolakis, Antonis; Wisiorek, Axel; Nann, Leah; Jabbar, Haris; Udupa, Sahana ORCID: 0000-0003-3647-9570; Schütze, Hinrich (May 2022): Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments. ACL 2022, May 2022, Dublin, Ireland. [PDF, 629kB]

Senel, Lütfi Kerem; Schick, Timo; Schütze, Hinrich (May 2022): CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment. 60th Annual Meeting of the Association for Computational Linguistics, May 2022, Dublin, Ireland. [PDF, 290kB]

Udupa, Sahana ORCID: 0000-0003-3647-9570; Maronikolakis, Antonis; Schütze, Hinrich; Wisiorek, Axel (9. June 2022): Ethical Scaling for Content Moderation: Extreme Speech and the (In)Significance of Artificial Intelligence. Discussion Papers / Shorenstein Center on Media, Politics and Public Policy at Harvard Kennedy School,

Weissweiler, Leonie; Hofmann, Valentin; Sabet, Masoud Jalili; Schütze, Hinrich (May 2022): CaMEL: Case Marker Extraction without Labels. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), May 2022, Dublin, Ireland. [PDF, 380kB]

2021

Dufter, Philipp; Kassner, Nora; Schütze, Hinrich (June 2021): Static Embeddings as Efficient Knowledge Bases? 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 6-11, 2021, Online. [PDF, 340kB]

Hofmann, Valentin; Pierrehumbert, Janet B.; Schütze, Hinrich (August 2021): Dynamic Contextualized Word Embeddings. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, August 2021, Online. [PDF, 4MB]

Hofmann, Valentin; Pierrehumbert, Janet B.; Schütze, Hinrich (August 2021): Superbizarre Is Not Superb: Derivational Morphology Improves BERT’s Interpretation of Complex Words. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, August 2021, Online. [PDF, 1MB]

Imani, Ayyoob; Sabet, Masoud Jalili; Senel, Lütfi Kerem; Dufter, Philipp; Yvon, François; Schütze, Hinrich (November 2021): Graph Algorithms for Multiparallel Word Alignment. 2021 Conference on Empirical Methods in Natural Language Processing, November 7-11, 2021, Online and Punta Cana, Dominican Republic. [PDF, 618kB]

Kassner, Nora; Dufter, Philipp; Schütze, Hinrich (April 2021): Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models. 16th Conference of the European Chapter of the Association for Computational Linguistics, April 2021, Online. [PDF, 2MB]

Kassner, Nora; Tafjord, Oyvind; Schütze, Hinrich; Clark, Peter (November 2021): BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief. 2021 Conference on Empirical Methods in Natural Language Processing, November 7-11, 2021, Online and Punta Cana, Dominican Republic. [PDF, 466kB]

Maronikolakis, Antonis; Dufter, Philipp; Schütze, Hinrich (November 2021): Wine is Not v i n. On the Compatibility of Tokenizations Across Languages. EMNLP 2021, November 7–11, 2021, Punta Cana, Dominican Republic. [PDF, 3MB]

Marz, Luisa; Schweter, Stefan; Poerner, Nina; Roth, Benjamin; Schütze, Hinrich (2. September 2021): Data Centric Domain Adaptation for Historical Text with OCR Errors. International Conference on Document Analysis and Recognition, September 2021, Lausanne, Switzerland. [PDF, 364kB]

Schick, Timo; Schütze, Hinrich (April 2021): Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference. 16th Conference of the European Chapter of the Association for Computational Linguistics, April 19-23, 2021, Online. [PDF, 493kB]

Schick, Timo; Schütze, Hinrich (November 2021): Few-Shot Text Generation with Natural Language Instructions. 2021 Conference on Empirical Methods in Natural Language Processing, November 7–11, 2021, Online and Punta Cana, Dominican Republic. [PDF, 504kB]

Schick, Timo; Schütze, Hinrich (November 2021): Generating Datasets with Pretrained Language Models. 2021 Conference on Empirical Methods in Natural Language Processing, November 7-11, 2021, Online and Punta Cana, Dominican Republic. [PDF, 431kB]

Schick, Timo; Schütze, Hinrich (June 2021): It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2021, Online. [PDF, 475kB]

Schick, Timo; Udupa, Sahana ORCID: 0000-0003-3647-9570; Schütze, Hinrich (17. December 2021): Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP. In: Transactions of the Association for Computational Linguistics, Vol. 9: pp. 1408-1424 [PDF, 411kB]

Senel, Lutfi Kerem; Schütze, Hinrich (April 2021): Does He Wink or Does He Nod? A Challenging Benchmark for Evaluating Word Understanding of Language Models. 16th Conference of the European Chapter of the Association for Computational Linguistics, April 19-23, 2021, Online. [PDF, 239kB]

Udupa, Sahana ORCID: 0000-0003-3647-9570; Hickok, Elonnai; Maronikolakis, Antonis; Schuetze, Hinrich; Csuka, Laura; Wisiorek, Axel; Nann, Leah (June 2021): Artificial Intelligence, Extreme Speech and the Challenges of Online Content Moderation. München Ludwig-Maximilians-Universität München [PDF, 17MB]

Zhao, Mengjie; Schütze, Hinrich (November 2021): Discrete and Soft Prompting for Multilingual Models. 2021 Conference on Empirical Methods in Natural Language Processing, November 2021, Online and Punta Cana, Dominican Republic. [PDF, 383kB]

Zhao, Mengjie; Zhu, Yi; Shareghi, Ehsan; Vulic, Ivan; Reichart, Roi; Korhonen, Anna; Schütze, Hinrich (August 2021): A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters. 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, August 2021, Online. [PDF, 737kB]

2020

Dufter, Philipp; Schmitt, Martin; Schütze, Hinrich (December 2020): Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted Attention. The 28th International Conference on Computational Linguistics, 8. - 13. Dezember 2020, Online. [PDF, 395kB]

Dufter, Philipp; Schütze, Hinrich (November 2020): Identifying Elements Essential for BERT’s Multilinguality. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 16th – 20th November 2020, Online. [PDF, 863kB]

Dufter, Philipp; Schütze, Hinrich (1. May 2020): Identifying Necessary Elements for BERT’s Multilinguality. [PDF, 631kB]

Hofmann, Valentin; Pierrehumbert, Janet; Schütze, Hinrich (November 2020): DagoBERT: Generating Derivational Morphology with a Pretrained Language Model. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 16. – 20. November 2020, Online. [PDF, 2MB]

Hofmann, Valentin; Pierrehumbert, Janet; Schütze, Hinrich (6. July 2020): Predicting the Growth of Morphological Families from Social and Linguistic Factors. UNSPECIFIED, July 6 – 8, 2020, Seattle, USA [PDF, 2MB]

Hofmann, Valentin; Schütze, Hinrich; Pierrehumbert, Janet (6. July 2020): A Graph Auto-encoder Model of Derivational Morphology. UNSPECIFIED, July 6 – 8, 2020, Seattle, USA [PDF, 1MB]

Kassner, Nora; Schütze, Hinrich (6. July 2020): Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly. The 58th Annual Meeting of the Association for Computational Linguistic, July 6 – 8, 2020, Seattle, USA. [PDF, 251kB]

Liang, Sheng; Dufter, Philipp; Schütze, Hinrich (December 2020): Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations. The 28th International Conference on Computational Linguistics, 8. - 13. Dezember 2020, Online. [PDF, 850kB]

McClelland, James; Hill, Felix; Rudolph, Maja; Baldridge, Jason; Schütze, Hinrich (28. September 2020): Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. In: Proceedings of the National Academy of Sciences, No. 42: pp. 25966-25974 [PDF, 818kB]

Poerner, Nina; Waltinger, Ulli; Schütze, Hinrich (6. July 2020): Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity. UNSPECIFIED, July 6 – 8, 2020, Seattle, USA [PDF, 386kB]

Sabet, Masoud Jalili; Dufter, Philipp; Schütze, Hinrich (27. April 2020): SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings. In: Findings of ACL: EMNLP 2020 [PDF, 1MB]

Schick, Timo; Schmid, Helmut; Schütze, Hinrich (December 2020): Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification. The 28th International Conference on Computational Linguistics, 8. - 11. Dezember 2020, Online. [PDF, 275kB]

Schick, Timo; Schütze, Hinrich (6. July 2020): BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance. The 58th Annual Meeting of the Association for Computational Linguistics, July 6 – 8, 2020, Seattle, USA. [PDF, 399kB]

Severini, Silvia; Hangya, Viktor; Fraser, Alexander; Schütze, Hinrich (December 2020): Combining Word Embeddings with Bilingual Orthography Embeddings for Bilingual Dictionary Induction. The 28th International Conference on Computational Linguistics, 8. - 13. Dezember 2020, Online. [PDF, 235kB]

Zhao, Mengjie; Dufter, Philipp; Yaghoobzadeh, Yadollah; Schütze, Hinrich (2020): Quantifying the Contextualization of Word Representations with Semantic Class Probing. In: Findings of ACL: EMNLP 2020 [PDF, 495kB]

Zhao, Mengjie; Lin, Tao; Mi, Fei; Jaggi, Martin; Schütze, Hinrich (November 2020): Masking as an Efficient Alternative to Finetuning for Pretrained Language Models. The 2020 Conference on Empirical Methods in Natural Language Processing, 6. – 20. November 2020, Online. [PDF, 807kB]

2019

Adel, Heike; Schütze, Hinrich (September 2019): Type-aware Convolutional Neural Networks for Slot Filling. In: Journal of Artificial Intelligence Research, Vol. 66: pp. 297-339 [PDF, 496kB]

Barrault, Loic; Bojar, Ondrej; Costa-jussa, Marta R.; Federmann, Christian; Fishel, Mark; Graham, Yvette; Haddow, Barry; Huck, Matthias; Koehn, Philipp; Malmasi, Shervin; Monz, Christof; Muller, Mathias; Pal, Santanu; Post, Matt; Zampieri, Marcos (2019): Findings of the 2019 Conference on Machine Translation (WMT19). In: Fourth Conference on Machine Translation (Wmt 2019): pp. 1-61

Dufter, Philipp; Schütze, Hinrich (3. November 2019): Analytical Methods for Interpretable Ultradense Word Embeddings. UNSPECIFIED, November 3–7, 2019, Hong Kong, China [PDF, 423kB]

Evert, Stefan; Heinrich, Philipp; Henselmann, Klaus; Rabenstein, Ulrich; Scherr, Elisabeth; Schmitt, Martin; Schröder, Lutz (2019): Combining Machine Learning and Semantic Features in the Classification of Corporate Disclosures. In: Journal of Logic Language and Information, Vol. 28, No. 2: pp. 309-330

Gerdjikov, Stefan; Mihov, Stoyan; Schulz, Klaus U. (2019): Space-efficient bimachine construction based on the equalizer accumulation principle. In: Theoretical Computer Science, Vol. 790: pp. 80-95

Hangya, Viktor; Fraser, Alexander (2019): Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 1224-1234

Huck, Matthias; Hangya, Viktor; Fraser, Alexander (2019): Better OOV Translation with Bilingual Terminology Mining. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 5809-5815

Kayabas, Ayla; Schmid, Helmut; Topcu, Ahmet E.; Kilic, Ozkan (2019): TRMOR: a finite-state-based morphological analyzer for Turkish. In: Turkish Journal of Electrical Engineering and Computer Sciences, Vol. 27, No. 5: pp. 3837-3851

Kemos, Apostolos; Adel, Heike; Schütze, Hinrich (June 2019): Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging. UNSPECIFIED, 2. - 7. June 2019, Minneapolis, USA. [PDF, 369kB]

McClelland, James; Hill, Felix; Rudolph, Maja; Baldridge, Jason; Schütze, Hinrich (12. December 2019): Extending Machine Language Models toward Human-Level Language Understanding. [PDF, 897kB]

Reul, Christian; Christ, Dennis; Hartelt, Alexander; Balbach, Nico; Wehner, Maximilian; Springmann, Uwe; Wick, Christoph; Grundig, Christine; Buettner, Andreas; Puppe, Frank (2019): OCR4all-An Open-Source Tool Providing a (Semi-)Automatic OCR Workflow for Historical Printings. In: Applied Sciences-Basel, Vol. 9, No. 22, 4853

Schick, Timo; Schuetze, Hinrich (2019): Learning Semantic Representations for Novel Words: Leveraging Both Form and Context. In: Thirty-Third Aaai Conference on Artificial Intelligence / Thirty-First Innovative Applications of Artificial Intelligence Conference / Ninth Aaai Symposium on Educational Advances in Artificial Intelligence: pp. 6965-6973

Schick, Timo; Schütze, Hinrich (June 2019): Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts. 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2. - 7. June 2019, Minneapolis, USA. [PDF, 304kB]

Schick, Timo; Schütze, Hinrich (January 2019): Learning Semantic Representations for Novel Words: Leveraging Both Form and Context. Thirty-Third AAAI Conference on Artificial Intelligence; AAAI-2019, 27. January – 01. February 2019, Honolulu, Hawaii, USA. [PDF, 165kB]

Schick, Timo; Schütze, Hinrich (April 2019): Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking. The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), February 7-12, 2020, New York, USA. [PDF, 372kB]

Schmitt, Martin; Schuetze, Hinrich (2019): SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 902-914

Sedinkina, Marina; Breitkopf, Nikolas; Schuetze, Hinrich (2019): Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 346-359

Sedinkina, Marina; Breitkopf, Nikolas; Schütze, Hinrich (28. July 2019): Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes. UNSPECIFIED, July 28 - August 2, 2019, Florence, Italy [PDF, 301kB]

Stojanovski, Dario; Fraser, Alexander (2019): Combining Local and Document-Level Context: The LMU Munich Neural Machine Translation System at WMT19. In: Fourth Conference on Machine Translation (Wmt 2019): pp. 400-406

Stojanovski, Dario; Hangya, Viktor; Huck, Matthias; Fraser, Alexander (2019): The LMU Munich Unsupervised Machine Translation System for WMT19. In: Fourth Conference on Machine Translation (Wmt 2019): pp. 393-399

Sydorova, Alona; Poerner, Nina; Roth, Benjamin (2019): Interpretable Question Answering on Knowledge Bases and Text. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 4943-4951

Yaghoobzadeh, Yadollah; Kann, Katharina; Hazen, Timothy; Agirre, Eneko; Schütze, Hinrich (28. July 2019): Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings. UNSPECIFIED, July 28 - August 2, 2019, Florence, Italy [PDF, 690kB]

Yaghoobzadeh, Yadollah; Kann, Katharina; Hazen, Timothy J.; Agirre, Eneko; Schuetze, Hinrich (2019): Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 5740-5753

Zhao, Mengjie; Schuetze, Hinrich (2019): A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction. In: 57Th Annual Meeting of the Association for Computational Linguistics (Acl 2019): pp. 3506-3517

Zhao, Mengjie; Schütze, Hinrich (28. July 2019): A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction. UNSPECIFIED, July 28 - August 2, 2019, Florence, Italy [PDF, 544kB]

2018

Cornolti, Marco; Ferragina, Paolo; Ciaramita, Massimiliano; Rüd, Stefan; Schütze, Hinrich (December 2018): SMAPH: A Piggyback Approach for Entity-Linking in Web Queries. In: ACM Transactions on Information Systems, Vol. 37, No. 1, 13 [PDF, 1MB]

Dufter, Philipp; Schütze, Hinrich (November 2018): A Stronger Baseline for Multilingual Word Embeddings. [PDF, 173kB]

Dufter, Philipp; Zhao, Mengjie; Schmitt, Martin; Fraser, Alexander; Schütze, Hinrich (July 2018): Embedding Learning Through Multilingual Concept Induction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), July 15-20, 2018, Melbourne, Australia. [PDF, 453kB]

Kann, Katharina; Schütze, Hinrich (October 2018): Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting. 2018 Conference on Empirical Methods in Natural Language Processing, 31. October - 04. November 2018, Brussels, Belgium. [PDF, 879kB]

Poerner, Nina; Roth, Benjamin; Schütze, Hinrich (July 2018): Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. The 56th Annual Meeting of the Association for Computational Linguistics, 15. - 20. July 2018, Melbourne, Australia. [PDF, 669kB]

Poerner, Nina; Sabet, Masoud Jalili; Roth, Benjamin; Schütze, Hinrich (October 2018): Aligning Very Small Parallel Corpora Using Cross-Lingual Word Embeddings and a Monogamy Objective. [PDF, 409kB]

Roth, Benjamin; Conforti, Costanza; Poerner, Nina; Schütze, Hinrich; Kumar Karn, Sanjeev (7. December 2018): Neural architectures for open-type relation argument extraction. In: Natural Language Engineering, Vol. 25, No. 2: pp. 219-238 [PDF, 611kB]

Schmitt, Martin; Steinheber, Simon; Schreiber, Konrad (October 2018): Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks. 2018 Conference on Empirical Methods in Natural Language Processing; EMNLP 2018, 31. October - 02. November 2018, Brussels, Belgium. [PDF, 193kB]

Yaghoobzadeh, Yadollah; Adel, Heike; Schütze, Hinrich (April 2018): Corpus-Level Fine-Grained Entity Typing. In: Journal of Artificial intelligence Research, Vol. 61: pp. 835-862 [PDF, 1MB]

Yaghoobzadeh, Yadollah; Schütze, Hinrich (October 2018): Multi-Multi-View Learning: Multilingual and Multi-Representation Entity Typing. 2018 Conference on Empirical Methods in Natural Language Processing, 31. October - 04. November 2018, Brussels, Belgium. [PDF, 292kB]

Yin, Wenpeng; Schütze, Hinrich (2018): Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms. In: Transactions of the Association for Computational Linguistics, Vol. 6: pp. 687-702 [PDF, 1MB]

Yin, Wenpeng; Yaghoobzadeh, Yadollah; Schütze, Hinrich (August 2018): Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs. The 27th International Conference on Computational Linguistics; COLING2018, 20. - 26. August 2018, Santa Fe, New Mexico, USA. [PDF, 424kB]