Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement

Poerner, Nina; Roth, Benjamin und Schütze, Hinrich (Juli 2018): Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. The 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 15. - 20. July 2018. Gurevych, Iryna und Miyao, Yusuke (Hrsg.): In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA: Association for Computational Linguistics (ACL). S. 340-350

Abstract

The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems. Both paradigms require no manual annotation and are therefore broadly applicable.We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP. We show empirically that LIMSSE, LRP and DeepLIFT are the mosteffective explanation methods and recommend them for explaining DNNs in NLP.

• Dokumententyp: Konferenzbeitrag (Paper)
• EU Funded Grant Agreement Number: 740516
• EU-Projekte: Horizon 2020 > ERC Grants > ERC Advanced Grant > ERC Grant 740516: NonSequeToR - Non-sequence models for tokenization replacement
• Fakultätsübergreifende Einrichtungen: Centrum für Informations- und Sprachverarbeitung (CIS)
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme ; 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik ; 400 Sprache > 400 Sprache ; 400 Sprache > 410 Linguistik
• Ort: Stroudsburg, PA
• Bemerkung: ISBN 978-1-948087-32-2
• Sprache: Englisch