PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism

Liu, Yongkang; Feng, Shi; Wang, Daling; Zhang, Yifei and Schütze, Hinrich (July 2023): PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism. 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), Toronto, Canada, July 2023. Rogers, Anna; Boyd-Graber, Jordan and Okazaki, Naoaki (eds.) : In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Stroudsburg, PA: Association for Computational Linguistics (ACL). pp. 3295-3310

Abstract

We investigate response generation for multi-turn dialogue in generative chatbots. Existing generative modelsbased on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the history, which makesmodels unable to capture the subtle variability observed in different dialogues and cannot distinguish the differencesbetween dialogues that are similar in composition. In this paper, we propose Pseudo-Variational Gated Recurrent Unit (PVGRU). The key novelty of PVGRU is a recurrent summarizing variable thataggregates the accumulated distribution variations of subsequences. We train PVGRU without relying on posterior knowledge, thus avoiding the training-inference inconsistency problem. PVGRU can perceive subtle semantic variability through summarizing variables that are optimized by two objectives we employ for training: distribution consistency and reconstruction. In addition, we build a Pseudo-Variational Hierarchical Dialogue(PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity andrelevance of responses on two benchmark datasets.

• Item Type: Conference or Workshop Item (Paper)
• EU Funded Grant Agreement Number: 740516
• EU Projects: Horizon 2020 > ERC Grants > ERC Advanced Grant > ERC Grant 740516: NonSequeToR - Non-sequence models for tokenization replacement
• Research Centers: Center for Information and Language Processing (CIS)
• Subjects: 400 Language > 400 Language; 400 Language > 410 Linguistics
• Place of Publication: Stroudsburg, PA
• Language: English