How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

25 March 2016

Papers citing "How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation"

43 / 293 papers shown

Title
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder Xiaodong Gu Kyunghyun Cho Jung-Woo Ha Sunghun Kim DRL 41 129 0 31 May 2018
A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS) Kamal Al-Sabahi Zuping Zhang Mohammed Nadher 19 109 0 20 May 2018
A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation Juraj Juraska P. Karagiannis Kevin K. Bowden M. Walker 39 88 0 16 May 2018
Automatic Article Commenting: the Task and Dataset Lianhui Qin Lemao Liu Victoria Bi Yan Wang Xiaojiang Liu Zhiting Hu Zhao Hai Shuming Shi 25 26 0 09 May 2018
Polite Dialogue Generation Without Parallel Data Tong Niu Joey Tianyi Zhou OffRL MoMe 30 168 0 08 May 2018
No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling Xin Eric Wang Wenhu Chen Yuan-fang Wang William Yang Wang 16 157 0 24 Apr 2018
A Hierarchical Latent Structure for Variational Conversation Modeling Yookoon Park Jaemin Cho Gunhee Kim BDL DRL 24 97 0 10 Apr 2018
Improving Variational Encoder-Decoders in Dialogue Generation Xiaoyu Shen Hui Su Shuzi Niu Vera Demberg DRL 32 99 0 06 Feb 2018
Personalizing Dialogue Agents: I have a dog, do you have pets too? Saizheng Zhang Emily Dinan Jack Urbanek Arthur Szlam Douwe Kiela Jason Weston 55 1,433 0 22 Jan 2018
Why Do Neural Dialog Systems Generate Short and Meaningless Replies? A Comparison between Dialog and Translation Bolin Wei Shuai Lu Lili Mou Hao Zhou Pascal Poupart Ge Li Zhi Jin 35 29 0 06 Dec 2017
Neural Response Generation with Dynamic Vocabularies Yu-Huan Wu Wei Wu Dejian Yang Can Xu Zhoujun Li Ming Zhou 26 74 0 30 Nov 2017
Detecting Egregious Conversations between Customers and Virtual Agents T. Sandbank Michal Shmueli-Scheuer Jonathan Herzig D. Konopnicki John T. Richards David Piorkowski 23 20 0 15 Nov 2017
A Survey on Dialogue Systems: Recent Advances and New Frontiers Hongshen Chen Xiaorui Liu Dawei Yin Jiliang Tang VLM LLMAG 38 695 0 06 Nov 2017
Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models Yi Luan Chris Brockett W. Dolan Jianfeng Gao Michel Galley 45 82 0 20 Oct 2017
Harvesting Creative Templates for Generating Stylistically Varied Restaurant Reviews Shereen Oraby Sheideh Homayon M. Walker 11 8 0 15 Sep 2017
Perspectives for Evaluating Conversational AI Mahipal Jadeja Neelanshi Varia 14 17 0 14 Sep 2017
A Deep Reinforcement Learning Chatbot Iulian Serban Chinnadhurai Sankar M. Germain Saizheng Zhang Zhouhan Lin ... Dendi Suhubdy Vincent Michalski A. Nguyen Joelle Pineau Yoshua Bengio 36 234 0 07 Sep 2017
Evaluating Visual Conversational Agents via Cooperative Human-AI Games Prithvijit Chattopadhyay Deshraj Yadav Viraj Prabhu Arjun Chandrasekaran Abhishek Das Stefan Lee Dhruv Batra Devi Parikh 38 78 0 17 Aug 2017
Reinforced Video Captioning with Entailment Rewards Ramakanth Pasunuru Joey Tianyi Zhou 28 114 0 07 Aug 2017
Referenceless Quality Estimation for Natural Language Generation Ondrej Dusek Jekaterina Novikova Verena Rieser 34 29 0 05 Aug 2017
Domain Aware Neural Dialog System Sajal Choudhary Prerna Srivastava L. Ungar João Sedoc VLM 24 23 0 02 Aug 2017
Challenges in Data-to-Document Generation Sam Wiseman Stuart M. Shieber Alexander M. Rush 42 581 0 25 Jul 2017
Why We Need New Evaluation Metrics for NLG Jekaterina Novikova Ondrej Dusek Amanda Cercas Curry Verena Rieser 51 453 0 21 Jul 2017
Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation Shikhar Sharma Layla El Asri Hannes Schulz Jeremie Zumer 26 220 0 29 Jun 2017
Deal or No Deal? End-to-End Learning for Negotiation Dialogues M. Lewis Denis Yarats Yann N. Dauphin Devi Parikh Dhruv Batra LLMAG 32 409 0 16 Jun 2017
Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model Jiasen Lu A. Kannan Jianwei Yang Devi Parikh Dhruv Batra BDL 38 136 0 05 Jun 2017
A Deep Reinforced Model for Abstractive Summarization Romain Paulus Caiming Xiong R. Socher AI4TS 53 1,549 0 11 May 2017
A Conditional Variational Framework for Dialog Generation Xiaoyu Shen Hui Su Yanran Li Wenjie Li Shuzi Niu Yang Zhao Akiko Aizawa Guoping Long 30 115 0 30 Apr 2017
Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings He He Anusha Balakrishnan Mihail Eric Percy Liang 37 200 0 24 Apr 2017
Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets Wei-Lun Chao Hexiang Hu Fei Sha 22 37 0 24 Apr 2017
Data Distillation for Controlling Specificity in Dialogue Generation Jiwei Li Will Monroe Dan Jurafsky 23 22 0 22 Feb 2017
A Knowledge-Grounded Neural Conversation Model Marjan Ghazvininejad Chris Brockett Ming-Wei Chang W. Dolan Jianfeng Gao Wen-tau Yih Michel Galley BDL 33 569 0 07 Feb 2017
Hierarchical Recurrent Attention Network for Response Generation Chen Xing Wei Wu Yu-Huan Wu Ming Zhou Yalou Huang Wei-Ying Ma 30 209 0 25 Jan 2017
Adversarial Learning for Neural Dialogue Generation Jiwei Li Will Monroe Tianlin Shi Sébastien Jean Alan Ritter Dan Jurafsky 21 898 0 23 Jan 2017
RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems Chongyang Tao Lili Mou Dongyan Zhao Rui Yan 28 217 0 11 Jan 2017
Visual Dialog Abhishek Das Satwik Kottur Khushi Gupta Avi Singh Deshraj Yadav José M. F. Moura Devi Parikh Dhruv Batra 69 990 0 26 Nov 2016
A Simple, Fast Diverse Decoding Algorithm for Neural Generation Jiwei Li Will Monroe Dan Jurafsky 33 239 0 25 Nov 2016
GuessWhat?! Visual object discovery through multi-modal dialogue H. D. Vries Florian Strub A. Chandar Olivier Pietquin Hugo Larochelle Aaron Courville VLM 50 427 0 23 Nov 2016
Chinese Poetry Generation with Planning based Neural Network Zhe Wang Wei He Hua Wu Haiyang Wu Wei Li Haifeng Wang Enhong Chen 35 133 0 31 Oct 2016
An Attentional Neural Conversation Model with Improved Specificity Kaisheng Yao Baolin Peng Geoffrey Zweig Kam-Fai Wong 37 30 0 03 Jun 2016
Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation Iulian Serban Tim Klinger Gerald Tesauro Kartik Talamadupula Bowen Zhou Yoshua Bengio Aaron Courville 19 190 0 02 Jun 2016
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues Iulian Serban Alessandro Sordoni Ryan J. Lowe Laurent Charlin Joelle Pineau Aaron Courville Yoshua Bengio BDL 69 1,104 0 19 May 2016
On the Evaluation of Dialogue Systems with Next Utterance Classification Ryan J. Lowe Iulian Serban Michael Noseworthy Laurent Charlin Joelle Pineau ELM 29 66 0 18 May 2016