v1v2 (latest)

Psychological Metrics for Dialog System Evaluation

24 May 2023

Joao Sedoc

Papers citing "Psychological Metrics for Dialog System Evaluation"

30 / 30 papers shown

Title
Reasoning and the Trusting Behavior of DeepSeek and GPT: An Experiment Revealing Hidden Fault Lines in Large Language Models Rubing Li João Sedoc Arun Sundararajan LRM 88 1 0 20 Feb 2025
Whose Opinions Do Language Models Reflect? Shibani Santurkar Esin Durmus Faisal Ladhak Cinoo Lee Percy Liang Tatsunori Hashimoto 83 442 0 30 Mar 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 209 1,640 0 15 Dec 2022
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage Kurt Shuster Jing Xu M. Komeili Da Ju Eric Michael Smith ... Naman Goyal Arthur Szlam Y-Lan Boureau Melanie Kambadur Jason Weston LM&Ro KELM 110 242 0 05 Aug 2022
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 336 477 0 21 Jun 2022
Empathic Conversations: A Multi-level Dataset of Contextualized Conversations Damilola Omitaomu Shabnam Tafreshi Tingting Liu Sven Buechel Chris Callison-Burch J. Eichstaedt Lyle Ungar João Sedoc 81 49 0 25 May 2022
ProsocialDialog: A Prosocial Backbone for Conversational Agents Hyunwoo J. Kim Youngjae Yu Liwei Jiang Ximing Lu Daniel Khashabi Gunhee Kim Yejin Choi Maarten Sap 99 126 0 25 May 2022
What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation Sarik Ghazarian Behnam Hedayatnia Alexandros Papangelis Yang Liu Dilek Z. Hakkani-Tür 67 19 0 25 Mar 2022
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents Eric Michael Smith Orion Hsu Rebecca Qian Stephen Roller Y-Lan Boureau Jason Weston 67 68 0 12 Jan 2022
A General Language Assistant as a Laboratory for Alignment Amanda Askell Yuntao Bai Anna Chen Dawn Drain Deep Ganguli ... Tom B. Brown Jack Clark Sam McCandlish C. Olah Jared Kaplan ALM 120 789 0 01 Dec 2021
Automatic Evaluation and Moderation of Open-domain Dialogue Systems Chen Zhang João Sedoc L. F. D’Haro Rafael E. Banchs Alexander I. Rudnicky 70 38 0 03 Nov 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 119 849 0 22 Jun 2021
Detoxifying Language Models Risks Marginalizing Minority Voices Albert Xu Eshaan Pathak Eric Wallace Suchin Gururangan Maarten Sap Dan Klein 69 129 0 13 Apr 2021
Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems Vitou Phy Yang Zhao Akiko Aizawa 45 55 0 01 Nov 2020
An Evaluation Protocol for Generative Conversational Systems Seolhwa Lee Heuiseok Lim Jo˜ao Sedoc ELM 75 10 0 24 Oct 2020
Dialogue Response Ranking Training with Large-Scale Human Feedback Data Xiang Gao Yizhe Zhang Michel Galley Chris Brockett Bill Dolan ALM 69 105 0 15 Sep 2020
Unsupervised Evaluation of Interactive Dialog with DialoGPT Shikib Mehri M. Eskénazi 66 178 0 23 Jun 2020
USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation Shikib Mehri M. Eskénazi 67 226 0 01 May 2020
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing Brian Thompson Matt Post LRM 56 190 0 30 Apr 2020
Recipes for building an open-domain chatbot Stephen Roller Emily Dinan Naman Goyal Da Ju Mary Williamson ... Myle Ott Kurt Shuster Eric Michael Smith Y-Lan Boureau Jason Weston ALM 123 1,014 0 28 Apr 2020
Designing Precise and Robust Dialogue Response Evaluators Tianyu Zhao Divesh Lala Tatsuya Kawahara 44 53 0 10 Apr 2020
Towards a Human-like Open-Domain Chatbot Daniel De Freitas Minh-Thang Luong David R. So Jamie Hall Noah Fiedel ... Zi Yang Apoorv Kulshreshtha Gaurav Nemade Yifeng Lu Quoc V. Le 116 939 0 27 Jan 2020
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview Deven Santosh Shah H. Andrew Schwartz Dirk Hovy AI4CE 104 260 0 09 Nov 2019
Survey on Evaluation Methods for Dialogue Systems Jan Deriu Álvaro Rodrigo Arantxa Otegi Guillermo Echegoyen S. Rosset Eneko Agirre Mark Cieliebak 85 284 0 10 May 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 352 5,860 0 21 Apr 2019
Personalizing Dialogue Agents: I have a dog, do you have pets too? Saizheng Zhang Emily Dinan Jack Urbanek Arthur Szlam Douwe Kiela Jason Weston 118 1,465 0 22 Jan 2018
Reading Wikipedia to Answer Open-Domain Questions Danqi Chen Adam Fisch Jason Weston Antoine Bordes RALM 121 2,019 0 31 Mar 2017
A Neural Conversational Model Oriol Vinyals Quoc V. Le BDL 143 1,768 0 19 Jun 2015
Echoes of power: Language effects and power differences in social interaction Cristian Danescu-Niculescu-Mizil Lillian Lee B. Pang Jon M. Kleinberg 105 349 0 15 Dec 2011
Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs Cristian Danescu-Niculescu-Mizil Lillian Lee 97 428 0 15 Jun 2011