Studying the Effects of Cognitive Biases in Evaluation of Conversational
Agents

v1v2 (latest)

Studying the Effects of Cognitive Biases in Evaluation of Conversational Agents

18 February 2020

Sashank Santhanam

Alireza Karduni

ArXiv (abs)PDF HTML

Papers citing "Studying the Effects of Cognitive Biases in Evaluation of Conversational Agents"

13 / 13 papers shown

Title
Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search Jiqun Liu Jamshed Karimnazarov Ryen W. White 73 0 0 02 Apr 2025
ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models Aparna Elangovan Ling Liu Lei Xu S. Bodapati Dan Roth ELM 103 10 0 28 May 2024
Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs Clemencia Siro Mohammad Aliannejadi Maarten de Rijke 56 1 0 19 Apr 2024
Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems Clemencia Siro Mohammad Aliannejadi Maarten de Rijke 69 3 0 15 Apr 2024
Evaluating and Improving Value Judgments in AI: A Scenario-Based Study on Large Language Models' Depiction of Social Conventions Jaeyoun You Bongwon Suh 82 0 0 04 Oct 2023
Collect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection Oana Inel Tim Draws Lora Aroyo 110 6 0 22 Aug 2023
"If I Had All the Time in the World": Ophthalmologists' Perceptions of Anchoring Bias Mitigation in Clinical AI Support A. Bach Trine Munch Norgaard Jens Christian Brok N. V. Berkel 55 33 0 07 Mar 2023
How Accurate Does It Feel? -- Human Perception of Different Types of Classification Mistakes A. Papenmeier Dagmar Kern Daniel Hienert Yvonne Kammerer C. Seifert 72 20 0 13 Feb 2023
Achieving Reliable Human Assessment of Open-Domain Dialogue Systems Tianbo Ji Yvette Graham Gareth J. F. Jones Chenyang Lyu Qun Liu ALM 75 39 0 11 Mar 2022
Alexa, in you, I trust! Fairness and Interpretability Issues in E-commerce Search through Smart Speakers A. Dash Abhijnan Chakraborty Saptarshi Ghosh Animesh Mukherjee Krishna P. Gummadi 136 10 0 08 Feb 2022
Towards Standard Criteria for human evaluation of Chatbots: A Survey Hongru Liang Huaqing Li 56 13 0 24 May 2021
Local Knowledge Powered Conversational Agents Sashank Santhanam Ming-Yu Liu Raul Puri Mohammad Shoeybi M. Patwary Bryan Catanzaro 95 4 0 20 Oct 2020
Learning to Plan and Realize Separately for Open-Ended Dialogue Systems Sashank Santhanam Zhuo Cheng Brodie Mather Bonnie J. Dorr Archna Bhatia Bryanna Hebenstreit Alan Zemel Adam Dalton T. Strzalkowski Samira Shaikh 58 6 0 26 Sep 2020