The Potential and Challenges of Evaluating Attitudes, Opinions, and
Values in Large Language Models

The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models

16 June 2024

Michael A. Hedderich

Barbara Plank

Papers citing "The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models"

15 / 15 papers shown

Title
Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases Shanshan Xu T. Y. S. S. Santosh Yanai Elazar Quirin Vogel Barbara Plank Matthias Grabmair AILaw 83 0 0 25 Feb 2025
Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models Haoran Ye Yuhang Xie Yuanyi Ren Hanjun Fang Xin Zhang Guojie Song LM&MA 42 1 0 18 Sep 2024
What are human values, and how do we align AI to them? Oliver Klingefjord Ryan Lowe Joe Edelman 36 22 0 27 Mar 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 67 56 0 05 Mar 2024
The Political Preferences of LLMs David Rozado 40 36 0 02 Feb 2024
Large Language Models Understand and Can be Enhanced by Emotional Stimuli Cheng-rong Li Jindong Wang Yixuan Zhang Kaijie Zhu Wenxin Hou Jianxun Lian Fang Luo Qiang Yang Xingxu Xie LRM 80 121 0 14 Jul 2023
AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction Junsol Kim Byungkyu Lee SyDa 38 32 0 16 May 2023
PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits Hang Jiang Xiajie Zhang Xubo Cao Cynthia Breazeal Deb Roy Jad Kabbara 57 76 0 04 May 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 244 1,764 0 07 Apr 2023
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity Gabriel Simmons 108 57 0 24 Sep 2022
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 252 443 0 21 Jun 2022
Does Moral Code Have a Moral Code? Probing Delphi's Moral Philosophy Kathleen C. Fraser S. Kiritchenko Esma Balkir 119 37 0 25 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 384 12,081 0 04 Mar 2022
Can Machines Learn Morality? The Delphi Experiment Liwei Jiang Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jenny T Liang ... Yulia Tsvetkov Oren Etzioni Maarten Sap Regina A. Rini Yejin Choi FaML 129 111 0 14 Oct 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 281 1,124 0 18 Apr 2021