Political Compass or Spinning Arrow? Towards More Meaningful Evaluations
for Values and Opinions in Large Language Models

v1v2 (latest)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

26 February 2024

Paul Röttger

Valentin Hofmann

Valentina Pyatkin

Hannah Rose Kirk

Hinrich Schütze

ArXiv (abs)PDF HTML

Papers citing "Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models"

14 / 14 papers shown

Title
Linear Representations of Political Perspective Emerge in Large Language Models Junsol Kim James Evans Aaron Schein 132 7 0 03 Mar 2025
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 285 2 0 26 Oct 2024
PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action Yijia Shao Tianshi Li Weiyan Shi Yanchen Liu Diyi Yang PILM 132 31 0 29 Aug 2024
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions Siru Ouyang Shuohang Wang Yang Liu Ming Zhong Yizhu Jiao Dan Iter Reid Pryzant Chenguang Zhu Heng Ji Jiawei Han 90 31 0 19 Oct 2023
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values Hannah Rose Kirk Andrew M. Bean Bertie Vidgen Paul Röttger Scott A. Hale ALM 89 50 0 11 Oct 2023
AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics V. Ghafouri Vibhor Agarwal Yong Zhang Nishanth R. Sastry Jose Such Guillermo Suarez-Tangil AI4MH 92 23 0 28 Aug 2023
Whose Opinions Do Language Models Reflect? Shibani Santurkar Esin Durmus Faisal Ladhak Cinoo Lee Percy Liang Tatsunori Hashimoto 83 442 0 30 Mar 2023
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective Jindong Wang Xixu Hu Wenxin Hou Hao Chen Runkai Zheng ... Weirong Ye Xiubo Geng Binxing Jiao Yue Zhang Xingxu Xie AI4MH 131 236 0 22 Feb 2023
Who is GPT-3? An Exploration of Personality, Values and Demographics Marilù Miotto Nicola Rossberg Bennett Kleinberg ELM PILM 70 114 0 28 Sep 2022
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 338 484 0 21 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 886 13,207 0 04 Mar 2022
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models Wei Ping Chejian Xu Shuohang Wang Zhe Gan Yu Cheng Jianfeng Gao Ahmed Hassan Awadallah Yangqiu Song VLM ELM AAML 69 225 0 04 Nov 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 331 368 0 01 Feb 2021
Aligning AI With Shared Human Values Dan Hendrycks Collin Burns Steven Basart Andrew Critch Jingkai Li Basel Alomair Jacob Steinhardt 145 574 0 05 Aug 2020

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.