Political Compass or Spinning Arrow? Towards More Meaningful Evaluations
for Values and Opinions in Large Language Models

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models

26 February 2024

Paul Röttger

Valentin Hofmann

Valentina Pyatkin

Hannah Rose Kirk

Hinrich Schütze

Papers citing "Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models"

13 / 13 papers shown

Title
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions Saffron Huang Esin Durmus Miles McCain Kunal Handa Alex Tamkin Jerry Hong Michael Stern Arushi Somani Xiuruo Zhang Deep Ganguli VLM 51 1 0 21 Apr 2025
Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models Mats Faulborn Indira Sen Max Pellert Andreas Spitz David Garcia ELM 45 0 0 20 Mar 2025
Linear Representations of Political Perspective Emerge in Large Language Models Junsol Kim James Evans Aaron Schein 77 2 0 03 Mar 2025
Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models Haoran Ye T. Zhang Yuhang Xie Liyuan Zhang Yuanyi Ren Xin Zhang Guojie Song PILM 81 0 0 04 Feb 2025
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 76 1 0 26 Oct 2024
Exposing Assumptions in AI Benchmarks through Cognitive Modelling Jonathan H. Rystrøm Kenneth C. Enevoldsen 39 0 0 25 Sep 2024
Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models Haoran Ye Yuhang Xie Yuanyi Ren Hanjun Fang Xin Zhang Guojie Song LM&MA 37 1 0 18 Sep 2024
Measuring Political Bias in Large Language Models: What Is Said and How It Is Said Yejin Bang Delong Chen Nayeon Lee Pascale Fung 32 25 0 27 Mar 2024
The Political Preferences of LLMs David Rozado 38 36 0 02 Feb 2024
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 25 125 0 02 Aug 2023
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 250 440 0 21 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 269 346 0 01 Feb 2021