Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

14 March 2025

Papers citing "Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation"

50 / 64 papers shown

Title
CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring Benjamin Arnav Pablo Bernabeu-Pérez Nathan Helm-Burger Tim Kostolansky Hannes Whittingham Mary Phuong LRM 104 1 0 29 May 2025
Pitfalls of Rule- and Model-based Verifiers -- A Case Study on Mathematical Reasoning Yuzhen Huang Weihao Zeng Xingshan Zeng Qi Zhu Junxian He LRM 70 0 0 28 May 2025
Beyond Explainability: The Case for AI Validation Dalit Ken-Dror Feldman Daniel Benoliel 24 0 0 27 May 2025
SGM: A Framework for Building Specification-Guided Moderation Filters M. Fatehkia Enes Altinisik Husrev Taha Sencar 39 1 0 26 May 2025
Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models Baihui Zheng Boren Zheng Kerui Cao Y. Tan Zhendong Liu ... Jian Yang Wenbo Su Xiaoyong Zhu Bo Zheng Kaifu Zhang ELM 77 0 0 26 May 2025
Security Concerns for Large Language Models: A Survey Miles Q. Li Benjamin C. M. Fung PILM ELM 120 0 0 24 May 2025
Mitigating Deceptive Alignment via Self-Monitoring Jiaming Ji Wenqi Chen Kaile Wang Donghai Hong Sitong Fang ... Jiayi Zhou Juntao Dai Sirui Han Yike Guo Yaodong Yang LRM 42 2 0 24 May 2025
Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems Yihe Fan Wenqi Zhang Xudong Pan Min Yang 72 0 0 23 May 2025
Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions, and Improve with Training Dillon Plunkett Adam Morris Keerthi Reddy Jorge Morales MILM 51 0 0 21 May 2025
Preference Learning with Lie Detectors can Induce Honesty or Evasion Chris Cundy Adam Gleave 49 0 0 20 May 2025
Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations Li Ji-An Hua-Dong Xiong Robert C. Wilson Marcelo G. Mattar M. Benna 68 0 0 19 May 2025
Reasoning Models Don't Always Say What They Think Yanda Chen Joe Benton Ansh Radhakrishnan Jonathan Uesato Carson E. Denison ... Vlad Mikulik Samuel R. Bowman Jan Leike Jared Kaplan E. Perez ReLM LRM 146 49 1 08 May 2025
Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models Guanghao Zhou Panjia Qiu Chong Chen Jiadong Wang Zheming Yang Jian Xu Minghui Qiu OffRL LRM 184 8 0 30 Apr 2025
IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery Aniketh Garikaparthi Manasi Patwardhan Lovekesh Vig Arman Cohan VLM LRM 116 1 0 23 Apr 2025
Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systems Simon Lermen Mateusz Dziemian Natalia Pérez-Campanero Antolín 98 0 0 10 Apr 2025
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations Pedro Ferreira Wilker Aziz Ivan Titov LRM 68 0 0 07 Apr 2025
How to evaluate control measures for LLM agents? A trajectory from today to superintelligence Tomek Korbak Mikita Balesni Buck Shlegeris Geoffrey Irving ELM 95 1 0 07 Apr 2025
Inference-Time Scaling for Generalist Reward Modeling Zijun Liu P. Wang Ran Xu Shirong Ma Chong Ruan Ziwei Sun Yang Liu Y. Wu OffRL LRM 182 54 0 03 Apr 2025
Probabilistic Uncertain Reward Model Wangtao Sun Xiang Cheng Xing Yu Haotian Xu Zhao Yang Shizhu He Jun Zhao Kang Liu 154 0 0 28 Mar 2025
Chain-of-Thought Reasoning In The Wild Is Not Always Faithful Iván Arcuschin Jett Janiak Robert Krzyzanowski Senthooran Rajamanoharan Neel Nanda Arthur Conmy ReLM LRM 155 20 0 11 Mar 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 380 2,000 0 22 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 291 338 0 22 Jan 2025
GPT-4o System Card OpenAI OpenAI : Aaron Hurst Adam Lerer Adam P. Goucher ... Yuchen He Yuchen Zhang Yujia Jin Yunxing Dai Yury Malkov MLLM 212 1,038 0 25 Oct 2024
Language Models Learn to Mislead Humans via RLHF Jiaxin Wen Ruiqi Zhong Akbir Khan Ethan Perez Jacob Steinhardt Minlie Huang Samuel R. Bowman He He Shi Feng 81 43 0 19 Sep 2024
Prover-Verifier Games improve legibility of LLM outputs Jan Hendrik Kirchner Yining Chen Harri Edwards Jan Leike Nat McAleese Yuri Burda LRM AAML 64 32 0 18 Jul 2024
Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models Carson E. Denison M. MacDiarmid Fazl Barez David Duvenaud Shauna Kravec ... Jared Kaplan Buck Shlegeris Samuel R. Bowman Ethan Perez Evan Hubinger 118 44 0 14 Jun 2024
Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms Rafael Rafailov Yaswanth Chittepu Ryan Park Harshit S. Sikchi Joey Hejna Bradley Knox Chelsea Finn S. Niekum 121 69 0 05 Jun 2024
From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step Yuntian Deng Yejin Choi Stuart M. Shieber ReLM LRM 70 76 0 23 May 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 127 158 0 22 Apr 2024
Eliciting Latent Knowledge from Quirky Language Models Alex Troy Mallen Madeline Brumley Julia Kharchenko Nora Belrose HILM RALM KELM 77 33 0 02 Dec 2023
On Measuring Faithfulness or Self-consistency of Natural Language Explanations Letitia Parcalabescu Anette Frank LRM 100 29 0 13 Nov 2023
How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions Lorenzo Pacchiardi A. J. Chan Sören Mindermann Ilan Moscovitz Alexa Y. Pan Y. Gal Owain Evans J. Brauner LLMAG HILM 75 54 0 26 Sep 2023
Inverse Scaling: When Bigger Isn't Better I. R. McKenzie Alexander Lyzhov Michael Pieler Alicia Parrish Aaron Mueller ... Yuhui Zhang Zhengping Zhou Najoung Kim Sam Bowman Ethan Perez 81 140 0 15 Jun 2023
Gorilla: Large Language Model Connected with Massive APIs Shishir G. Patil Tianjun Zhang Xin Wang Joseph E. Gonzalez ELM CLL ALM SyDa 93 568 0 24 May 2023
Reasoning with Language Model is Planning with World Model Shibo Hao Yi Gu Haodi Ma Joshua Jiahua Hong Zhen Wang D. Wang Zhiting Hu ReLM LRM LLMAG 156 600 0 24 May 2023
Tree of Thoughts: Deliberate Problem Solving with Large Language Models Shunyu Yao Dian Yu Jeffrey Zhao Izhak Shafran Thomas Griffiths Yuan Cao Karthik Narasimhan LM&Ro LRM AI4CE 168 2,037 0 17 May 2023
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models Pan Lu Baolin Peng Hao Cheng Michel Galley Kai-Wei Chang Ying Nian Wu Song-Chun Zhu Jianfeng Gao KELM MLLM LRM 120 324 0 19 Apr 2023
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Yongliang Shen Kaitao Song Xu Tan Dongsheng Li Weiming Lu Yueting Zhuang MLLM 125 911 0 30 Mar 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark Gales HILM LRM 193 445 0 15 Mar 2023
Toolformer: Language Models Can Teach Themselves to Use Tools Timo Schick Jane Dwivedi-Yu Roberto Dessì Roberta Raileanu Maria Lomeli Luke Zettlemoyer Nicola Cancedda Thomas Scialom SyDa RALM 164 1,772 0 09 Feb 2023
Solving math word problems with process- and outcome-based feedback J. Uesato Nate Kushman Ramana Kumar Francis Song Noah Y. Siegel L. Wang Antonia Creswell G. Irving I. Higgins FaML ReLM AIMat LRM 123 362 0 25 Nov 2022
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks Wenhu Chen Xueguang Ma Xinyi Wang William W. Cohen ReLM ReCod LRM 177 825 0 22 Nov 2022
PAL: Program-aided Language Models Luyu Gao Aman Madaan Shuyan Zhou Uri Alon Pengfei Liu Yiming Yang Jamie Callan Graham Neubig ReLM LRM 104 460 0 18 Nov 2022
Measuring Progress on Scalable Oversight for Large Language Models Sam Bowman Jeeyoon Hyun Ethan Perez Edwin Chen Craig Pettit ... Tristan Hume Yuntao Bai Zac Hatfield-Dodds Benjamin Mann Jared Kaplan ALM ELM 79 132 0 04 Nov 2022
Scaling Laws for Reward Model Overoptimization Leo Gao John Schulman Jacob Hilton ALM 112 568 0 19 Oct 2022
Measuring and Narrowing the Compositionality Gap in Language Models Ofir Press Muru Zhang Sewon Min Ludwig Schmidt Noah A. Smith M. Lewis ReLM KELM LRM 202 642 0 07 Oct 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 532 4,077 0 24 May 2022
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning Antonia Creswell Murray Shanahan I. Higgins ReLM LRM 112 364 0 19 May 2022
STaR: Bootstrapping Reasoning With Reasoning E. Zelikman Yuhuai Wu Jesse Mu Noah D. Goodman ReLM LRM 144 511 0 28 Mar 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 526 3,737 0 21 Mar 2022