Papers citing "Constitutional AI: Harmlessness from AI Feedback"

50 / 1,106 papers shown

Title
LLM-Safety Evaluations Lack Robustness Tim Beyer Sophie Xhonneux Simon Geisler Gauthier Gidel Leo Schwinn Stephan Günnemann ALM ELM 171 0 0 04 Mar 2025
Proportionality in Thumbs Up and Down Voting Sonja Kraiczy Georgios Papasotiropoulos Grzegorz Pierczynski P. Skowron 44 0 0 03 Mar 2025
SFO: Piloting VLM Feedback for Offline RL Jacob Beck OffRL 36 0 0 02 Mar 2025
Efficient Jailbreaking of Large Models by Freeze Training: Lower Layers Exhibit Greater Sensitivity to Harmful Content Hongyuan Shen Min Zheng Jincheng Wang Yang Zhao 42 0 0 28 Feb 2025
A database to support the evaluation of gender biases in GPT-4o output Luise Mehner Lena Alicija Philine Fiedler Sabine Ammon Dorothea Kolossa 60 0 0 28 Feb 2025
Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers Shalev Lifshitz Sheila A. McIlraith Yilun Du LRM 55 5 0 27 Feb 2025
Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones? Yudi Zhang Lu Wang Meng Fang Yali Du Chenghua Huang ... Qingwei Lin Mykola Pechenizkiy Dongmei Zhang Saravan Rajmohan Qi Zhang ALM 78 0 0 26 Feb 2025
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance Chenghua Huang Lu Wang Fangkai Yang Pu Zhao ZeLin Li Qingwei Lin Dongmei Zhang Saravan Rajmohan Qi Zhang OffRL 52 1 0 24 Feb 2025
The Call for Socially Aware Language Technologies Diyi Yang Dirk Hovy David Jurgens Barbara Plank VLM 61 11 0 24 Feb 2025
Policy Learning with a Natural Language Action Space: A Causal Approach Bohan Zhang Yixin Wang Paramveer S. Dhillon CML 46 0 0 24 Feb 2025
DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents Taiyi Wang Zhihao Wu Jianheng Liu Jianye Hao Jun Wang Kun Shao OffRL 36 13 0 24 Feb 2025
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs Jan Betley Daniel Tan Niels Warncke Anna Sztyber-Betley Xuchan Bao Martín Soto Nathan Labenz Owain Evans AAML 80 9 0 24 Feb 2025
RLTHF: Targeted Human Feedback for LLM Alignment Yifei Xu Tusher Chakraborty Emre Kıcıman Bibek Aryal Eduardo Rodrigues ... Rafael Padilha Leonardo Nunes Shobana Balakrishnan Songwu Lu Ranveer Chandra 110 1 0 24 Feb 2025
Guardians of the Agentic System: Preventing Many Shots Jailbreak with Agentic System Saikat Barua Mostafizur Rahman Md Jafor Sadek Rafiul Islam Shehnaz Khaled Ahmedul Kabir LLMAG 58 1 0 23 Feb 2025
IPO: Your Language Model is Secretly a Preference Classifier Shivank Garg Ayush Singh Shweta Singh Paras Chopra 145 1 0 22 Feb 2025
A Survey of Model Architectures in Information Retrieval Zhichao Xu Fengran Mo Zhiqi Huang Crystina Zhang Puxuan Yu Bei Wang Jimmy J. Lin Vivek Srikumar KELM 3DV 58 2 0 21 Feb 2025
C3AI: Crafting and Evaluating Constitutions for Constitutional AI Yara Kyrychenko Ke Zhou Edyta Bogucka Daniele Quercia ELM 45 3 0 21 Feb 2025
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment Zhili Liu Yunhao Gou Kai Chen Lanqing Hong Jiahui Gao ... Yu Zhang Zhenguo Li Xin Jiang Qiang Liu James T. Kwok MoE 96 9 0 20 Feb 2025
Policy-to-Language: Train LLMs to Explain Decisions with Flow-Matching Generated Rewards Xinyi Yang Liang Zeng Heng Dong Chao Yu X. Wu H. Yang Yu Wang Milind Tambe Tonghan Wang 76 2 0 18 Feb 2025
Sleepless Nights, Sugary Days: Creating Synthetic Users with Health Conditions for Realistic Coaching Agent Interactions Taedong Yun Eric Yang Mustafa Safdari Jong Ha Lee Vaishnavi Vinod Kumar ... Isaac Galatzer-Levy Yugang Jia John Canny Arthur Gretton Maja J. Matarić SyDa 86 1 0 18 Feb 2025
Scaling Autonomous Agents via Automatic Reward Modeling And Planning Zhenfang Chen Delin Chen Rui Sun Wenjun Liu Chuang Gan LLMAG 60 3 0 17 Feb 2025
SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities Fengqing Jiang Zhangchen Xu Yuetai Li Luyao Niu Zhen Xiang Bo-wen Li Bill Yuchen Lin Radha Poovendran KELM ELM LRM 83 14 0 17 Feb 2025
Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? Yufei He Yuexin Li Jiaying Wu Yuan Sui Yulin Chen Bryan Hooi ALM 94 5 0 16 Feb 2025
Efficient Multitask Learning in Small Language Models Through Upside-Down Reinforcement Learning Yu-Chen Lin Sanat Sharma Hari Manikandan Jayant Kumar Tracy Holloway King Jing Zheng LRM 60 1 0 14 Feb 2025
Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models Xin Zhou Yiwen Guo Ruotian Ma Tao Gui Qi Zhang Xuanjing Huang LRM 92 2 0 13 Feb 2025
Escaping Collapse: The Strength of Weak Data for Large Language Model Training Kareem Amin Sara Babakniya Alex Bie Weiwei Kong Umar Syed Sergei Vassilvitskii 70 1 0 13 Feb 2025
Trustworthy AI on Safety, Bias, and Privacy: A Survey Xingli Fang Jianwei Li Varun Mulchandani Jung-Eun Kim 45 0 0 11 Feb 2025
DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization Xuefeng Liu Songhao Jiang Siyu Chen Zhuoran Yang Yuxin Chen Ian Foster Rick L. Stevens LM&MA OffRL 58 0 0 11 Feb 2025
AI Alignment at Your Discretion Maarten Buyl Hadi Khalaf C. M. Verdun Lucas Monteiro Paes Caio Vieira Machado Flavio du Pin Calmon 45 0 0 10 Feb 2025
Jailbreaking to Jailbreak Jeremy Kritz Vaughn Robinson Robert Vacareanu Bijan Varjavand Michael Choi Bobby Gogov Scale Red Team Summer Yue Willow Primack Zifan Wang 209 1 0 09 Feb 2025
Refining Positive and Toxic Samples for Dual Safety Self-Alignment of LLMs with Minimal Human Interventions Jingxin Xu Guoshun Nan Sheng Guan Sicong Leng Yong-Jin Liu Zixiao Wang Yuyang Ma Zhili Zhou Yanzhao Hou Xiaofeng Tao LM&MA 55 0 0 08 Feb 2025
Evolving LLMs' Self-Refinement Capability via Iterative Preference Optimization Yongcheng Zeng Xinyu Cui Xuanfa Jin Guoqing Liu Zexu Sun Quan He Dong Li Ning Yang Jianye Hao Hongzhi Zhang LLMAG LRM 90 1 0 08 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 99 2 0 06 Feb 2025
Adaptive Self-improvement LLM Agentic System for ML Library Development Genghan Zhang Weixin Liang Olivia Hsu K. Olukotun 152 0 0 04 Feb 2025
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs Angelina Wang Michelle Phan Daniel E. Ho Sanmi Koyejo 54 2 0 04 Feb 2025
Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models Haoran Ye T. Zhang Yuhang Xie Liyuan Zhang Yuanyi Ren Xin Zhang Guojie Song PILM 81 0 0 04 Feb 2025
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering Zongxi Li Y. Li Haoran Xie S. J. Qin 70 0 0 03 Feb 2025
Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalization Minttu Alakuijala Ya Gao Georgy Ananov Samuel Kaski Pekka Marttinen Alexander Ilin Harri Valpola LLMAG CLL 80 0 0 03 Feb 2025
Understanding the Capabilities and Limitations of Weak-to-Strong Generalization Wei Yao Wenkai Yang Zhilin Wang Yankai Lin Yong Liu ELM 107 1 0 03 Feb 2025
Diverse Preference Optimization Jack Lanchantin Angelica Chen S. Dhuliawala Ping Yu Jason Weston Sainbayar Sukhbaatar Ilia Kulikov 93 4 0 30 Jan 2025
On The Truthfulness of 'Surprisingly Likely' Responses of Large Language Models Naman Goel HILM 57 0 0 28 Jan 2025
Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models Robin Young 49 0 0 28 Jan 2025
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models Jingwei Yi Yueqi Xie Bin Zhu Emre Kiciman Guangzhong Sun Xing Xie Fangzhao Wu AAML 62 65 0 28 Jan 2025
Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning Hao Sun M. Schaar 94 14 0 28 Jan 2025
Training Dialogue Systems by AI Feedback for Improving Overall Dialogue Impression Kai Yoshida M. Mizukami Seiya Kawano Canasai Kruengkrai Hiroaki Sugiyama Koichiro Yoshino ALM OffRL 78 1 0 28 Jan 2025
Evolution and The Knightian Blindspot of Machine Learning Joel Lehman Elliot Meyerson Tarek El-Gaaly Kenneth O. Stanley Tarin Ziyaee 86 1 0 22 Jan 2025
MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking Sebastian Farquhar Vikrant Varma David Lindner David Elson Caleb Biddulph Ian Goodfellow Rohin Shah 82 1 0 22 Jan 2025
InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model Yuhang Zang Xiaoyi Dong Pan Zhang Yuhang Cao Ziyu Liu ... Haodong Duan W. Zhang Kai Chen Dahua Lin Jiaqi Wang VLM 74 19 0 21 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 82 44 0 20 Jan 2025
RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs Jiaxing Wu Lin Ning Luyang Liu Harrison Lee Neo Wu Chao Wang Sushant Prakash S. O’Banion Bradley Green Jun Xie 71 1 0 20 Jan 2025