v1v2 (latest)

Concrete Problems in AI Safety

21 June 2016

Papers citing "Concrete Problems in AI Safety"

50 / 95 papers shown

Title
Redefining Superalignment: From Weak-to-Strong Alignment to Human-AI Co-Alignment to Sustainable Symbiotic Society Yi Zeng Yijiao Wang Enmeng Lu Dongcheng Zhao Bing Han ... Chao Liu Yaodong Yang Yi Zeng Boyuan Chen Jinyu Fan 142 0 0 24 Apr 2025
Learning to Reason under Off-Policy Guidance Jianhao Yan Yafu Li Zican Hu Zhi Wang Ganqu Cui Xiaoye Qu Yu Cheng Yue Zhang OffRL LRM 102 11 0 21 Apr 2025
SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM Xinyu Zhang Jiadong Wang Zifei Cheng Wenhao Zhuang Zheng Lin ... Shouyu Yin Chaohang Wen Haotian Zhang Bin Chen Bing Yu LRM 141 9 0 19 Apr 2025
Adversarial Training of Reward Models Alexander Bukharin Haifeng Qian Shengyang Sun Adithya Renduchintala Soumye Singhal Ziyi Wang Oleksii Kuchaiev Olivier Delalleau T. Zhao AAML 151 2 0 08 Apr 2025
FindTheFlaws: Annotated Errors for Detecting Flawed Reasoning and Scalable Oversight Research Gabriel Recchia Chatrik Singh Mangat Issac Li Gayatri Krishnakumar ALM 170 0 0 29 Mar 2025
Probabilistic Uncertain Reward Model Wangtao Sun Xiang Cheng Xing Yu Haotian Xu Zhao Yang Shizhu He Jun Zhao Kang Liu 146 0 0 28 Mar 2025
VisTa: Visual-contextual and Text-augmented Zero-shot Object-level OOD Detection Bin Zhang Xiaoyang Qu Guokuan Li Jiguang Wan Jianzong Wang VLM 133 0 0 28 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 197 175 0 18 Mar 2025
From Autonomous Agents to Integrated Systems, A New Paradigm: Orchestrated Distributed Intelligence Krti Tallam AI4CE 84 2 0 17 Mar 2025
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models Teng Wang Zhangyi Jiang Zhenqi He Wenhan Yang Yanan Zheng Zeyu Li Zifan He Shenyang Tong Hailei Gong LRM 137 2 0 16 Mar 2025
Fine-Tuning Diffusion Generative Models via Rich Preference Optimization Hanyang Zhao Haoxian Chen Yucheng Guo Genta Indra Winata Tingting Ou Ziyu Huang D. Yao Wenpin Tang 114 0 0 13 Mar 2025
Robust Concept Erasure Using Task Vectors Minh Pham Kelly O. Marshall Chinmay Hegde Niv Cohen 173 20 0 21 Feb 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang Haizhou Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 212 10 0 28 Jan 2025
Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning Zijian Guo Weichao Zhou Wenchao Li OffRL 140 2 0 28 Jan 2025
Constrained Hybrid Metaheuristic Algorithm for Probabilistic Neural Networks Learning Piotr A. Kowalski Szymon Kucharczyk Jacek Mańdziuk 80 0 0 28 Jan 2025
MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking Sebastian Farquhar Vikrant Varma David Lindner David Elson Caleb Biddulph Ian Goodfellow Rohin Shah 155 2 0 22 Jan 2025
Episodic memory in AI agents poses risks that should be studied and mitigated Chad DeChant 122 3 0 20 Jan 2025
Two Types of AI Existential Risk: Decisive and Accumulative Atoosa Kasirzadeh 100 17 0 20 Jan 2025
Learning to Assist Humans without Inferring Rewards Vivek Myers Evan Ellis Sergey Levine Benjamin Eysenbach Anca Dragan 110 4 0 17 Jan 2025
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment Chaoqi Wang Zhuokai Zhao Yibo Jiang Zhaorun Chen Chen Zhu ... Jiayi Liu Lizhu Zhang Xiangjun Fan Hao Ma Sinong Wang 126 5 0 16 Jan 2025
URSA: Understanding and Verifying Chain-of-thought Reasoning in Multimodal Mathematics Ruilin Luo Zhuofan Zheng Yifan Wang Xinzhe Ni Zicheng Lin ... Yiyao Yu C. Shi Ruihang Chu Jin Zeng Yujiu Yang LRM 139 24 0 08 Jan 2025
FairSense: Long-Term Fairness Analysis of ML-Enabled Systems Yining She Sumon Biswas Christian Kastner Eunsuk Kang 103 0 0 03 Jan 2025
Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications Sinan Ibrahim Mostafa Mostafa Ali Jnadi Hadi Salloum Pavel Osinenko OffRL 104 16 0 31 Dec 2024
Uncertainty quantification for improving radiomic-based models in radiation pneumonitis prediction Chanon Puttanawarut Romen Samuel Wabina Nat Sirirutbunkajorn AI4CE 90 0 0 27 Dec 2024
ProcessBench: Identifying Process Errors in Mathematical Reasoning Chujie Zheng Zizhuo Zhang Beichen Zhang Runji Lin Keming Lu Bowen Yu Dayiheng Liu Jingren Zhou Junyang Lin LRM 176 72 0 09 Dec 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 447 3 0 11 Oct 2024
TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees Weibin Liao Xu Chu Yasha Wang LRM 106 7 0 10 Oct 2024
Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering Joris Postmus Steven Abreu LLMSV 301 3 0 09 Oct 2024
Moral Alignment for LLM Agents Elizaveta Tennant Stephen Hailes Mirco Musolesi 89 5 0 02 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 133 4 0 02 Oct 2024
Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction Melkamu Mersha Khang Lam Joseph Wood Ali AlShami Jugal Kalita XAI AI4TS 275 33 0 30 Aug 2024
Non-maximizing policies that fulfill multi-criterion aspirations in expectation Simon Dima Simon Fischer J. Heitzig Joss Oliver 77 1 0 08 Aug 2024
Need of AI in Modern Education: in the Eyes of Explainable AI (xAI) Supriya Manna Dionis Barcari 109 3 0 31 Jul 2024
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift Seongho Son William Bankes Sayak Ray Chowdhury Brooks Paige Ilija Bogunovic 88 4 0 26 Jul 2024
From Distributional to Overton Pluralism: Investigating Large Language Model Alignment Thom Lake Eunsol Choi Greg Durrett 96 12 0 25 Jun 2024
Learning Task Decomposition to Assist Humans in Competitive Programming Jiaxin Wen Ruiqi Zhong Pei Ke Zhihong Shao Hongning Wang Minlie Huang ReLM 90 9 0 07 Jun 2024
Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise T. Pouplin Alan Jeffares Nabeel Seedat Mihaela van der Schaar 434 3 0 05 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 159 5 0 05 Jun 2024
Safe Reinforcement Learning in Black-Box Environments via Adaptive Shielding Daniel Bethell Simos Gerasimou R. Calinescu Calum Imrie OffRL OnRL 121 0 0 28 May 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 161 39 0 02 Feb 2024
Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods Yotam Wolf Noam Wies Dorin Shteyman Binyamin Rothberg Yoav Levine Amnon Shashua LLMSV 109 13 0 29 Jan 2024
CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observations Xuzhe Dang Stefan Edelkamp 129 4 0 06 Nov 2023
Learning Personalized Decision Support Policies Umang Bhatt Valerie Chen Katherine M. Collins Parameswaran Kamalaruban Emma Kallina Adrian Weller Ameet Talwalkar OffRL 190 11 0 13 Apr 2023
Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities Zhaoyi Xu J. Saleh 65 346 0 19 Aug 2020
Reinforcement Learning with a Corrupted Reward Channel Tom Everitt Victoria Krakovna Laurent Orseau Marcus Hutter Shane Legg 90 102 0 23 May 2017
Learning Optimized Risk Scores Berk Ustun Cynthia Rudin 162 84 0 01 Oct 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,130 0 01 Jul 2016
Avoiding Imposters and Delinquents: Adversarial Crowdsourcing and Peer Prediction Jacob Steinhardt Gregory Valiant Moses Charikar 55 44 0 16 Jun 2016
Safe Exploration in Finite Markov Decision Processes with Gaussian Processes M. Turchetta Felix Berkenkamp Andreas Krause 84 189 0 15 Jun 2016
Estimating individual treatment effect: generalization bounds and algorithms Uri Shalit Fredrik D. Johansson David Sontag CML OOD 68 12 0 13 Jun 2016