A Holistic Approach to Undesired Content Detection in the Real World

5 August 2022

Tyna Eloundou

Papers citing "A Holistic Approach to Undesired Content Detection in the Real World"

43 / 43 papers shown

Title
GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning Yong-Jin Liu Shengfang Zhai Mingzhe Du Yulin Chen Tri Cao ... Xuzhao Li Kun Wang Junfeng Fang Jiaheng Zhang Bryan Hooi OffRL LRM 16 0 0 16 May 2025
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 31 0 0 11 May 2025
LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities Kalyan Nakka Jimmy Dani Ausmit Mondal Nitesh Saxena AAML 35 0 0 08 May 2025
Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs Sai Krishna Mendu Harish Yenala Aditi Gulati Shanu Kumar Parag Agrawal 36 0 0 04 May 2025
aiXamine: Simplified LLM Safety and Security Fatih Deniz Dorde Popovic Yazan Boshmaf Euisuh Jeong M. Ahmad Sanjay Chawla Issa M. Khalil ELM 80 0 0 21 Apr 2025
Alleviating the Fear of Losing Alignment in LLM Fine-tuning Kang Yang Guanhong Tao X. Chen Jun Xu 36 0 0 13 Apr 2025
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng Chenyu Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 111 2 0 07 Mar 2025
SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models Seanie Lee Dong Bok Lee Dominik Wagner Minki Kang Haebin Seong Tobias Bocklet Juho Lee Sung Ju Hwang 12 1 0 18 Feb 2025
FLAME: Flexible LLM-Assisted Moderation Engine Ivan Bakulin Ilia Kopanichuk Iaroslav Bespalov Nikita Radchenko V. Shaposhnikov Dmitry Dylov Ivan Oseledets 98 0 0 13 Feb 2025
Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences Shanshan Han Salman Avestimehr Chaoyang He 76 1 0 12 Feb 2025
GuardReasoner: Towards Reasoning-based LLM Safeguards Yue Liu Hongcheng Gao Shengfang Zhai Jun Xia Tianyi Wu Zhiwei Xue Yuxiao Chen Kenji Kawaguchi Jiaheng Zhang Bryan Hooi AI4TS LRM 131 16 0 30 Jan 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 112 10 0 28 Jan 2025
Digital Guardians: Can GPT-4, Perspective API, and Moderation API reliably detect hate speech in reader comments of German online newspapers? Manuel Weber Moritz Huber Maximilian Auch Alexander Döschl Max-Emanuel Keller P. Mandl 37 0 0 03 Jan 2025
A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection Gabriel Chua Shing Yee Chan Shaun Khoo 83 1 0 20 Nov 2024
On Calibration of LLM-based Guard Models for Reliable Content Moderation Hongfu Liu Hengguan Huang Hao Wang Xiangming Gu Ye Wang 60 2 0 14 Oct 2024
Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond Shanshan Han 87 1 0 09 Oct 2024
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models Seanie Lee Haebin Seong Dong Bok Lee Minki Kang Xiaoyin Chen Dominik Wagner Yoshua Bengio Juho Lee Sung Ju Hwang 67 3 0 02 Oct 2024
Identity-related Speech Suppression in Generative AI Content Moderation Oghenefejiro Isaacs Anigboro Charlie M. Crawford Danaë Metaxa Sorelle A. Friedler Sorelle A. Friedler 26 0 0 09 Sep 2024
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning Zhen Xiang Linzhi Zheng Yanjie Li Junyuan Hong Qinbin Li ... Zidi Xiong Chulin Xie Carl Yang Dawn Song Bo Li LLMAG 45 24 0 13 Jun 2024
Enhancing Jailbreak Attack Against Large Language Models through Silent Tokens Jiahao Yu Haozheng Luo Jerry Yao-Chieh Hu Wenbo Guo Han Liu Xinyu Xing 40 19 0 31 May 2024
OR-Bench: An Over-Refusal Benchmark for Large Language Models Justin Cui Wei-Lin Chiang Ion Stoica Cho-Jui Hsieh ALM 38 35 0 31 May 2024
Fennec: Fine-grained Language Model Evaluation and Correction Extended through Branching and Bridging Xiaobo Liang Haoke Zhang Helan hu Juntao Li Jun Xu Min Zhang ALM 49 2 0 20 May 2024
ContextQ: Generated Questions to Support Meaningful Parent-Child Dialogue While Co-Reading Griffin Dietz Smith Siddhartha Prasad Matt J. Davidson Leah Findlater R. Benjamin Shapiro 40 5 0 06 May 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 53 6 0 12 Apr 2024
Comprehensive Assessment of Jailbreak Attacks Against LLMs Junjie Chu Yugeng Liu Ziqing Yang Xinyue Shen Michael Backes Yang Zhang AAML 40 67 0 08 Feb 2024
All in How You Ask for It: Simple Black-Box Method for Jailbreak Attacks Kazuhiro Takemoto 42 21 0 18 Jan 2024
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations Hakan Inan Kartikeya Upasani Jianfeng Chi Rashi Rungta Krithika Iyer ... Michael Tontchev Qing Hu Brian Fuller Davide Testuggine Madian Khabsa AI4MH 36 375 0 07 Dec 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 143 120 0 09 Nov 2023
"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models Xinyue Shen Zhenpeng Chen Michael Backes Yun Shen Yang Zhang SILM 40 249 0 07 Aug 2023
The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations A. Salinas Parth Vipul Shah Yuzhong Huang Robert McCormack Fred Morstatter 34 34 0 03 Aug 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 47 128 0 26 Jul 2023
HateModerate: Testing Hate Speech Detectors against Content Moderation Policies Jiangrui Zheng Xueqing Liu Guanqun Yang Mirazul Haque Xing Qian Ravishka Rathnasuriya Wei Yang G. Budhrani 47 3 0 23 Jul 2023
ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation Javier García Gilabert Carlos Escolano Marta R. Costa-jussá CLL MU 29 2 0 19 May 2023
Multi-step Jailbreaking Privacy Attacks on ChatGPT Haoran Li Dadi Guo Wei Fan Mingshi Xu Jie Huang Fanpu Meng Yangqiu Song SILM 62 322 0 11 Apr 2023
Secret-Keeping in Question Answering Nathaniel W. Rollings Kent O'Sullivan Sakshum Kulshrestha KELM 30 0 0 16 Mar 2023
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks Daniel Kang Xuechen Li Ion Stoica Carlos Guestrin Matei A. Zaharia Tatsunori Hashimoto AAML 30 237 0 11 Feb 2023
Ignore Previous Prompt: Attack Techniques For Language Models Fábio Perez Ian Ribeiro SILM 51 402 0 17 Nov 2022
Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages Paul Röttger Debora Nozza Federico Bianchi Dirk Hovy 29 10 0 20 Oct 2022
Towards Zero-Label Language Learning Zirui Wang Adams Wei Yu Orhan Firat Yuan Cao SyDa 188 102 0 19 Sep 2021
Challenges in Detoxifying Language Models Johannes Welbl Amelia Glaese J. Uesato Sumanth Dathathri John F. J. Mellor Lisa Anne Hendricks Kirsty Anderson Pushmeet Kohli Ben Coppin Po-Sen Huang LM&MA 250 193 0 15 Sep 2021
Towards generalisable hate speech detection: a review on obstacles and solutions Wenjie Yin A. Zubiaga 117 164 0 17 Feb 2021
Data Augmentation using Pre-trained Transformer Models Varun Kumar Ashutosh Choudhary Eunah Cho VLM 216 315 0 04 Mar 2020
Generating Natural Language Adversarial Examples M. Alzantot Yash Sharma Ahmed Elgohary Bo-Jhang Ho Mani B. Srivastava Kai-Wei Chang AAML 258 915 0 21 Apr 2018