Adding Instructions during Pretraining: Effective Way of Controlling
Toxicity in Language Models

Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models

14 February 2023

Shrimai Prabhumoye

Mohammad Shoeybi

Bryan Catanzaro

Papers citing "Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models"

11 / 11 papers shown

Title
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases Shrimai Prabhumoye Rafal Kocielnik Mohammad Shoeybi Anima Anandkumar Bryan Catanzaro 47 20 0 15 Dec 2021
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets Irene Solaiman Christy Dennison 63 224 0 18 Jun 2021
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts Alisa Liu Maarten Sap Ximing Lu Swabha Swayamdipta Chandra Bhagavatula Noah A. Smith Yejin Choi MU 85 369 0 07 May 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 378 2,051 0 31 Dec 2020
Case Study: Deontological Ethics in NLP Shrimai Prabhumoye Brendon Boldt Ruslan Salakhutdinov A. Black AILaw 31 28 0 09 Oct 2020
The Radicalization Risks of GPT-3 and Advanced Neural Language Models Kris McGuffie Alex Newhouse 47 150 0 15 Sep 2020
GeDi: Generative Discriminator Guided Sequence Generation Ben Krause Akhilesh Deepak Gotmare Bryan McCann N. Keskar Shafiq Joty R. Socher Nazneen Rajani 82 398 0 14 Sep 2020
Plug and Play Language Models: A Simple Approach to Controlled Text Generation Sumanth Dathathri Andrea Madotto Janice Lan Jane Hung Eric Frank Piero Molino J. Yosinski Rosanne Liu KELM 100 966 0 04 Dec 2019
Adversarial NLI: A New Benchmark for Natural Language Understanding Yixin Nie Adina Williams Emily Dinan Joey Tianyi Zhou Jason Weston Douwe Kiela 93 991 0 31 Oct 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 290 1,861 0 17 Sep 2019
Defending Against Neural Fake News Rowan Zellers Ari Holtzman Hannah Rashkin Yonatan Bisk Ali Farhadi Franziska Roesner Yejin Choi AAML 100 1,019 0 29 May 2019