Gradient-Based Constrained Sampling from Language Models

25 May 2022

Papers citing "Gradient-Based Constrained Sampling from Language Models"

50 / 64 papers shown

Title
Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo João Loula Benjamin LeBrun Li Du Ben Lipkin Clemente Pasti ... Ryan Cotterel Vikash K. Mansinghka Alexander K. Lew Tim Vieira Timothy J. O'Donnell 109 6 0 17 Apr 2025
Sequence-level Large Language Model Training with Contrastive Preference Optimization Zhili Feng Dhananjay Ram Cole Hawkins Aditya Rawal Jinman Zhao Sheng Zha 90 1 0 23 Feb 2025
MultiQ&A: An Analysis in Measuring Robustness via Automated Crowdsourcing of Question Perturbations and Answers Nicole Cho William Watson AAML HILM 241 0 0 06 Feb 2025
Non-Halting Queries: Exploiting Fixed Points in LLMs Ghaith Hammouri Kemal Derya B. Sunar 48 0 0 08 Oct 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 207 21 0 28 Feb 2024
A Geometric Notion of Causal Probing Clément Guerner Anej Svete Tianyu Liu Alex Warstadt Ryan Cotterell LLMSV 79 16 0 27 Jul 2023
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey Sachin Kumar Vidhisha Balachandran Lucille Njoo Antonios Anastasopoulos Yulia Tsvetkov ELM 117 90 0 14 Oct 2022
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 220 823 0 27 May 2022
Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation Kexin Yang Dayiheng Liu Wenqiang Lei Baosong Yang Mingfeng Xue Boxing Chen Jun Xie 55 29 0 28 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 360 6,854 0 13 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 174 1,608 0 07 Apr 2022
Mix and Match: Learning-free Controllable Text Generation using Energy Language Models Fatemehsadat Mireshghallah Kartik Goyal Taylor Berg-Kirkpatrick 58 80 0 24 Mar 2022
Controllable Natural Language Generation with Contrastive Prefixes Jing Qian Li Dong Yelong Shen Furu Wei Weizhu Chen 57 98 0 27 Feb 2022
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics Lianhui Qin Sean Welleck Daniel Khashabi Yejin Choi AI4CE 90 149 0 23 Feb 2022
NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics Ximing Lu Sean Welleck Peter West Liwei Jiang Jungo Kasai ... Lianhui Qin Youngjae Yu Rowan Zellers Noah A. Smith Yejin Choi 47 164 0 16 Dec 2021
Controlling Conditional Language Models without Catastrophic Forgetting Tomasz Korbak Hady ElSahar Germán Kruszewski Marc Dymetman CLL AI4CE 55 34 0 01 Dec 2021
A Plug-and-Play Method for Controlled Text Generation Damian Pascual Béni Egressy Clara Meister Ryan Cotterell Roger Wattenhofer 113 93 0 20 Sep 2021
Noisy Channel Language Model Prompting for Few-Shot Text Classification Sewon Min Michael Lewis Hannaneh Hajishirzi Luke Zettlemoyer VLM 74 219 0 09 Aug 2021
Controlled Text Generation as Continuous Optimization with Multiple Constraints Sachin Kumar Eric Malmi Aliaksei Severyn Yulia Tsvetkov BDL AI4CE 73 78 0 04 Aug 2021
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation Guangyi Liu Zichao Yang Tianhua Tao Xiaodan Liang Junwei Bao Zhen Li Bowen Zhou Shuguang Cui Zhiting Hu 46 20 0 29 Jun 2021
Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics V. Jayaram John Thickstun DiffM 55 25 0 17 May 2021
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts Alisa Liu Maarten Sap Ximing Lu Swabha Swayamdipta Chandra Bhagavatula Noah A. Smith Yejin Choi MU 98 371 0 07 May 2021
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics Artidoro Pagnoni Vidhisha Balachandran Yulia Tsvetkov HILM 260 310 0 27 Apr 2021
FUDGE: Controlled Text Generation With Future Discriminators Kevin Kaichuang Yang Dan Klein 93 332 0 12 Apr 2021
Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed Eric Luhman Troy Luhman DiffM 226 277 0 07 Jan 2021
A Distributional Approach to Controlled Text Generation Muhammad Khalifa Hady ElSahar Marc Dymetman 133 117 0 21 Dec 2020
Adversarial Semantic Collisions Congzheng Song Alexander M. Rush Vitaly Shmatikov AAML 59 52 0 09 Nov 2020
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction Xinyao Ma Maarten Sap Hannah Rashkin Yejin Choi 63 73 0 26 Oct 2020
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints Ximing Lu Peter West Rowan Zellers Ronan Le Bras Chandra Bhagavatula Yejin Choi NAI 74 146 0 24 Oct 2020
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning Lianhui Qin Vered Shwartz Peter West Chandra Bhagavatula Jena D. Hwang Ronan Le Bras Antoine Bosselut Yejin Choi OffRL LRM 107 80 0 12 Oct 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 133 1,194 0 24 Sep 2020
GeDi: Generative Discriminator Guided Sequence Generation Ben Krause Akhilesh Deepak Gotmare Bryan McCann N. Keskar Shafiq Joty R. Socher Nazneen Rajani 107 406 0 14 Sep 2020
CoCon: A Self-Supervised Approach for Controlled Text Generation Alvin Chan Yew-Soon Ong B. Pung Aston Zhang Jie Fu 61 86 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 729 41,894 0 28 May 2020
Imitation Attacks and Defenses for Black-box Machine Translation Systems Eric Wallace Mitchell Stern D. Song AAML 68 123 0 30 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 141 2,423 0 23 Apr 2020
Unsupervised Discovery of Implicit Gender Bias Anjalie Field Yulia Tsvetkov 57 49 0 17 Apr 2020
Minimizing FLOPs to Learn Efficient Sparse Representations Biswajit Paria Chih-Kuan Yeh Ian En-Hsu Yen N. Xu Pradeep Ravikumar Barnabás Póczós 64 68 0 12 Apr 2020
Plug and Play Language Models: A Simple Approach to Controlled Text Generation Sumanth Dathathri Andrea Madotto Janice Lan Jane Hung Eric Frank Piero Molino J. Yosinski Rosanne Liu KELM 123 969 0 04 Dec 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 241 10,815 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 404 20,114 0 23 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 218 7,498 0 02 Oct 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 454 1,725 0 18 Sep 2019
CTRL: A Conditional Transformer Language Model for Controllable Generation N. Keskar Bryan McCann Lav Varshney Caiming Xiong R. Socher AI4CE 111 1,239 0 11 Sep 2019
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 271 642 0 03 Sep 2019
Universal Adversarial Triggers for Attacking and Analyzing NLP Eric Wallace Shi Feng Nikhil Kandpal Matt Gardner Sameer Singh AAML SILM 112 865 0 20 Aug 2019
Neural Text Generation with Unlikelihood Training Sean Welleck Ilia Kulikov Stephen Roller Emily Dinan Kyunghyun Cho Jason Weston MU 45 575 0 12 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 582 24,422 0 26 Jul 2019
Generative Modeling by Estimating Gradients of the Data Distribution Yang Song Stefano Ermon SyDa DiffM 218 3,893 0 12 Jul 2019
Training Neural Machine Translation To Apply Terminology Constraints Georgiana Dinu Prashant Mathur Marcello Federico Yaser Al-Onaizan AI4CE 54 159 0 03 Jun 2019