A Simple Contrastive Learning Objective for Alleviating Neural Text
Degeneration

A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration

5 May 2022

Svitlana Vakulenko

Maarten de Rijke

Papers citing "A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration"

12 / 12 papers shown

Title
Teaching Models to Understand (but not Generate) High-risk Data Ryan Yixiang Wang Matthew Finlayson Luca Soldaini Swabha Swayamdipta Robin Jia 139 0 0 05 May 2025
Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss Jing Xu Andrew Lee Sainbayar Sukhbaatar Jason Weston 20 86 0 27 Dec 2023
Improving Summarization with Human Edits Zonghai Yao Benjamin J Schloss Sai P. Selvaraj 29 3 0 09 Oct 2023
SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking Chris Cundy Stefano Ermon 16 10 0 08 Jun 2023
Look-back Decoding for Open-Ended Text Generation Nan Xu Chunting Zhou Asli Celikyilmaz Xuezhe Ma 31 9 0 22 May 2023
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond Haw-Shiuan Chang Zonghai Yao Alolika Gon Hong-ye Yu Andrew McCallum 43 10 0 20 May 2023
Contrastive Learning Reduces Hallucination in Conversations Weiwei Sun Zhengliang Shi Shen Gao Pengjie Ren Maarten de Rijke Z. Ren 42 62 0 20 Dec 2022
The CRINGE Loss: Learning what language not to model Leonard Adolphs Tianyu Gao Jing Xu Kurt Shuster Sainbayar Sukhbaatar Jason Weston MU 23 35 0 10 Nov 2022
DIRECTOR: Generator-Classifiers For Supervised Language Modeling Kushal Arora Kurt Shuster Sainbayar Sukhbaatar Jason Weston VLM 30 40 0 15 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 125 203 0 16 Feb 2021
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 281 31,267 0 16 Jan 2013