Repetition In Repetition Out: Towards Understanding Neural Text
Degeneration from the Data Perspective

Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective

16 October 2023

Taro Watanabe

Papers citing "Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective"

9 / 9 papers shown

Title
Repetition Neurons: How Do Language Models Produce Repetitions? Tatsuya Hiraoka Kentaro Inui MILM 75 6 0 21 Feb 2025
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage Taewhoo Lee Chanwoong Yoon Kyochul Jang Donghyeon Lee Minju Song Hyunjae Kim Jaewoo Kang ELM 35 1 0 22 Oct 2024
Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models Seungduk Kim Seungtaek Choi Myeongho Jeong 33 6 0 22 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Locally Typical Sampling Clara Meister Tiago Pimentel Gian Wiher Ryan Cotterell 143 86 0 01 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 370 8,495 0 28 Jan 2022
Relating Neural Text Degeneration to Exposure Bias Ting-Rui Chiang Yun-Nung Chen 53 17 0 17 Sep 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 256 1,996 0 31 Dec 2020
A Theoretical Analysis of the Repetition Problem in Text Generation Z. Fu Wai Lam Anthony Man-Cho So Bei Shi 77 90 0 29 Dec 2020