ProphetNet: Predicting Future N-gram for Sequence-to-Sequence
Pre-training

ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

13 January 2020

Dayiheng Liu

Papers citing "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training"

18 / 18 papers shown

Title
L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models Xiaohao Liu Xiaobo Xia Weixiang Zhao Manyi Zhang Xianzhi Yu Xiu Su Shuo Yang See-Kiong Ng Tat-Seng Chua KELM LRM 32 0 0 23 May 2025
Looking beyond the next token Abitha Thankaraj Yiding Jiang J. Zico Kolter Yonatan Bisk LRM 67 1 0 15 Apr 2025
VocalNet: Speech LLM with Multi-Token Prediction for Faster and High-Quality Generation Yuhao Wang Heyang Liu Ziyang Cheng Ronghua Wu Qunshan Gu Yanfeng Wang Yu Wang 289 3 0 05 Apr 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 68 6 0 17 Mar 2025
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search Kaixin Wu Yixin Ji Ziyang Chen Qiang Wang Cunxiang Wang ... Jia Xu Zhongyi Liu Jinjie Gu Yuan Zhou Linjian Mo KELM CLL 106 0 0 02 Dec 2024
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Zhiyu Li Dianbo Sui DiffM LRM 84 20 0 18 Oct 2024
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 146 2,029 0 18 Dec 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 219 6,420 0 26 Sep 2019
SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi Danqi Chen Yinhan Liu Daniel S. Weld Luke Zettlemoyer Omer Levy 96 1,953 0 24 Jul 2019
Defending Against Neural Fake News Rowan Zellers Ari Holtzman Hannah Rashkin Yonatan Bisk Ali Farhadi Franziska Roesner Yejin Choi AAML 79 1,012 0 29 May 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 104 1,553 0 08 May 2019
MASS: Masked Sequence to Sequence Pre-training for Language Generation Kaitao Song Xu Tan Tao Qin Jianfeng Lu Tie-Yan Liu 72 962 0 07 May 2019
Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia Xinya Du Claire Cardie KELM 37 161 0 15 May 2018
Learning to Decode for Future Success Jiwei Li Will Monroe Dan Jurafsky 46 59 0 23 Jan 2017
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 301 1,897 0 10 Jan 2017
SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents Ramesh Nallapati Feifei Zhai Bowen Zhou 241 1,257 0 14 Nov 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 89 8,067 0 16 Jun 2016
A Neural Attention Model for Abstractive Sentence Summarization Alexander M. Rush S. Chopra Jason Weston CVBM 62 2,695 0 02 Sep 2015