Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks

19 December 2022

Papers citing "Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks"

27 / 27 papers shown

Title
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 206 11,636 0 18 Jul 2023
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 146 3,072 0 20 Oct 2022
CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing Andrew Rosenbaum Saleh Soltan Wael Hamza Amir Saffari Macro Damonte Isabel Groves 52 32 0 13 Oct 2022
Generate rather than Retrieve: Large Language Models are Strong Context Generators Wenhao Yu Dan Iter Shuohang Wang Yichong Xu Mingxuan Ju Soumya Sanyal Chenguang Zhu Michael Zeng Meng Jiang RALM AIMat 281 328 0 21 Sep 2022
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization Yi Tay Vinh Q. Tran Sebastian Ruder Jai Gupta Hyung Won Chung Dara Bahri Zhen Qin Simon Baumgartner Cong Yu Donald Metzler 78 155 0 23 Jun 2021
How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation Marco Gaido Beatrice Savoldi L. Bentivogli Matteo Negri Marco Turchi 87 15 0 28 May 2021
ByT5: Towards a token-free future with pre-trained byte-to-byte models Linting Xue Aditya Barua Noah Constant Rami Al-Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel 73 494 0 28 May 2021
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering Gautier Izacard Edouard Grave RALM 102 1,141 0 02 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 500 41,106 0 28 May 2020
Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation Xuanli He Gholamreza Haffari Mohammad Norouzi 44 45 0 03 May 2020
UnifiedQA: Crossing Format Boundaries With a Single QA System Daniel Khashabi Sewon Min Tushar Khot Ashish Sabharwal Oyvind Tafjord Peter Clark Hannaneh Hajishirzi 105 731 0 02 May 2020
Injecting Numerical Reasoning Skills into Language Models Mor Geva Ankit Gupta Jonathan Berant AIMat LRM 50 224 0 09 Apr 2020
Byte Pair Encoding is Suboptimal for Language Model Pretraining Kaj Bostrom Greg Durrett 44 205 0 07 Apr 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 141 10,720 0 29 Oct 2019
BPE-Dropout: Simple and Effective Subword Regularization Ivan Provilkov Dmitrii Emelianenko Elena Voita 53 279 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 270 19,824 0 23 Oct 2019
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension Adam Fisch Alon Talmor Robin Jia Minjoon Seo Eunsol Choi Danqi Chen 52 304 0 22 Oct 2019
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 150 3,490 0 19 Aug 2018
Character-Level Language Modeling with Deeper Self-Attention Rami Al-Rfou Dokook Choe Noah Constant Mandy Guo Llion Jones 96 388 0 09 Aug 2018
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates Taku Kudo 153 1,153 0 29 Apr 2018
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension Amrita Saha Rahul Aralikatte Mitesh M. Khapra Karthik Sankaranarayanan 49 197 0 21 Apr 2018
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension Mandar Joshi Eunsol Choi Daniel S. Weld Luke Zettlemoyer RALM 173 2,610 0 09 May 2017
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine Matthew Dunn Levent Sagun Mike Higgins V. U. Güney Volkan Cirik Kyunghyun Cho RALM 65 455 0 18 Apr 2017
NewsQA: A Machine Comprehension Dataset Adam Trischler Tong Wang Xingdi Yuan Justin Harris Alessandro Sordoni Philip Bachman Kaheer Suleman 68 891 0 29 Nov 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 162 8,067 0 16 Jun 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 157 7,683 0 31 Aug 2015
Generating Sequences With Recurrent Neural Networks Alex Graves GAN 111 4,025 0 04 Aug 2013