ByT5: Towards a token-free future with pre-trained byte-to-byte models

28 May 2021

Sharan Narang

Papers citing "ByT5: Towards a token-free future with pre-trained byte-to-byte models"

50 / 110 papers shown

Title
Differentiating Emigration from Return Migration of Scholars Using Name-Based Nationality Detection Models Faeze Ghorbanpour Thiago Zordan Malaguth Aliakbar Akbaritabar 29 0 0 09 May 2025
Crosslingual Reasoning through Test-Time Scaling Zheng-Xin Yong Muhammad Farid Adilazuarda Jonibek Mansurov Ruochen Zhang Niklas Muennighoff Carsten Eickhoff Genta Indra Winata Julia Kreutzer Stephen H. Bach Alham Fikri Aji LRM ELM 157 0 0 08 May 2025
Token-free Models for Sarcasm Detection Sumit Mamtani Maitreya Sonawane Kanika Agarwal Nishanth Sanjeev 48 0 0 02 May 2025
RepText: Rendering Visual Text via Replicating Haozhao Wang Yongjun Xu Yongqian Li Jiajun Li Chaowei Zhang J. Wang Kejia Yang Z. Chen VLM 66 0 0 28 Apr 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 151 0 0 25 Mar 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 45 3 0 17 Mar 2025
Tokenization is Sensitive to Language Variation Anna Wegmann Dong Nguyen David Jurgens 84 1 0 24 Feb 2025
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs Longxu Dou Qian Liu Fan Zhou Changyu Chen Zili Wang ... Tianyu Pang Chao Du Xinyi Wan Wei Lu Min Lin 106 1 0 18 Feb 2025
Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers Markus J. Buehler AI4CE 35 1 0 04 Jan 2025
MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translation Langlin Huang Mengyu Bu Yang Feng 33 0 0 03 Nov 2024
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 37 2 0 28 Oct 2024
LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems Nan Xu Xuezhe Ma LRM 59 3 0 18 Oct 2024
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models Kushal Tatariya Vladimir Araujo Thomas Bauwens Miryam de Lhoneux VLM 35 0 0 15 Oct 2024
LeanAgent: Lifelong Learning for Formal Theorem Proving Adarsh Kumarappan Mo Tiwari Peiyang Song Robert Joseph George Chaowei Xiao Anima Anandkumar CLL LLMAG LRM 72 8 0 08 Oct 2024
Cogs in a Machine, Doing What They're Meant to Do -- The AMI Submission to the WMT24 General Translation Task Atli Jasonarson Hinrik Hafsteinsson Bjarki Ármannsson Steinþór Steingrímsson SyDa 32 2 0 04 Oct 2024
Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? Seth Aycock David Stap Di Wu Christof Monz Khalil Simaán 36 2 0 27 Sep 2024
Imagine yourself: Tuning-Free Personalized Image Generation Zecheng He Bo Sun Felix Juefei-Xu Haoyu Ma Ankit Ramchandani ... Ning Zhang Peizhao Zhang Roshan Sumbaly Peter Vajda Animesh Sinha DiffM 37 16 0 20 Sep 2024
Sample-Efficient Diffusion for Text-To-Speech Synthesis Justin Lovelace Soham Ray Kwangyoun Kim Kilian Q. Weinberger Felix Wu 36 2 0 01 Sep 2024
Advancing Post-OCR Correction: A Comparative Study of Synthetic Data Shuhao Guan Derek Greene 34 6 0 05 Aug 2024
GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models Jian Ma Yonglin Deng Chen Chen H. Lu Zhenyu Yang Zhenyu Yang VLM DiffM 97 6 0 02 Jul 2024
Wavelet-Based Image Tokenizer for Vision Transformers Zhenhai Zhu Radu Soricut ViT 50 3 0 28 May 2024
Identifying and Aligning Medical Claims Made on Social Media with Medical Evidence Anthony James Hughes Xingyi Song 18 1 0 18 May 2024
Neural Semantic Parsing with Extremely Rich Symbolic Meaning Representations Xiao Zhang Gosse Bouma Johan Bos NAI 33 0 0 19 Apr 2024
Gaining More Insight into Neural Semantic Parsing with Challenging Benchmarks Xiao Zhang Chunliu Wang Rik van Noord Johan Bos 33 3 0 12 Apr 2024
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation Marco Cognetta Tatsuya Hiraoka Naoaki Okazaki Rico Sennrich Yuval Pinter 29 2 0 30 Mar 2024
LUM-ViT: Learnable Under-sampling Mask Vision Transformer for Bandwidth Limited Optical Signal Acquisition Lingfeng Liu Dong Ni Hangjie Yuan ViT 35 0 0 03 Mar 2024
Advancing Generative AI for Portuguese with Open Decoder Gervásio PT* Rodrigo Santos Joao Silva Luís Gomes João Rodrigues António Branco 46 10 0 29 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
MELA: Multilingual Evaluation of Linguistic Acceptability Ziyin Zhang Yikang Liu Wei Huang Junyu Mao Rui Wang Hai Hu 24 3 0 15 Nov 2023
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval Nandan Thakur Jianmo Ni Gustavo Hernández Ábrego John Wieting Jimmy J. Lin Daniel Cer RALM 31 12 0 10 Nov 2023
Generating Pragmatic Examples to Train Neural Program Synthesizers Saujas Vaduguru Daniel Fried Yewen Pu NAI 26 5 0 09 Nov 2023
Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew Eylon Gueta Omer Goldman Reut Tsarfaty 11 1 0 01 Nov 2023
Analyzing Cognitive Plausibility of Subword Tokenization Lisa Beinborn Yuval Pinter 29 17 0 20 Oct 2023
Text-to-OverpassQL: A Natural Language Interface for Complex Geodata Querying of OpenStreetMap Michael Staniek Raphael Schumann Maike Zufle Stefan Riezler 32 6 0 30 Aug 2023
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Dustin Podell Zion English Kyle Lacey A. Blattmann Tim Dockhorn Jonas Muller Joe Penna Robin Rombach 94 2,128 0 04 Jul 2023
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora Svanhvít Lilja Ingólfsdóttir Pétur Orri Ragnarsson H. Jónsson Haukur Barri Símonarson Vilhjálmur Þorsteinsson Vésteinn Snæbjarnarson SyDa 32 9 0 29 May 2023
TranSFormer: Slow-Fast Transformer for Machine Translation Bei Li Yi Jing Xu Tan Zhen Xing Tong Xiao Jingbo Zhu 46 7 0 26 May 2023
Sāmayik: A Benchmark and Dataset for English-Sanskrit Translation Ayush Maheshwari Ashim Gupta Amrith Krishna Atul Kumar Singh Ganesh Ramakrishnan G. Anil Kumar Jitin Singla 27 0 0 23 May 2023
Exploring Large Language Models for Classical Philology Frederick Riemenschneider Anette Frank 19 15 0 23 May 2023
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models Peiqin Lin Chengzhi Hu Zheyu Zhang André F. T. Martins Hinrich Schütze 32 1 0 23 May 2023
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages Sebastian Ruder J. Clark Alexander Gutkin Mihir Kale Min Ma ... Dan Garrette R. Ingle Melvin Johnson Dmitry Panteleev Partha P. Talukdar ELM 22 38 0 19 May 2023
Language Model Tokenizers Introduce Unfairness Between Languages Aleksandar Petrov Emanuele La Malfa Philip H. S. Torr Adel Bibi 23 97 0 17 May 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 26 98 0 16 May 2023
What is the best recipe for character-level encoder-only modelling? Kris Cao 32 2 0 09 May 2023
Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages Sonal Sannigrahi Rachel Bawden 34 0 0 04 May 2023
Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages Verena Blaschke Hinrich Schütze Barbara Plank 39 14 0 20 Apr 2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining Hyung Won Chung Noah Constant Xavier Garcia Adam Roberts Yi Tay Sharan Narang Orhan Firat 23 50 0 18 Apr 2023
An Information Extraction Study: Take In Mind the Tokenization! Christos Theodoropoulos Marie-Francine Moens 29 6 0 27 Mar 2023
Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization Bashar Al-Rfooh Gheith A. Abandah Rami Al-Rfou 24 4 0 25 Mar 2023
An Overview on Language Models: Recent Developments and Outlook Chengwei Wei Yun Cheng Wang Bin Wang C.-C. Jay Kuo 25 42 0 10 Mar 2023