v1v2 (latest)

Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles

11 October 2024

Itai Gat

Papers citing "Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles"

20 / 20 papers shown

Title
CharED: Character-wise Ensemble Decoding for Large Language Models Kevin Gu Eva Tuecke Dmitriy Katz R. Horesh David Alvarez-Melis Mikhail Yurochkin 49 2 0 25 Jun 2024
How to Compute the Probability of a Word Tiago Pimentel Clara Meister 82 20 0 20 Jun 2024
Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration Yi-Chong Huang Xiaocheng Feng Baohang Li Yang Xiang Hui Wang Bing Qin Ting Liu FedML 78 29 0 19 Apr 2024
MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling Tomasz Limisiewicz Terra Blevins Hila Gonen Orevaoghene Ahia Luke Zettlemoyer 78 17 0 15 Mar 2024
Token Alignment via Character Matching for Subword Completion Ben Athiwaratkun Shiqi Wang Mingyue Shang Yuchen Tian Zijian Wang Sujan Kumar Gonugondla Sanjay Krishna Gouda Rob Kwiatowski Ramesh Nallapati Bing Xiang 86 5 0 13 Mar 2024
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance Omer Goldman Avi Caciularu Matan Eyal Kris Cao Idan Szpektor Reut Tsarfaty 66 29 0 10 Mar 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 265 562 0 07 Mar 2024
Getting the most out of your tokenizer for pre-training and domain adaptation Gautier Dagan Gabriele Synnaeve Baptiste Rozière 91 26 0 01 Feb 2024
Knowledge Fusion of Large Language Models Fanqi Wan Xinting Huang Deng Cai Xiaojun Quan Wei Bi Shuming Shi MoMe 94 70 0 19 Jan 2024
Code Llama: Open Foundation Models for Code Baptiste Rozière Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat ... Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom Gabriel Synnaeve ELM ALM 113 2,055 0 24 Aug 2023
Tokenization and the Noiseless Channel Vilém Zouhar Clara Meister Juan Luis Gastaldi Li Du Mrinmaya Sachan Ryan Cotterell 64 37 0 29 Jun 2023
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion Dongfu Jiang Xiang Ren Bill Yuchen Lin ELM 73 320 0 05 Jun 2023
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers L. Yu Daniel Simig Colin Flaherty Armen Aghajanyan Luke Zettlemoyer M. Lewis 63 91 0 12 May 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,247 0 27 Feb 2023
Efficient Training of Language Models to Fill in the Middle Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman C. McLeavey Jerry Tworek Mark Chen 73 195 0 28 Jul 2022
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 808 42,055 0 28 May 2020
BPE-Dropout: Simple and Effective Subword Regularization Ivan Provilkov Dmitrii Emelianenko Elena Voita 79 286 0 29 Oct 2019
On NMT Search Errors and Model Errors: Cat Got Your Tongue? Felix Stahlberg Bill Byrne LRM 85 154 0 27 Aug 2019
The Curious Case of Neural Text Degeneration Ari Holtzman Jan Buys Li Du Maxwell Forbes Yejin Choi 187 3,184 0 22 Apr 2019
Six Challenges for Neural Machine Translation Philipp Koehn Rebecca Knowles AAML AIMat 373 1,225 0 12 Jun 2017