Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

22 February 2024

Papers citing "Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs"

36 / 36 papers shown

Title
Boosting Performance on ARC is a Matter of Perspective Daniel Franzen Jan Disselhoff David Hartmann RALM LRM 49 0 0 08 May 2025
Geospatial Mechanistic Interpretability of Large Language Models Stef De Sabbata Stefano Mizzaro Kevin Roitero AI4CE 37 0 0 06 May 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 48 3 0 17 Mar 2025
Adversarial Tokenization Renato Lui Geh Zilei Shao Mathias Niepert SILM AAML 87 0 0 04 Mar 2025
Bringing Comparative Cognition To Computers Konstantinos Voudouris Lucy G. Cheke Eric Schulz ELM 78 0 0 04 Mar 2025
The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs Tanja Baeumel Josef van Genabith Simon Ostermann LRM 62 1 0 27 Feb 2025
Scaling LLM Pre-training with Vocabulary Curriculum Fangyuan Yu 78 2 0 25 Feb 2025
AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science Chenyue Li Wen Deng Mengqian Lu Binhang Yuan ELM AI4Cl LRM 90 0 0 03 Feb 2025
DateLogicQA: Benchmarking Temporal Biases in Large Language Models Gagan Bhatia MingZe Tang Cristina Mahanta Madiha Kazi 79 0 0 17 Dec 2024
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 46 2 0 06 Nov 2024
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 40 2 0 28 Oct 2024
Supervised Chain of Thought Xiang Zhang Dujian Ding LRM AI4CE 31 1 0 18 Oct 2024
Language Models Encode Numbers Using Digit Representations in Base 10 Amit Arnold Levy Mor Geva 24 5 0 15 Oct 2024
Grounding Partially-Defined Events in Multimodal Data Kate Sanders Reno Kriz David Etter Hannah Recknor Alexander Martin Cameron Carpenter Jingyang Lin Benjamin Van Durme 27 2 0 07 Oct 2024
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks Alex Cloud Jacob Goldman-Wetzler Evžen Wybitul Joseph Miller Alexander Matt Turner 30 2 0 06 Oct 2024
Large Language Models as Markov Chains Oussama Zekri Ambroise Odonnat Abdelhakim Benechehab Linus Bleistein Nicolas Boullé I. Redko 48 10 0 03 Oct 2024
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training Pavel Chizhov Catherine Arnett Elizaveta Korotkova Ivan P. Yamshchikov 48 2 0 06 Sep 2024
Where is the signal in tokenization space? Renato Lui Geh Honghua Zhang Kareem Ahmed Benjie Wang Mathias Niepert 30 4 0 16 Aug 2024
Can LLMs predict the convergence of Stochastic Gradient Descent? Hiroki Sakaji Abdelhakim Benechehab Wataru Kuramoto LRM 62 2 0 03 Aug 2024
Perceptions of Linguistic Uncertainty by Language Models and Humans Catarina G Belém Markelle Kelly M. Steyvers Sameer Singh Padhraic Smyth 43 3 0 22 Jul 2024
Improving Self Consistency in LLMs through Probabilistic Tokenization Ashutosh Sathe Divyanshu Aggarwal Sunayana Sitaram 42 4 0 04 Jul 2024
VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation Kun Qian Shunji Wan Claudia Tang Youzhi Wang Xuanming Zhang Maximillian Chen Zhou Yu AAML 45 8 0 25 Jun 2024
MatText: Do Language Models Need More than Text & Scale for Materials Modeling? Nawaf Alampara Santiago Miret Kevin Maik Jablonka 58 9 0 25 Jun 2024
Understanding and Mitigating Tokenization Bias in Language Models Buu Phan Marton Havasi Matthew Muckley Karen Ullrich 44 3 0 24 Jun 2024
Evaluating Numerical Reasoning in Text-to-Image Models Ivana Kajić Olivia Wiles Isabela Albuquerque Matthias Bauer Su Wang Jordi Pont-Tuset Aida Nematzadeh EGVM ReLM 90 0 0 20 Jun 2024
Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering Parag Jain Mirella Lapata 41 0 0 14 Jun 2024
VersiCode: Towards Version-controllable Code Generation Tongtong Wu Weigang Wu Xingyu Wang Kang Xu Suyu Ma Bo Jiang Ping Yang Zhenchang Xing Yuan-Fang Li Gholamreza Haffari 45 4 0 11 Jun 2024
Through the Thicket: A Study of Number-Oriented LLMs derived from Random Forest Models M. Romaszewski Przemysław Sekuła P. Głomb M. Cholewa Katarzyna Kołodziej 37 0 0 07 Jun 2024
Large Language Models as In-context AI Generators for Quality-Diversity Bryan Lim Manon Flageat Antoine Cully 37 4 0 24 Apr 2024
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge Khuyagbaatar Batsuren Ekaterina Vylomova Verna Dankers Tsetsuukhei Delgerbaatar Omri Uzan Yuval Pinter Gábor Bella 32 9 0 20 Apr 2024
Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions Leena Mathur Paul Pu Liang Louis-Philippe Morency LLMAG 38 7 0 17 Apr 2024
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning Eli Schwartz Leshem Choshen J. Shtok Sivan Doveh Leonid Karlinsky Assaf Arbelle 28 13 0 30 Mar 2024
Adversarial Math Word Problem Generation Roy Xie Chengxuan Huang Junlin Wang Bhuwan Dhingra AAML 33 1 0 27 Feb 2024
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,077 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 395 8,559 0 28 Jan 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 279 1,996 0 31 Dec 2020