Investigating the Limitations of Transformers with Simple Arithmetic Tasks

25 February 2021

Papers citing "Investigating the Limitations of Transformers with Simple Arithmetic Tasks"

28 / 28 papers shown

Title
Recursive Decomposition with Dependencies for Generic Divide-and-Conquer Reasoning Sergio Hernández-Gutiérrez Minttu Alakuijala Alexander Nikitin Pekka Marttinen LRM 60 2 0 05 May 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 45 3 0 17 Mar 2025
Int2Int: a framework for mathematics with transformers François Charton ViT 46 0 0 22 Feb 2025
Mathematical Language Models: A Survey Wei Liu Hanglei Hu Jie Zhou Yuyang Ding Junsong Li ... Mengliang He Qin Chen Bo Jiang Aimin Zhou Liang He LRM 79 12 0 03 Jan 2025
MLissard: Multilingual Long and Simple Sequential Reasoning Benchmarks M. Bueno R. Lotufo Rodrigo Nogueira LRM 31 0 0 08 Oct 2024
Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice Jian-Qiao Zhu Haijiang Yan Thomas L. Griffiths 84 2 0 29 May 2024
OmniPred: Language Models as Universal Regressors Xingyou Song Oscar Li Chansoo Lee Bangding Yang Daiyi Peng Sagi Perel Yutian Chen 56 14 0 22 Feb 2024
Carrying over algorithm in transformers J. Kruthoff 24 0 0 15 Jan 2024
Positional Description Matters for Transformers Arithmetic Ruoqi Shen Sébastien Bubeck Ronen Eldan Yin Tat Lee Yuanzhi Li Yi Zhang 34 37 0 22 Nov 2023
Auto-Regressive Next-Token Predictors are Universal Learners Eran Malach LRM 24 36 0 13 Sep 2023
GPT Can Solve Mathematical Problems Without a Calculator Z. Yang Ming Ding Qingsong Lv Zhihuan Jiang Zehai He Yuyi Guo Jinfeng Bai Jie Tang RALM LRM 36 52 0 06 Sep 2023
It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models Xingcheng Xu Zihao Pan Haipeng Zhang Yanqing Yang LRM 18 2 0 16 Aug 2023
Arithmetic with Language Models: from Memorization to Computation Davide Maltoni Matteo Ferrara KELM LRM 38 5 0 02 Aug 2023
Evaluating Transformer Language Models on Arithmetic Operations Using Number Decomposition Matteo Muffo A. Cocco Enrico Bertino ReLM 20 25 0 21 Apr 2023
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 71 311 0 19 Dec 2022
Logical Tasks for Measuring Extrapolation and Rule Comprehension Ippei Fujisawa Ryota Kanai ELM LRM 28 4 0 14 Nov 2022
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems Jialiang Xu Mengyu Zhou Xinyi He Shi Han Dongmei Zhang AAML ELM AIMat 20 5 0 14 Nov 2022
Development of a Neural Network-Based Mathematical Operation Protocol for Embedded Hexadecimal Digits Using Neural Architecture Search (NAS) Victor Robila Kexin Pei Junfeng Yang 18 0 0 12 Nov 2022
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 46 156 0 19 Oct 2022
Limitations of Language Models in Arithmetic and Symbolic Induction Jingu Qian Hong Wang Zekun Li Shiyang Li Xifeng Yan ReLM LRM 40 74 0 09 Aug 2022
Innovations in Neural Data-to-text Generation: A Survey Mandar Sharma Ajay K. Gogineni Naren Ramakrishnan 32 10 0 25 Jul 2022
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence Myeongjun Jang Frank Mtumbuka Thomas Lukasiewicz 28 9 0 08 May 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 43 149 0 15 Apr 2022
AA-TransUNet: Attention Augmented TransUNet For Nowcasting Tasks Yimin Yang S. Mehrkanoon ViT AI4TS 34 41 0 10 Feb 2022
Linear algebra with transformers Franccois Charton AIMat 29 56 0 03 Dec 2021
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics Sean Welleck Peter West Jize Cao Yejin Choi 11 28 0 28 Sep 2021
Teaching Autoregressive Language Models Complex Tasks By Demonstration Gabriel Recchia 26 22 0 05 Sep 2021
Representing Numbers in NLP: a Survey and a Vision Avijit Thawani Jay Pujara Pedro A. Szekely Filip Ilievski 32 114 0 24 Mar 2021