The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces

17 October 2024

Ahmed Oumar El-Shangiti

Tatsuya Hiraoka

Benjamin Heinzerling

Kentaro Inui

Papers citing "The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces"

18 / 18 papers shown

Title
Number Representations in LLMs: A Computational Parallel to Human Perception H.V. AlquBoj Hilal AlQuabeh Velibor Bojkovic Tatsuya Hiraoka Ahmed Oumar El-Shangiti Munachiso Nwadike Kentaro Inui 58 0 0 22 Feb 2025
Interpreting and Improving Large Language Models in Arithmetic Calculation Wei Zhang Chaoqun Wan Yonggang Zhang Yiu-ming Cheung Xinmei Tian Xu Shen Jieping Ye LRM 78 20 0 03 Sep 2024
What does the Knowledge Neuron Thesis Have to do with Knowledge? Jingcheng Niu Andrew Liu Zining Zhu Gerald Penn 89 33 0 03 May 2024
Linearity of Relation Decoding in Transformer Language Models Evan Hernandez Arnab Sen Sharma Tal Haklay Kevin Meng Martin Wattenberg Jacob Andreas Yonatan Belinkov David Bau KELM 48 95 0 17 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 249 11,636 0 18 Jul 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 235 297 0 28 Apr 2023
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 180 1,308 0 10 Feb 2022
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models Nora Kassner Philipp Dufter Hinrich Schütze 67 135 0 01 Feb 2021
Probing Pretrained Language Models for Lexical Semantics Ivan Vulić Edoardo Ponti Robert Litschko Goran Glavaš Anna Korhonen KELM 63 238 0 12 Oct 2020
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries Benjamin Heinzerling Kentaro Inui KELM 48 129 0 20 Aug 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 552 41,106 0 28 May 2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 84 886 0 10 Feb 2020
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 109 1,396 0 28 Nov 2019
Do NLP Models Know Numbers? Probing Numeracy in Embeddings Eric Wallace Yizhong Wang Sujian Li Sameer Singh Matt Gardner 66 263 0 17 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 545 2,639 0 03 Sep 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.3K 93,936 0 11 Oct 2018
Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers Georgios P. Spithourakis Sebastian Riedel 48 83 0 21 May 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 154 11,520 0 15 Feb 2018