States Hidden in Hidden States: LLMs Emerge Discrete State
Representations Implicitly

States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly

16 July 2024

Zhiyuan Liu

Maosong Sun

ArXiv (abs)PDF HTML

Papers citing "States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly"

15 / 15 papers shown

Title
Do Large Language Models Latently Perform Multi-Hop Reasoning? Sohee Yang E. Gribovskaya Nora Kassner Mor Geva Sebastian Riedel ReLM LRM 122 113 0 26 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 167 1,287 0 05 Feb 2024
Zephyr: Direct Distillation of LM Alignment Lewis Tunstall E. Beeching Nathan Lambert Nazneen Rajani Kashif Rasul ... Nathan Habib Nathan Sarrazin Omar Sanseviero Alexander M. Rush Thomas Wolf ALM 107 398 0 25 Oct 2023
Linearity of Relation Decoding in Transformer Language Models Evan Hernandez Arnab Sen Sharma Tal Haklay Kevin Meng Martin Wattenberg Jacob Andreas Yonatan Belinkov David Bau KELM 82 100 0 17 Aug 2023
TheoremQA: A Theorem-driven Question Answering dataset Wenhu Chen Ming Yin Max Ku Pan Lu Yixin Wan Xueguang Ma Jianyu Xu Xinyi Wang Tony Xia AIMat 104 139 0 21 May 2023
Towards Automated Circuit Discovery for Mechanistic Interpretability Arthur Conmy Augustine N. Mavor-Parker Aengus Lynch Stefan Heimersheim Adrià Garriga-Alonso 66 319 0 28 Apr 2023
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 153 386 0 07 Dec 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 350 4,596 0 27 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,665 0 07 Jul 2021
Implicit Representations of Meaning in Neural Language Models Belinda Z. Li Maxwell Nye Jacob Andreas NAI MILM 67 177 0 01 Jun 2021
Generalizing to Unseen Domains: A Survey on Domain Generalization Jindong Wang Cuiling Lan Chang-Shu Liu Yidong Ouyang Tao Qin Wang Lu Yiqiang Chen Wenjun Zeng Philip S. Yu OOD 249 1,233 0 02 Mar 2021
Probing Classifiers: Promises, Shortcomings, and Advances Yonatan Belinkov 303 456 0 24 Feb 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 182 847 0 29 Dec 2020
Understanding the Role of Individual Units in a Deep Neural Network David Bau Jun-Yan Zhu Hendrik Strobelt Àgata Lapedriza Bolei Zhou Antonio Torralba GAN 72 453 0 10 Sep 2020
Designing and Interpreting Probes with Control Tasks John Hewitt Percy Liang 86 538 0 08 Sep 2019