Future Lens: Anticipating Subsequent Tokens from a Single Hidden State

8 November 2023

Papers citing "Future Lens: Anticipating Subsequent Tokens from a Single Hidden State"

20 / 20 papers shown

Title
Modes of Sequence Models and Learning Coefficients Zhongtian Chen Daniel Murfet 90 1 0 25 Apr 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 55 0 0 13 Mar 2025
Do Multilingual LLMs Think In English? Lisa Schut Y. Gal Sebastian Farquhar 44 3 0 24 Feb 2025
Discovering Chunks in Neural Embeddings for Interpretability Shuchen Wu Stephan Alaniz Eric Schulz Zeynep Akata 47 0 0 03 Feb 2025
The Geometry of Tokens in Internal Representations of Large Language Models Karthik Viswanathan Yuri Gardinazzi Giada Panerai Alberto Cazzaniga Matteo Biagetti AIFin 94 4 0 17 Jan 2025
Transformers Use Causal World Models in Maze-Solving Tasks Alex F Spies William Edwards Michael Ivanitskiy Adrians Skapars Tilman Rauker Katsumi Inoue A. Russo Murray Shanahan 155 1 0 16 Dec 2024
All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling Emanuele Marconato Sébastien Lachapelle Sebastian Weichwald Luigi Gresele 69 3 0 30 Oct 2024
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 50 12 0 08 Oct 2024
Extracting Paragraphs from LLM Token Activations Nicholas Pochinkov Angelo Benoit Lovkush Agarwal Zainab Ali Majid Lucile Ter-Minassian 32 1 0 10 Sep 2024
On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs Nitay Calderon Roi Reichart 42 10 0 27 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 82 19 0 02 Jul 2024
Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs Jannik Kossen Jiatong Han Muhammed Razzak Lisa Schut Shreshth A. Malik Yarin Gal HILM 60 35 0 22 Jun 2024
Better & Faster Large Language Models via Multi-token Prediction Fabian Gloeckle Badr Youbi Idrissi Baptiste Rozière David Lopez-Paz Gabriele Synnaeve 26 94 0 30 Apr 2024
Does Transformer Interpretability Transfer to RNNs? Gonccalo Paulo Thomas Marshall Nora Belrose 63 6 0 09 Apr 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 37 63 0 11 Mar 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 34 88 0 11 Jan 2024
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers Anna Langedijk Hosein Mohebbi Gabriele Sarti Willem H. Zuidema Jaap Jumelet 32 10 0 05 Oct 2023
Finding Neurons in a Haystack: Case Studies with Sparse Probing Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii Dimitris Bertsimas MILM 162 188 0 02 May 2023
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 282 1,996 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,824 0 14 Dec 2020