Extracting Paragraphs from LLM Token Activations

Extracting Paragraphs from LLM Token Activations

10 September 2024

Nicholas Pochinkov

Lovkush Agarwal

Zainab Ali Majid

Lucile Ter-Minassian

ArXiv (abs)PDF HTML

Papers citing "Extracting Paragraphs from LLM Token Activations"

13 / 13 papers shown

Title
Better & Faster Large Language Models via Multi-token Prediction Fabian Gloeckle Badr Youbi Idrissi Baptiste Rozière David Lopez-Paz Gabriele Synnaeve 99 117 0 30 Apr 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 105 114 0 11 Jan 2024
Language Models are Universal Embedders Xin Zhang Zehan Li Yanzhao Zhang Dingkun Long Pengjun Xie Meishan Zhang Min Zhang KELM ELM 244 7 0 12 Oct 2023
Towards Best Practices of Activation Patching in Language Models: Metrics and Methods Fred Zhang Neel Nanda LLMSV 189 112 0 27 Sep 2023
MTEB: Massive Text Embedding Benchmark Niklas Muennighoff Nouamane Tazi L. Magne Nils Reimers 520 404 0 13 Oct 2022
Extracting Latent Steering Vectors from Pretrained Language Models Nishant Subramani Nivedita Suresh Matthew E. Peters LLMSV 78 98 0 10 May 2022
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 425 1,041 0 17 Apr 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 161 828 0 29 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 811 42,055 0 28 May 2020
MPNet: Masked and Permuted Pre-training for Language Understanding Kaitao Song Xu Tan Tao Qin Jianfeng Lu Tie-Yan Liu 102 1,121 0 20 Apr 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 442 20,181 0 23 Oct 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,226 0 27 Aug 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018