PICASO: Permutation-Invariant Context Composition with State Space Models

24 February 2025

Papers citing "PICASO: Permutation-Invariant Context Composition with State Space Models"

28 / 28 papers shown

Title
B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory Luca Zancato Arjun Seshadri Yonatan Dukler Aditya Golatkar Yantao Shen Benjamin Bowman Matthew Trager Alessandro Achille Stefano Soatto 65 9 0 08 Jul 2024
State Soup: In-Context Skill Learning, Retrieval and Mixing Maciej Pióro Maciej Wołczyk Razvan Pascanu J. Oswald João Sacramento 43 1 0 12 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 141 66 0 11 Jun 2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality Tri Dao Albert Gu Mamba 97 502 0 31 May 2024
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection Yun Zhu Jia-Chen Gu Caitlin Sikora Ho Ko Yinxiao Liu ... Lei Shu Liangchen Luo Lei Meng Bang Liu Jindong Chen RALM 62 19 0 25 May 2024
Jamba: A Hybrid Transformer-Mamba Language Model Opher Lieber Barak Lenz Hofit Bata Gal Cohen Jhonathan Osin ... Nir Ratner N. Rozen Erez Shwartz Mor Zusman Y. Shoham 76 220 0 28 Mar 2024
In-context Learning with Retrieved Demonstrations for Language Models: A Survey an Luo Xin Xu Yue Liu Panupong Pasupat Mehran Kazemi RALM 116 69 0 21 Jan 2024
Retrieval-Augmented Generation for Large Language Models: A Survey Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun Meng Wang Haofen Wang 3DV RALM 150 1,767 1 18 Dec 2023
Mamba: Linear-Time Sequence Modeling with Selective State Spaces Albert Gu Tri Dao Mamba 146 2,670 0 01 Dec 2023
Tangent Transformers for Composition, Privacy and Removal Tian Yu Liu Aditya Golatkar Stefano Soatto 60 9 0 16 Jul 2023
Tangent Model Composition for Ensembling and Continual Fine-tuning Tianlin Liu Stefano Soatto LRM MoMe CLL 52 17 0 16 Jul 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 103 1,589 0 06 Jul 2023
Prompt Algebra for Task Composition Pramuditha Perera Matthew Trager Luca Zancato Alessandro Achille Stefano Soatto VLM 48 8 0 01 Jun 2023
In-Context Retrieval-Augmented Language Models Ori Ram Yoav Levine Itay Dalmedigos Dor Muhlgay Amnon Shashua Kevin Leyton-Brown Y. Shoham KELM RALM LRM 82 596 0 31 Jan 2023
Demystifying Prompts in Language Models via Perplexity Estimation Hila Gonen Srini Iyer Terra Blevins Noah A. Smith Luke Zettlemoyer LRM 115 210 0 08 Dec 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 138 981 1 10 Mar 2022
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 242 1,085 0 08 Dec 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 400 1,183 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 385 1,374 0 17 Jan 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 749 41,932 0 28 May 2020
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 144 1,792 0 26 Nov 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,193 0 27 Aug 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 168 2,468 0 19 May 2019
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering Todor Mihaylov Peter Clark Tushar Khot Ashish Sabharwal 110 1,528 0 08 Sep 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 158 2,587 0 14 Mar 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 690 131,526 0 12 Jun 2017
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset Payal Bajaj Daniel Fernando Campos Nick Craswell Li Deng Jianfeng Gao ... Mir Rosenberg Xia Song Alina Stoica Saurabh Tiwary Tong Wang RALM 139 2,724 0 28 Nov 2016
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 308 2,859 0 26 Sep 2016