B'MOJO: Hybrid State Space Realizations of Foundation Models with
Eidetic and Fading Memory

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory

8 July 2024

Aditya Golatkar

Benjamin Bowman

Alessandro Achille

Papers citing "B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory"

9 / 9 papers shown

Title
PICASO: Permutation-Invariant Context Composition with State Space Models Tian Yu Liu Alessandro Achille Matthew Trager Aditya Golatkar L. Zancato Stefano Soatto LRM 62 0 0 24 Feb 2025
Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models Elvis Nunez L. Zancato Benjamin Bowman Aditya Golatkar W. Xia Stefano Soatto 81 2 0 17 Dec 2024
Marconi: Prefix Caching for the Era of Hybrid LLMs Rui Pan Zhuang Wang Zhen Jia Can Karakus Luca Zancato Tri Dao Ravi Netravali Yida Wang 95 4 0 28 Nov 2024
Mechanistic Design and Scaling of Hybrid Architectures Michael Poli Armin W. Thomas Eric N. D. Nguyen Pragaash Ponnusamy Bjorn Deiseroth ... Brian Hie Stefano Ermon Christopher Ré Ce Zhang Stefano Massaroli MoE 57 21 0 26 Mar 2024
Multi-Modal Hallucination Control by Visual Information Grounding Alessandro Favero L. Zancato Matthew Trager Siddharth Choudhary Pramuditha Perera Alessandro Achille Ashwin Swaminathan Stefano Soatto MLLM 87 62 0 20 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 61 117 0 29 Feb 2024
Simple linear attention language models balance the recall-throughput tradeoff Simran Arora Sabri Eyuboglu Michael Zhang Aman Timalsina Silas Alberti Dylan Zinsley James Zou Atri Rudra Christopher Ré 46 18 0 28 Feb 2024
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 250 460 0 24 Sep 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 261 4,489 0 23 Jan 2020