State Soup: In-Context Skill Learning, Retrieval and Mixing

State Soup: In-Context Skill Learning, Retrieval and Mixing

12 June 2024

Maciej Wołczyk

João Sacramento

Papers citing "State Soup: In-Context Skill Learning, Retrieval and Mixing"

4 / 4 papers shown

Title
PICASO: Permutation-Invariant Context Composition with State Space Models Tian Yu Liu Alessandro Achille Matthew Trager Aditya Golatkar L. Zancato Stefano Soatto LRM 62 0 0 24 Feb 2025
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 58 117 0 29 Feb 2024
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 88 266 0 11 Mar 2023
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 231 4,469 0 23 Jan 2020