State Soup: In-Context Skill Learning, Retrieval and Mixing

State Soup: In-Context Skill Learning, Retrieval and Mixing

12 June 2024

Maciej Wołczyk

João Sacramento

ArXiv (abs)PDF HTML

Papers citing "State Soup: In-Context Skill Learning, Retrieval and Mixing"

10 / 10 papers shown

Title
PICASO: Permutation-Invariant Context Composition with State Space Models Tian Yu Liu Alessandro Achille Matthew Trager Aditya Golatkar Luca Zancato Stefano Soatto LRM 134 0 0 24 Feb 2025
Function Vectors in Large Language Models Eric Todd Millicent Li Arnab Sen Sharma Aaron Mueller Byron C. Wallace David Bau 55 120 0 23 Oct 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 85 156 0 07 Jun 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 326 297 0 11 Mar 2023
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 180 835 0 14 Apr 2022
Neural networks with late-phase weights J. Oswald Seijin Kobayashi Alexander Meulemans Christian Henning Benjamin Grewe João Sacramento 69 35 0 25 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 259 6,786 0 03 Sep 2014
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 402 33,560 0 16 Oct 2013

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.