In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

v1v2 (latest)

In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

3 March 2025

Thierry Etchegoyhen

ArXiv (abs)PDF HTML

Papers citing "In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models"

15 / 15 papers shown

Title
Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection Haoming Wang Boyuan Yang Xiangyu Yin Wei Gao 98 0 0 15 Apr 2025
EuroLLM: Multilingual Language Models for Europe Pedro Henrique Martins Patrick Fernandes Joao Alves Nuno M. Guerreiro Ricardo Rei ... Pierre Colombo Barry Haddow José G. C. de Souza Alexandra Birch André F. T. Martins 75 36 0 24 Sep 2024
Is In-Context Learning Sufficient for Instruction Following in LLMs? Hao Zhao Maksym Andriushchenko Francesco Croce Nicolas Flammarion 89 13 0 30 May 2024
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning Bill Yuchen Lin Abhilasha Ravichander Ximing Lu Nouha Dziri Melanie Sclar Khyathi Chandu Chandra Bhagavatula Yejin Choi 59 190 0 04 Dec 2023
In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning Xiaochuang Han 40 19 0 08 Aug 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 387 3,981 0 29 May 2023
LIMA: Less Is More for Alignment Chunting Zhou Pengfei Liu Puxin Xu Srini Iyer Jiao Sun ... Susan Zhang Gargi Ghosh M. Lewis Luke Zettlemoyer Omer Levy ALM 102 840 0 18 May 2023
Democratizing Neural Machine Translation with OPUS-MT Jörg Tiedemann Mikko Aulamo Daria Bakshandaeva M. Boggia Stig-Arne Gronroos Tommi Nieminen Alessandro Raganato Yves Scherrer Raúl Vázquez Sami Virpioja 68 32 0 04 Dec 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 392 2,388 0 09 Nov 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 880 12,973 0 04 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 163 1,485 0 25 Feb 2022
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 211 3,750 0 03 Sep 2021
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 140 2,555 0 22 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 817 42,055 0 28 May 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 608 4,822 0 23 Jan 2020