Learning by Distilling Context

Learning by Distilling Context

30 September 2022

Charles Burton Snell

Ruiqi Zhong

Papers citing "Learning by Distilling Context"

15 / 15 papers shown

Title
Training Plug-n-Play Knowledge Modules with Deep Context Distillation Lucas Caccia Alan Ansell E. Ponti Ivan Vulić Alessandro Sordoni SyDa 187 0 0 11 Mar 2025
In-Context Learning Distillation for Efficient Few-Shot Fine-Tuning Yifei Duan Liu Li Zirui Zhai Jinxia Yao 78 0 0 17 Dec 2024
Efficient LLM Context Distillation Rajesh Upadhayayaya Zachary Smith Chritopher Kottmyer Manish Raj Osti 47 1 0 03 Sep 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 43 22 0 18 Jun 2024
In-Context Editing: Learning Knowledge from Self-Induced Distributions Siyuan Qi Bangcheng Yang Kailin Jiang Xiaobo Wang Jiaqi Li Yifan Zhong Yaodong Yang Zilong Zheng KELM 109 8 0 17 Jun 2024
RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment Kevin Kaichuang Yang Dan Klein Asli Celikyilmaz Nanyun Peng Yuandong Tian ALM 34 30 0 24 Jul 2023
Learning to Compress Prompts with Gist Tokens Jesse Mu Xiang Lisa Li Noah D. Goodman VLM 44 206 0 17 Apr 2023
Discovering Language Model Behaviors with Model-Written Evaluations Ethan Perez Sam Ringer Kamilė Lukošiūtė Karina Nguyen Edwin Chen ... Danny Hernandez Deep Ganguli Evan Hubinger Nicholas Schiefer Jared Kaplan ALM 22 364 0 19 Dec 2022
Large Language Models Can Self-Improve Jiaxin Huang S. Gu Le Hou Yuexin Wu Xuezhi Wang Hongkun Yu Jiawei Han ReLM AI4MH LRM 47 564 0 20 Oct 2022
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model Jacob Eisenstein D. Andor Bernd Bohnet Michael Collins David M. Mimno LRM 191 24 0 05 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 389 8,495 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Meta-learning via Language Model In-context Tuning Yanda Chen Ruiqi Zhong Sheng Zha George Karypis He He 234 156 0 15 Oct 2021
Towards Zero-Label Language Learning Zirui Wang Adams Wei Yu Orhan Firat Yuan Cao SyDa 188 102 0 19 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 261 4,489 0 23 Jan 2020