Meta-learning via Language Model In-context Tuning

15 October 2021

Yanda Chen

Ruiqi Zhong

Sheng Zha

George Karypis

He He

ArXiv PDF HTML

Papers citing "Meta-learning via Language Model In-context Tuning"

40 / 40 papers shown

Title
GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning Rui Lv Z. Zhang Kai Zhang Qi Liu Weibo Gao J. H. Liu Jiaxia Yan Linan Yue Fangzhou Yao 137 0 0 04 May 2025
Mathematical Language Models: A Survey W. Liu Hanglei Hu Jie Zhou Yuyang Ding Junsong Li ... Mengliang He Qin Chen Bo Jiang Aimin Zhou Liang He LRM 79 12 0 03 Jan 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 82 4 0 31 Dec 2024
In-Context Editing: Learning Knowledge from Self-Induced Distributions Siyuan Qi Bangcheng Yang Kailin Jiang Xiaobo Wang Jiaqi Li Yifan Zhong Yaodong Yang Zilong Zheng KELM 101 8 0 17 Jun 2024
MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning Sanchit Sinha Yuguang Yue Victor Soto Mayank Kulkarni Jianhua Lu Aidong Zhang LRM 34 4 0 19 May 2024
Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors Guozheng Li Peng Wang Jiajun Liu Yikai Guo Ke Ji Ziyu Shang Zijie Xu LRM 35 7 0 27 Apr 2024
ICL Markup: Structuring In-Context Learning using Soft-Token Tags Marc-Etienne Brunet Ashton Anderson R. Zemel 21 4 0 12 Dec 2023
Dolphins: Multimodal Language Model for Driving Yingzi Ma Yulong Cao Jiachen Sun Marco Pavone Chaowei Xiao MLLM 28 49 0 01 Dec 2023
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models Ruida Wang Wangchunshu Zhou Mrinmaya Sachan 19 32 0 20 Oct 2023
In-Context Learning for Few-Shot Molecular Property Prediction Christopher Fifty J. Leskovec Sebastian Thrun 36 5 0 13 Oct 2023
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning J. Wang Chengyu Wang Chuanqi Tan Jun Huang Ming Gao KELM 26 4 0 26 Sep 2023
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes Yosuke Miyanishi M. Nguyen 26 2 0 19 Aug 2023
Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning Xindi Wang Yufei Wang Can Xu Xiubo Geng Bowen Zhang Chongyang Tao Frank Rudzicz Robert E. Mercer Daxin Jiang 18 11 0 28 Jul 2023
Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomy I. Ciucă Y. Ting 丁 Sandor Kruk K. Iyer 19 7 0 20 Jun 2023
Differentiable Instruction Optimization for Cross-Task Generalization Masaru Isonuma Junichiro Mori Ichiro Sakata 23 0 0 16 Jun 2023
Schema-learning and rebinding as mechanisms of in-context learning and emergence Siva K. Swaminathan Antoine Dedieu Rajkumar Vasudeva Raju Murray Shanahan Miguel Lazaro-Gredilla Dileep George 34 8 0 16 Jun 2023
IDAS: Intent Discovery with Abstractive Summarization Maarten De Raedt Fréderic Godin Thomas Demeester Chris Develder 30 16 0 31 May 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 46 187 0 29 May 2023
Im-Promptu: In-Context Composition from Image Prompts Bhishma Dedhia Michael Chang Jake C. Snell Thomas L. Griffiths N. Jha LRM MLLM 29 1 0 26 May 2023
How Does In-Context Learning Help Prompt Tuning? Simeng Sun Yang Liu Dan Iter Chenguang Zhu Mohit Iyyer VLM 23 17 0 22 Feb 2023
Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning? Chengwei Qin Q. Li Ruochen Zhao Shafiq R. Joty VLM LRM 21 15 0 16 Feb 2023
In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models Yukun Huang Yanda Chen Zhou Yu Kathleen McKeown 15 30 0 20 Dec 2022
Task Ambiguity in Humans and Language Models Alex Tamkin Kunal Handa Ava Shrestha Noah D. Goodman UQLM 41 22 0 20 Dec 2022
Successive Prompting for Decomposing Complex Questions Dheeru Dua Shivanshu Gupta Sameer Singh Matt Gardner ReLM LRM 29 111 0 08 Dec 2022
Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding Maximillian Chen Alexandros Papangelis Chenyang Tao Andrew Rosenbaum Seokhwan Kim Yang Liu Zhou Yu Dilek Z. Hakkani-Tür 39 32 0 25 Oct 2022
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners Seonghyeon Ye Doyoung Kim Joel Jang Joongbo Shin Minjoon Seo FedML VLM UQCV LRM 11 25 0 06 Oct 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 182 77 0 16 Sep 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 21 447 0 01 Aug 2022
Few-shot Adaptation Works with UnpredicTable Data Jun Shern Chan Michael Pieler Jonathan Jao Jérémy Scheurer Ethan Perez 25 5 0 01 Aug 2022
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations Rakesh R Menon Sayan Ghosh Shashank Srivastava LRM ELM 21 9 0 14 Apr 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 20 1,392 0 25 Feb 2022
MetaICL: Learning to Learn In Context Sewon Min M. Lewis Luke Zettlemoyer Hannaneh Hajishirzi LRM 52 466 0 29 Oct 2021
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 209 179 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,844 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,312 0 17 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,918 0 31 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 228 4,460 0 23 Jan 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,587 0 21 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 413 2,584 0 03 Sep 2019
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 308 11,681 0 09 Mar 2017