Fine-tune Language Models to Approximate Unbiased In-context Learning

5 October 2023

Papers citing "Fine-tune Language Models to Approximate Unbiased In-context Learning"

28 / 28 papers shown

Title
In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick Yeqi Gao Zhao Song Shenghao Xie 60 28 0 05 Jul 2023
CodeT5+: Open Code Large Language Models for Code Understanding and Generation Yue Wang Hung Le Akhilesh Deepak Gotmare Nghi D. Q. Bui Junnan Li Steven C. H. Hoi ALM 82 491 0 13 May 2023
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X Qinkai Zheng Xiao Xia Xu Zou Yuxiao Dong Shanshan Wang ... Andi Wang Yang Li Teng Su Zhilin Yang Jie Tang ELM ALM SyDa 123 341 0 30 Mar 2023
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning Zhen Wang Yikang Shen Leonid Karlinsky Rogerio Feris Huan Sun Yoon Kim VLM VPVLM 85 114 0 06 Mar 2023
How Does In-Context Learning Help Prompt Tuning? Simeng Sun Yang Liu Dan Iter Chenguang Zhu Mohit Iyyer VLM 65 19 0 22 Feb 2023
Using In-Context Learning to Improve Dialogue Safety Nicholas Meade Spandana Gella Devamanyu Hazarika Prakhar Gupta Di Jin Siva Reddy Yang Liu Dilek Z. Hakkani-Tür 99 39 0 02 Feb 2023
Execution-based Code Generation using Deep Reinforcement Learning Parshin Shojaee Aneesh Jain Sindhu Tipirneni Chandan K. Reddy 84 58 0 31 Jan 2023
Emergent Analogical Reasoning in Large Language Models Taylor Webb K. Holyoak Hongjing Lu ReLM ELM LRM AI4CE 105 312 0 19 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 116 494 0 15 Dec 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 141 513 0 01 Aug 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 288 2,511 0 15 Jun 2022
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models Ning Ding Yujia Qin Guang Yang Fu Wei Zonghan Yang ... Jianfei Chen Yang Liu Jie Tang Juan Li Maosong Sun 81 204 0 14 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 845 9,644 0 28 Jan 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Cer VLM LRM 198 289 0 15 Oct 2021
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners Ningyu Zhang Luoqiu Li Xiang Chen Shumin Deng Zhen Bi Chuanqi Tan Fei Huang Huajun Chen VLM 125 179 0 30 Aug 2021
Noisy Channel Language Model Prompting for Few-Shot Text Classification Sewon Min Michael Lewis Hannaneh Hajishirzi Luke Zettlemoyer VLM 83 220 0 09 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 228 3,989 0 28 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 587 4,084 0 18 Apr 2021
Automatic Code Generation using Pre-Trained Language Models Luis Perez Lizi Ottens Sudharshan Viswanathan SyDa ALM 59 24 0 21 Feb 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 390 1,389 0 17 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,299 0 01 Jan 2021
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler Zhining Liu Pengfei Wei Jing Jiang Wei Cao Jiang Bian Yi-Ju Chang 61 65 0 17 Oct 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 117 387 0 26 Jun 2020
Learning Data Manipulation for Augmentation and Weighting Zhiting Hu Bowen Tan Ruslan Salakhutdinov Tom Michael Mitchell Eric Xing 66 120 0 28 Oct 2019
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks Haebeom Lee Hayeon Lee Donghyun Na Saehoon Kim Minseop Park Eunho Yang Sung Ju Hwang BDL OODD 77 107 0 30 May 2019
Learning to Reweight Examples for Robust Deep Learning Mengye Ren Wenyuan Zeng Binh Yang R. Urtasun OOD NoLa 149 1,431 0 24 Mar 2018
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation Albert Gatt E. Krahmer LM&MA ELM 128 820 0 29 Mar 2017