Exploring Format Consistency for Instruction Tuning

v1v2 (latest)

Exploring Format Consistency for Instruction Tuning

28 July 2023

Zhiyuan Liu

Xiaojiang Liu

Maosong Sun

ArXiv (abs)PDF HTML

Papers citing "Exploring Format Consistency for Instruction Tuning"

15 / 15 papers shown

Title
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery Bowei He Lihao Yin Hui-Ling Zhen Xiaokun Zhang Mingxuan Yuan Chen Ma 155 0 0 18 Feb 2025
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs Jiancheng Dong Lei Jiang Wei Jin Lu Cheng 92 1 0 18 Aug 2024
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization Srinivasan Iyer Xi Lin Ramakanth Pasunuru Todor Mihaylov Daniel Simig ... Jeff Wang Christopher Dewan Asli Celikyilmaz Luke Zettlemoyer Veselin Stoyanov ALM 146 267 0 22 Dec 2022
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor Or Honovich Thomas Scialom Omer Levy Timo Schick ALM 126 375 0 19 Dec 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 206 3,150 0 20 Oct 2022
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks Yizhong Wang Swaroop Mishra Pegah Alipoormolabashi Yeganeh Kordi Amirreza Mirzaei ... Chitta Baral Yejin Choi Noah A. Smith Hannaneh Hajishirzi Daniel Khashabi ELM 123 858 0 16 Apr 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 333 349 0 02 Feb 2022
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 99 216 0 22 Nov 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 230 3,782 0 03 Sep 2021
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 288 185 0 18 Apr 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 168 752 0 18 Apr 2021
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 111 223 0 10 Feb 2020
Not Enough Data? Deep Learning to the Rescue! Ateret Anaby-Tavor Boaz Carmeli Esther Goldbraich Amir Kantor George Kour Segev Shlomov N. Tepper Naama Zwerdling 84 370 0 08 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 483 20,317 0 23 Oct 2019
Understanding Black-box Predictions via Influence Functions Pang Wei Koh Percy Liang TDI 219 2,905 0 14 Mar 2017