What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

25 December 2023

Papers citing "What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning"

50 / 172 papers shown

Title
DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph Zhehao Zhang Jiaao Chen Diyi Yang LRM 37 8 0 25 Jun 2024
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training Tong Zhu Xiaoye Qu Daize Dong Jiacheng Ruan Jingqi Tong Conghui He Yu Cheng MoE ALM 54 71 0 24 Jun 2024
RuleR: Improving LLM Controllability by Rule-based Data Recycling Ming Li Han Chen Chenguang Wang Dang Nguyen Dianqi Li Dinesh Manocha 28 7 0 22 Jun 2024
Efficient Continual Pre-training by Mitigating the Stability Gap Yiduo Guo Jie Fu Huishuai Zhang Dongyan Zhao Songlin Yang 30 13 0 21 Jun 2024
Timo: Towards Better Temporal Reasoning for Language Models Zhaochen Su Jun Zhang Tong Zhu Xiaoye Qu Juntao Li Min Zhang Yu Cheng LRM 47 17 0 20 Jun 2024
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models Ziche Liu Rui Ke Feng Jiang Feng Jiang Haizhou Li 69 1 0 20 Jun 2024
Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models Hongbang Yuan Yubo Chen Pengfei Cao Zhuoran Jin Kang Liu Jun Zhao 44 0 0 18 Jun 2024
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving Yuxuan Tong Xiwen Zhang Rui Wang R. Wu Junxian He AIMat LRM 46 33 0 18 Jun 2024
Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decoding Kaiyan Zhang Jianyu Wang Ning Ding Biqing Qi Ermo Hua Xingtai Lv Bowen Zhou 43 9 0 18 Jun 2024
Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models Lulu Zhao Weihao Zeng Xiaofeng Shi Hua Zhou Donglin Hao Yonghua Lin LM&MA 34 4 0 18 Jun 2024
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts Tong Zhu Daize Dong Xiaoye Qu Jiacheng Ruan Wenliang Chen Yu Cheng MoE 40 8 0 17 Jun 2024
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition Yuming Yang Wantong Zhao Caishuang Huang Junjie Ye Xiao Wang ... Kaixin Huang Yunke Zhang Tao Gui Qi Zhang Xuanjing Huang 44 3 0 17 Jun 2024
Concept-skill Transferability-based Data Selection for Large Vision-Language Models Jaewoo Lee Boyang Li Sung Ju Hwang VLM 43 8 0 16 Jun 2024
SCAR: Efficient Instruction-Tuning for Large Language Models via Style Consistency-Aware Response Ranking Zhuang Li Yuncheng Hua Thuy-Trang Vu Haolan Zhan Lizhen Qu Gholamreza Haffari 54 2 0 16 Jun 2024
Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Models Jie Chen Xintian Han Yu Ma Xun Zhou Liang Xiang ALM LRM 48 2 0 14 Jun 2024
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing Zhangchen Xu Fengqing Jiang Luyao Niu Yuntian Deng Radha Poovendran Yejin Choi Bill Yuchen Lin SyDa 42 127 0 12 Jun 2024
Discovering Preference Optimization Algorithms with and for Large Language Models Chris Xiaoxuan Lu Samuel Holt Claudio Fanconi Alex J. Chan Jakob Foerster M. Schaar R. T. Lange OffRL 40 16 0 12 Jun 2024
CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation Renhao Li Minghuan Tan Derek F. Wong Min Yang LLMAG 23 1 0 11 Jun 2024
Uncertainty Aware Learning for Language Model Alignment Yikun Wang Rui Zheng Liang Ding Qi Zhang Dahua Lin Dacheng Tao 45 4 0 07 Jun 2024
FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models Guangyi Liu Rui Ge Xinyu Zhu Jingyi Chai Yaxin Du Yang Liu Yanfeng Wang Siheng Chen FedML 41 14 0 07 Jun 2024
UltraMedical: Building Specialized Generalists in Biomedicine Kaiyan Zhang Sihang Zeng Ermo Hua Ning Ding Zhang-Ren Chen ... Xuekai Zhu Xingtai Lv Hu Jinfang Zhiyuan Liu Bowen Zhou LM&MA 43 22 0 06 Jun 2024
Automatic Instruction Evolving for Large Language Models Weihao Zeng Can Xu Yingxiu Zhao Jianguang Lou Weizhu Chen SyDa 48 9 0 02 Jun 2024
Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment Yueqin Yin Zhendong Wang Yujia Xie Weizhu Chen Mingyuan Zhou 37 4 0 31 May 2024
A Survey of Multimodal Large Language Model from A Data-centric Perspective Tianyi Bai Hao Liang Binwang Wan Yanran Xu Xi Li ... Ping-Chia Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang 58 36 0 26 May 2024
Automatically Generating Numerous Context-Driven SFT Data for LLMs across Diverse Granularity Shanghaoran Quan 43 4 0 26 May 2024
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer Zhihan Liu Miao Lu Shenao Zhang Boyi Liu Hongyi Guo Yingxiang Yang Jose H. Blanchet Zhaoran Wang 48 43 0 26 May 2024
Instruction Tuning With Loss Over Instructions Zhengyan Shi Adam X. Yang Bin Wu Laurence Aitchison Emine Yilmaz Aldo Lipani ALM 24 20 0 23 May 2024
Annotation-Efficient Preference Optimization for Language Model Alignment Yuu Jinnai Ukyo Honda 42 0 0 22 May 2024
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning Yuanhao Yue Chengyu Wang Jun Huang Peng Wang ALM 30 4 0 22 May 2024
Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction Tingchen Fu Deng Cai Lemao Liu Shuming Shi Rui Yan MoMe 58 13 0 22 May 2024
360Zhinao Technical Report 360Zhinao Team 40 0 0 22 May 2024
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process Ermo Hua Biqing Qi Kaiyan Zhang Yue Yu Ning Ding Xingtai Lv Kai Tian Bowen Zhou 43 3 0 20 May 2024
Aloe: A Family of Fine-tuned Open Healthcare LLMs Ashwin Kumar Gururajan Enrique Lopez-Cuena Jordi Bayarri-Planas Adrián Tormos Daniel Hinjos ... Lucia Urcelay-Ganzabal Marta Gonzalez-Mallo Sergio Alvarez-Napagao Eduard Ayguadé-Parra Ulises Cortés Dario Garcia-Gasulla ELM LM&MA 35 14 0 03 May 2024
Prefix Text as a Yarn: Eliciting Non-English Alignment in Foundation Language Model Runzhe Zhan Xinyi Yang Derek F. Wong Lidia S. Chao Yue Zhang 58 10 0 25 Apr 2024
SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning Yexiao He Ziyao Wang Zheyu Shen Guoheng Sun Yucong Dai Yongkai Wu Hongyi Wang Ang Li 38 11 0 23 Apr 2024
Aligning Language Models to Explicitly Handle Ambiguity Hyuhng Joon Kim Youna Kim Cheonbok Park Junyeob Kim Choonghyun Park Kang Min Yoo Sang-goo Lee Taeuk Kim 34 14 0 18 Apr 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 50 56 0 11 Apr 2024
Best Practices and Lessons Learned on Synthetic Data for Language Models Ruibo Liu Jerry W. Wei Fangyu Liu Chenglei Si Yanzhe Zhang ... Steven Zheng Daiyi Peng Diyi Yang Denny Zhou Andrew M. Dai SyDa EgoV 43 86 0 11 Apr 2024
Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models Haoran Sun Lixin Liu Junjie Li Fengyu Wang Baohua Dong Ran Lin Ruohui Huang 33 15 0 03 Apr 2024
Advancing LLM Reasoning Generalists with Preference Trees Lifan Yuan Ganqu Cui Hanbin Wang Ning Ding Xingyao Wang ... Zhenghao Liu Bowen Zhou Hao Peng Zhiyuan Liu Maosong Sun LRM 45 101 0 02 Apr 2024
Exploring the Mystery of Influential Data for Mathematical Reasoning Xinzhe Ni Yeyun Gong Zhibin Gou Yelong Shen Yujiu Yang Nan Duan Weizhu Chen 44 10 0 01 Apr 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 150 502 0 07 Mar 2024
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models Linhai Zhang Jialong Wu Deyu Zhou Guoqiang Xu 30 4 0 02 Mar 2024
PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval He Zhu Wenjia Zhang Nuoxian Huang Boyang Li Luyao Niu ... Yicheng Tao Junyou Su Zhaoya Gong Chenyu Fang Xing Liu LLMAG 53 10 0 29 Feb 2024
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation Yuan Ge Yilun Liu Chi Hu Weibin Meng Shimin Tao Xiaofeng Zhao Hongxia Ma Li Zhang Hao Yang Tong Xiao ALM 42 26 0 28 Feb 2024
Stable LM 2 1.6B Technical Report Marco Bellagente J. Tow Dakota Mahan Duy Phung Maksym Zhuravinskyi ... Paulo Rocha Harry Saini H. Teufel Niccoló Zanichelli Carlos Riquelme OSLM 49 52 0 27 Feb 2024
RECOST: External Knowledge Guided Data-efficient Instruction Tuning Qi Zhang Yiming Zhang Haobo Wang Junbo Zhao 60 11 0 27 Feb 2024
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection Liangxin Liu Xuebo Liu Derek F. Wong Dongfang Li Ziyi Wang Baotian Hu Min Zhang 53 17 0 26 Feb 2024
Towards Robust Instruction Tuning on Multimodal Large Language Models Wei Han Hui Chen Soujanya Poria MLLM 46 0 0 22 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Dinesh Manocha KELM VLM 44 102 0 20 Feb 2024