What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

25 December 2023

Papers citing "What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning"

50 / 169 papers shown

Title
Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional Reasoning Dayong Liang Changmeng Zheng Zhiyuan Wen Yi Cai Xiao Wei Qing Li LRM 31 0 0 14 May 2025
RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection Yixin Yang Qingxiu Dong Linli Yao Fangwei Zhu Zhifang Sui 48 0 0 08 May 2025
Text2Cypher: Data Pruning using Hard Example Selection Makbule Gulcin Ozsoy AAML 43 0 0 08 May 2025
ParetoHqD: Fast Offline Multiobjective Alignment of Large Language Models using Pareto High-quality Data Haoran Gu Handing Wang Yi Mei Mengjie Zhang Yaochu Jin 27 1 0 23 Apr 2025
The Rise of Small Language Models in Healthcare: A Comprehensive Survey Muskan Garg Shaina Raza Shebuti Rayana Xingyi Liu Sunghwan Sohn LM&MA AILaw 92 0 0 23 Apr 2025
DONOD: Robust and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning Jucheng Hu Steve Yang Dongzhan Zhou Lijun Wu 34 0 0 21 Apr 2025
MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space Yicheng Chen Yining Li Kai Hu Zerun Ma Haochen Ye Kai Chen 34 0 0 18 Apr 2025
RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability Y. Zhang Zihao Zeng Dongbai Li Yao Huang Zhijie Deng Yinpeng Dong LRM 35 4 0 14 Apr 2025
How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients Ming Li Yongqian Li Ziyue Li Tianyi Zhou LRM 27 1 0 14 Apr 2025
MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning Yangning Li Zihua Lan Lv Qingsong Hai-Tao Zheng Hai-Tao Zheng 31 0 0 09 Apr 2025
RAISE: Reinforenced Adaptive Instruction Selection For Large Language Models Lv Qingsong Yangning Li Zihua Lan Zishan Xu Jiwei Tang Hai-Tao Zheng Wenhao Jiang Hai-tao Zheng Philip S. Yu 32 0 0 09 Apr 2025
ToolACE-R: Tool Learning with Adaptive Self-Refinement Xingshan Zeng Wei Liu X. Huang Zezhong Wang Lingzhi Wang ... Yishuo Wang Lifeng Shang Xin Jiang Ruiming Tang Qiang Liu CLL 52 0 0 02 Apr 2025
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design Mohan Zhang Pingzhi Li Jie Peng Mufan Qiu Tianlong Chen MoE 47 0 0 02 Apr 2025
When Words Outperform Vision: VLMs Can Self-Improve Via Text-Only Training For Human-Centered Decision Making Zhe Hu Jing Li Yu Yin VLM 66 0 0 21 Mar 2025
Towards Automatic Continual Learning: A Self-Adaptive Framework for Continual Instruction Tuning Peiyi Lin Fukai Zhang Kai Niu Hao Fu CLL 64 0 0 20 Mar 2025
MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models J. Li Lu Yu Daixin Wang Qing Cui Jun Zhou Yanfang Ye Chuxu Zhang 64 0 0 19 Mar 2025
DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models Yi Shen Jingyang Zhang Jieyun Huang Shuming Shi Wenjing Zhang Jiangze Yan Rongjia Du Ning Wang Kai Wang LRM 80 15 0 06 Mar 2025
Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm Zehan Li Yuhao Du Xiaoqi Jiao Yiwen Guo Yuege Feng Xiang Wan Anningzhe Gao Jinpeng Hu 63 0 0 04 Mar 2025
CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom Yisen Li Lingfeng Yang Wenxuan Shen Pan Zhou Yao Wan Weiwei Lin Danny Chen 70 0 0 03 Mar 2025
Advancing MAPF towards the Real World: A Scalable Multi-Agent Realistic Testbed (SMART) Jingtian Yan Zhifei Li William Kang Yulun Zhang Stephen Smith Jiaoyang Li 48 0 0 03 Mar 2025
Large-Scale Data Selection for Instruction Tuning Hamish Ivison Muru Zhang Faeze Brahman Pang Wei Koh Pradeep Dasigi ALM 73 1 0 03 Mar 2025
CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning Ping Zhang Zhaorui Zhang Sheng Di Yao Xin Benben Liu 50 1 0 26 Feb 2025
Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning Hongyi Cal Jie Li Wenzhen Dong 66 0 0 26 Feb 2025
Larger or Smaller Reward Margins to Select Preferences for Alignment? Kexin Huang Junkang Wu Ziqian Chen Xue Wang Jinyang Gao Bolin Ding Jiancan Wu Xiangnan He Xuben Wang 52 0 0 25 Feb 2025
Advantage-Guided Distillation for Preference Alignment in Small Language Models Shiping Gao Fanqi Wan Jiajian Guo Xiaojun Quan Qifan Wang ALM 58 0 0 25 Feb 2025
MergeIT: From Selection to Merging for Efficient Instruction Tuning Hongyi Cai Yuqian Fu Hongming Fu Bo Zhao MoMe 53 0 0 25 Feb 2025
Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling Yiwen Ding Zhiheng Xi Wei He Zhuoyuan Li Yitao Zhai Xiaowei Shi Xunliang Cai Tao Gui Qi Zhang Xuanjing Huang LRM 75 3 0 24 Feb 2025
Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps Yen-Che Hsiao Abhishek Dutta LRM ReLM ELM 66 0 0 24 Feb 2025
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment Sizhe Wang Yongqi Tong Hengyuan Zhang Dawei Li Xin Zhang Tianlong Chen 85 5 0 21 Feb 2025
EDGE: Efficient Data Selection for LLM Agents via Guideline Effectiveness Yunxiao Zhang Guanming Xiong Haochen Li Wen Zhao LLMAG 71 0 0 18 Feb 2025
Navigating the Helpfulness-Truthfulness Trade-Off with Uncertainty-Aware Instruction Fine-Tuning Tianyi Wu Jingwei Ni Bryan Hooi Jiaheng Zhang Elliott Ash See-Kiong Ng Mrinmaya Sachan Markus Leippold 51 0 0 17 Feb 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 102 15 0 17 Feb 2025
InsBank: Evolving Instruction Subset for Ongoing Alignment Jiayi Shi Yiwei Li Shaoxiong Feng Peiwen Yuan X. U. Wang ... Chuyi Tan Boyuan Pan Huan Ren Yao Hu Kan Li ALM 92 0 0 17 Feb 2025
Refine Knowledge of Large Language Models via Adaptive Contrastive Learning Hai-Tao Zheng Haojing Huang Jiayi Kuang Yangning Li Shu Guo C. Qu Xiaoyu Tan Hai-Tao Zheng Ying Shen Philip S. Yu CLL 66 5 0 11 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 117 3 0 06 Feb 2025
STAIR: Improving Safety Alignment with Introspective Reasoning Y. Zhang Siyuan Zhang Yao Huang Zeyu Xia Zhengwei Fang Xiao Yang Ranjie Duan Dong Yan Yinpeng Dong Jun Zhu LRM LLMSV 56 3 0 04 Feb 2025
Preference Leakage: A Contamination Problem in LLM-as-a-judge Dawei Li Renliang Sun Yue Huang Ming Zhong Bohan Jiang J. Han Xuzhi Zhang Wei Wang Huan Liu 65 11 0 03 Feb 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zhilin Yang Zhiqi Huang Zihao Huang Ziyao Xu Z. Yang VLM ALM OffRL AI4TS LRM 111 141 0 22 Jan 2025
Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities Qirun Dai Dylan Zhang Jiaqi W. Ma Hao Peng TDI 55 1 0 21 Jan 2025
From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning Yafu Li Zhilin Wang Tingchen Fu Ganqu Cui Sen Yang Yu Cheng 45 1 0 21 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 93 12 0 31 Dec 2024
Multimodal Preference Data Synthetic Alignment with Reward Model Robert Wijaya Ngoc-Bao Nguyen Ngai-man Cheung MLLM SyDa 62 2 0 23 Dec 2024
Boosting LLM via Learning from Data Iteratively and Selectively Qi Jia Siyu Ren Ziheng Qin Fuzhao Xue Jinjie Ni Yang You 36 0 0 23 Dec 2024
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 85 2 0 23 Dec 2024
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs Aldo Pareja Nikhil Shivakumar Nayak Hao Wang Krishnateja Killamsetty Shivchander Sudalairaj ... Guangxuan Xu Kai Xu Ligong Han Luke Inglis Akash Srivastava 88 6 0 17 Dec 2024
Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models Yuchen Fan Yuzhong Hong Qiushi Wang Junwei Bao Hongfei Jiang Yang Song 82 1 0 17 Dec 2024
Smaller Language Models Are Better Instruction Evolvers Tingfeng Hui Lulu Zhao Guanting Dong Yaqi Zhang Hua Zhou Sen Su ALM 81 1 0 15 Dec 2024
Learning from "Silly" Questions Improves Large Language Models, But Only Slightly Tingyuan Zhu Shudong Liu Yidong Wang Derek F. Wong Han Yu T. Shinozaki Jindong Wang ALM LRM 79 0 0 21 Nov 2024
Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning Hang Zhou Yehui Tang Haochen Qin Yujie Yang Renren Jin Deyi Xiong Kai Han Yunhe Wang 57 2 0 21 Nov 2024
AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations Gaurav Verma Rachneet Kaur Nishan Srishankar Zhen Zeng T. Balch Manuela Veloso LLMAG 72 5 0 20 Nov 2024