Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for
Instruction Fine-Tuning

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

7 February 2024

Maksym Andriushchenko

Francesco Croce

Nicolas Flammarion

Papers citing "Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning"

15 / 15 papers shown

Title
Advancing MAPF towards the Real World: A Scalable Multi-Agent Realistic Testbed (SMART) Jingtian Yan Zhifei Li William Kang Yulun Zhang Stephen Smith Jiaoyang Li 43 0 0 03 Mar 2025
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning H. Fernando Han Shen Parikshit Ram Yi Zhou Horst Samulowitz Nathalie Baracaldo Tianyi Chen CLL 56 2 0 20 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 39 8 0 09 Oct 2024
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe Yuxin Xiao Shujian Zhang Wenxuan Zhou Marzyeh Ghassemi Sanqiang Zhao 109 0 0 07 Oct 2024
Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation Jing Yuan Luo Yunlong Song Victor Klemm Fan Shi Davide Scaramuzza Marco Hutter 33 2 0 04 Oct 2024
From Distributional to Overton Pluralism: Investigating Large Language Model Alignment Thom Lake Eunsol Choi Greg Durrett 44 9 0 25 Jun 2024
Unlocking Large Language Model's Planning Capabilities with Maximum Diversity Fine-tuning Wenjun Li Changyu Chen Pradeep Varakantham 49 2 0 15 Jun 2024
UltraMedical: Building Specialized Generalists in Biomedicine Kaiyan Zhang Sihang Zeng Ermo Hua Ning Ding Zhang-Ren Chen ... Xuekai Zhu Xingtai Lv Hu Jinfang Zhiyuan Liu Bowen Zhou LM&MA 43 20 0 06 Jun 2024
Is In-Context Learning Sufficient for Instruction Following in LLMs? Hao Zhao Maksym Andriushchenko Francesco Croce Nicolas Flammarion 67 11 0 30 May 2024
Enabling Weak LLMs to Judge Response Reliability via Meta Ranking Zijun Liu Boqun Kou Peng Li Ming Yan Ji Zhang Fei Huang Yang Liu 30 2 0 19 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 239 298 0 18 Jan 2024
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning Ming Li Lichang Chen Jiuhai Chen Shwai He Heng-Chiao Huang Jiuxiang Gu Tianyi Zhou 105 20 0 18 Oct 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 159 579 0 06 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 364 8,495 0 28 Jan 2022