Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

12 June 2024

Yejin Choi

Bill Yuchen Lin

SyDa

ArXiv PDF HTML

Papers citing "Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing"

50 / 102 papers shown

Title
Small Models Struggle to Learn from Strong Reasoners Yuetai Li Xiang Yue Zhangchen Xu Fengqing Jiang Luyao Niu Bill Yuchen Lin Bhaskar Ramasubramanian Radha Poovendran LRM 46 12 0 17 Feb 2025
Navigating the Helpfulness-Truthfulness Trade-Off with Uncertainty-Aware Instruction Fine-Tuning Tianyi Wu Jingwei Ni Bryan Hooi Jiaheng Zhang Elliott Ash See-Kiong Ng Mrinmaya Sachan Markus Leippold 59 0 0 17 Feb 2025
Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models Zikang Liu K. Zhou Wayne Xin Zhao Dawei Gao Yaliang Li Zhicheng Dou MLLM VLM LRM 94 0 0 17 Feb 2025
DeepThink: Aligning Language Models with Domain-Specific User Intents Yang Li Mingxuan Luo Yeyun Gong Chen Lin Jian Jiao Yi Liu Kaili Huang LRM ALM ELM 59 0 0 08 Feb 2025
Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis Lin Yuan Jun Xu Honghao Gui Mengshu Sun Qing Cui Lei Liang Jun Zhou AI4CE 231 0 0 06 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 117 4 0 06 Feb 2025
Breaking Focus: Contextual Distraction Curse in Large Language Models Yue Huang Yanbo Wang Zixiang Xu Chujie Gao Siyuan Wu Jiayi Ye Xiuying Chen Pin-Yu Chen Xuzhi Zhang AAML 48 1 0 03 Feb 2025
ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer Lin Yueyu Li Zhiyuan Peter Yue Liu Xiao 42 5 0 28 Jan 2025
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking Benjamin Feuer Micah Goldblum Teresa Datta Sanjana Nambiar Raz Besaleli Samuel Dooley Max Cembalest John P. Dickerson ALM 42 7 0 28 Jan 2025
NExtLong: Toward Effective Long-Context Training without Long Documents Chaochen Gao Xing Wu Zijia Lin Debing Zhang Songlin Hu SyDa 68 1 0 22 Jan 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 173 0 0 31 Dec 2024
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 96 12 0 31 Dec 2024
Understanding Synthetic Context Extension via Retrieval Heads Xinyu Zhao Fangcong Yin Greg Durrett 41 0 0 31 Dec 2024
WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models Huawen Feng Pu Zhao Qingfeng Sun Can Xu Fangkai Yang ... Qianli Ma Qingwei Lin Saravan Rajmohan Dongmei Zhang Qi Zhang AAML ALM 62 0 0 23 Dec 2024
NILE: Internal Consistency Alignment in Large Language Models Minda Hu Qiyuan Zhang Yufei Wang Bowei He Hongru Wang Jingyan Zhou Liangyou Li Yasheng Wang Chen Ma Irwin King 91 0 0 21 Dec 2024
Smaller Language Models Are Better Instruction Evolvers Tingfeng Hui Lulu Zhao Guanting Dong Yaqi Zhang Hua Zhou Sen Su ALM 87 1 0 15 Dec 2024
Reinforcement Learning Enhanced LLMs: A Survey Shuhe Wang Shengyu Zhang Jun Zhang Runyi Hu Xiaoya Li Tianwei Zhang Jiwei Li Fei Wu G. Wang Eduard H. Hovy OffRL 134 7 0 05 Dec 2024
DOGE: Towards Versatile Visual Document Grounding and Referring Yinan Zhou Yuxin Chen Haokun Lin Shuyu Yang Li Zhu Zhongang Qi Chen Ma Ying Shan ObjD 86 2 0 26 Nov 2024
LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training Xiaoye Qu Daize Dong Xuyang Hu Tong Zhu Weigao Sun Yu-Xi Cheng MoE 92 10 0 24 Nov 2024
A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection Gabriel Chua Shing Yee Chan Shaun Khoo 83 1 0 20 Nov 2024
Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions Yutao Hou Yajing Luo Zhiwen Ruan H. Wang Weifeng Ge Yuxiao Chen Guanhua Chen ELM 47 0 0 15 Nov 2024
Building a Taiwanese Mandarin Spoken Language Model: A First Attempt Chih-Kai Yang Yu-Kuan Fu Chen An Li Yi-Cheng Lin Yu-Xiang Lin ... Ulin Sanga Xuanjun Chen Po-Chun Hsu Shu-Wen Yang Hung-yi Lee AuLLM 46 1 0 11 Nov 2024
Stronger Models are NOT Stronger Teachers for Instruction Tuning Zhangchen Xu Fengqing Jiang Luyao Niu Bill Yuchen Lin Radha Poovendran ALM 56 5 0 11 Nov 2024
SSSD: Simply-Scalable Speculative Decoding Michele Marzollo Jiawei Zhuang Niklas Roemer Lorenz K. Müller Lukas Cavigelli LRM 39 2 0 08 Nov 2024
Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation Yifang Chen David Zhu SyDa 41 0 0 27 Oct 2024
Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis from Scratch Yuyang Ding Xinyu Shi Xiaobo Liang Juntao Li Qiaoming Zhu Min Zhang ELM AIMat SyDa LRM 30 9 0 24 Oct 2024
Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs Chris Liu Liang Zeng Jiaheng Liu Rui Yan Jujie He Chaojie Wang Shuicheng Yan Yang Liu Yahui Zhou AI4TS 51 70 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 48 4 0 23 Oct 2024
Beware of Calibration Data for Pruning Large Language Models Yixin Ji Yang Xiang Juntao Li Qingrong Xia Ping Li Xinyu Duan Zhefeng Wang Min Zhang 42 2 0 23 Oct 2024
VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning Yifan Peng Krishna C. Puvvada Zhehuai Chen Piotr .Zelasko He Huang Kunal Dhawan Ke Hu Shinji Watanabe Jagadeesh Balam Boris Ginsburg 61 2 0 23 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 73 3 0 20 Oct 2024
A Survey on Data Synthesis and Augmentation for Large Language Models Ke Wang Jiahui Zhu Minjie Ren Ziqiang Liu Shiwei Li ... Chenkai Zhang Xiaoyu Wu Qiqi Zhan Qingjie Liu Yunhong Wang SyDa 42 18 0 16 Oct 2024
Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as the Key Yingda Chen Xingjun Wang Jintao Huang Yunlin Mao Daoze Zhang Yuze Zhao ALM 26 0 0 14 Oct 2024
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks Fenia Christopoulou Ronald Cardenas Gerasimos Lampouras Haitham Bou-Ammar Jun Wang 27 1 0 07 Oct 2024
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences Genta Indra Winata David Anugraha Lucky Susanto Garry Kuwanto Derry Wijaya 42 7 0 03 Oct 2024
Federated Instruction Tuning of LLMs with Domain Coverage Augmentation Zezhou Wang Yaxin Du Zhuzhong Qian Yugang Jiang Zhuzhong Qian Siheng Chen FedML 161 0 0 30 Sep 2024
Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs Yung-Chieh Chan George Pu Apaar Shanker Parth Suresh Penn Jenks John Heyer Sam Denton SyDa 31 8 0 29 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 82 21 0 26 Sep 2024
NDP: Next Distribution Prediction as a More Broad Target Junhao Ruan Abudukeyumu Abudula Xinyu Liu Bei Li Yinqiao Li Chenglong Wang Yuchun Fan Yuan Ge Tong Xiao Jingbo Zhu 34 0 0 30 Aug 2024
Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data Juncheng Xie Shensian Syu Hung-yi Lee ALM 31 1 0 27 Aug 2024
I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm Yiming Liang Ge Zhang Xingwei Qu Tianyu Zheng Jiawei Guo ... Jiaheng Liu Chenghua Lin Lei Ma Wenhao Huang Jiajun Zhang ALM 51 5 0 15 Aug 2024
Prompto: An open source library for asynchronous querying of LLM endpoints Ryan Sze-Yin Chan Federico Nanni Edwin Brown Ed Chapman Angus R. Williams Jonathan Bright Evelina Gabasova LRM 24 1 0 12 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 58 569 0 06 Aug 2024
Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression Zilun Zhang Yutao Sun Tiancheng Zhao Leigang Sha Ruochen Xu Kyusong Lee Jianwei Yin CLL KELM 56 0 0 17 Jun 2024
Aligning to Thousands of Preferences via System Message Generalization Seongyun Lee Sue Hyun Park Seungone Kim Minjoon Seo ALM 44 38 0 28 May 2024
WildChat: 1M ChatGPT Interaction Logs in the Wild Wenting Zhao Xiang Ren Jack Hessel Claire Cardie Yejin Choi Yuntian Deng 44 180 0 02 May 2024
RewardBench: Evaluating Reward Models for Language Modeling Nathan Lambert Valentina Pyatkin Jacob Morrison Lester James V. Miranda Bill Yuchen Lin ... Sachin Kumar Tom Zick Yejin Choi Noah A. Smith Hanna Hajishirzi ALM 85 218 0 20 Mar 2024
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 50 19 0 05 Mar 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 182 456 0 02 Feb 2024
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing Walid Hariri AI4MH LM&MA 33 85 0 27 Mar 2023