Take the Bull by the Horns: Hard Sample-Reweighted Continual Training
Improves LLM Generalization

Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization

22 February 2024

Mingyuan Zhou

Papers citing "Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization"

16 / 16 papers shown

Title
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 140 50 1 01 Jul 2024
Orca 2: Teaching Small Language Models How to Reason Arindam Mitra Luciano Del Corro Shweti Mahajan Andres Codas Clarisse Simoes ... Hamid Palangi Guoqing Zheng Corby Rosset Hamed Khanpour Ahmed Hassan Awadallah ReLM LRM 72 141 0 18 Nov 2023
Irreducible Curriculum for Language Model Pretraining Simin Fan Martin Jaggi 112 10 0 23 Oct 2023
Platypus: Quick, Cheap, and Powerful Refinement of LLMs Ariel N. Lee Cole J. Hunter Nataniel Ruiz ALM ObjD 66 141 0 14 Aug 2023
Auditing and Generating Synthetic Data with Controllable Trust Trade-offs Brian M. Belgodere Pierre Dognin Adam Ivankay Igor Melnyk Youssef Mroueh ... Mattia Rigotti Jerret Ross Yair Schiff Radhika Vedpathak Richard A. Young 67 12 0 21 Apr 2023
Is synthetic data from generative models ready for image recognition? Ruifei He Shuyang Sun Xin Yu Chuhui Xue Wenqing Zhang Philip Torr Song Bai Xiaojuan Qi 84 301 0 14 Oct 2022
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 70 16 0 11 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 775 9,351 0 28 Jan 2022
GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training Krishnateja Killamsetty D. Sivasubramanian Ganesh Ramakrishnan A. De Rishabh K. Iyer OOD 129 201 0 27 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 441 2,093 0 31 Dec 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 419 20,127 0 23 Oct 2019
Accelerating Deep Learning by Focusing on the Biggest Losers Angela H. Jiang Daniel L.-K. Wong Giulio Zhou D. Andersen J. Dean ... Gauri Joshi M. Kaminsky M. Kozuch Zachary Chase Lipton Padmanabhan Pillai 58 121 0 02 Oct 2019
Adversarial Attack Generation Empowered by Min-Max Optimization Jingkang Wang Tianyun Zhang Sijia Liu Pin-Yu Chen Jiacen Xu M. Fardad Yangqiu Song AAML 48 37 0 09 Jun 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 217 1,517 0 24 May 2019
Robust Optimization over Multiple Domains Qi Qian Shenghuo Zhu Jiasheng Tang Rong Jin Baigui Sun Hao Li OOD 55 71 0 19 May 2018
Online Batch Selection for Faster Training of Neural Networks I. Loshchilov Frank Hutter ODL 85 301 0 19 Nov 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.