MixMin: Finding Data Mixtures via Convex Minimization

14 February 2025

Papers citing "MixMin: Finding Data Mixtures via Convex Minimization"

25 / 25 papers shown

Title
Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws Yiding Jiang Allan Zhou Zhili Feng Sadhika Malladi J. Zico Kolter 55 21 0 15 Oct 2024
Improving Pretraining Data Using Perplexity Correlations Tristan Thrush Christopher Potts Tatsunori Hashimoto 72 19 0 09 Sep 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 108 49 1 01 Jul 2024
Does your data spark joy? Performance gains from domain upsampling at the end of training Cody Blakeney Mansheej Paul Brett W. Larsen Sean Owen Jonathan Frankle 52 20 0 05 Jun 2024
No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models Angeline Pouget Lucas Beyer Emanuele Bugliarello Xiao Wang Andreas Steiner Xiao-Qi Zhai Ibrahim Alabdulmohsin VLM 49 9 0 22 May 2024
Scaling laws for learning with real and surrogate data Ayush Jain Andrea Montanari Eren Sasoglu 73 14 0 06 Feb 2024
Efficient Online Data Mixing For Language Model Pre-Training Alon Albalak Liangming Pan Colin Raffel Wenjie Wang 56 43 0 05 Dec 2023
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets Dominique Beaini Shenyang Huang Joao Alex Cunha Zhiyi Li Gabriela Moisescu-Pareja ... Thérence Bois Andrew Fitzgibbon Bla.zej Banaszewski Chad Martin Dominic Masters AI4CE 64 22 0 06 Oct 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 221 11,636 0 18 Jul 2023
GIO: Gradient Information Optimization for Training Dataset Selection Dante Everaert Christopher Potts 70 6 0 20 Jun 2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling Stella Biderman Hailey Schoelkopf Quentin G. Anthony Herbie Bradley Kyle O'Brien ... USVSN Sai Prashanth Edward Raff Aviya Skowron Lintang Sutawika Oskar van der Wal 81 1,231 0 03 Apr 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 800 13,788 0 15 Mar 2023
Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language Philipp Seidl Andreu Vall Sepp Hochreiter Günter Klambauer 113 38 0 06 Mar 2023
Data Selection for Language Models via Importance Resampling Sang Michael Xie Shibani Santurkar Tengyu Ma Percy Liang 90 186 0 06 Feb 2023
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 139 1,915 0 29 Mar 2022
RegMix: Data Mixing Augmentation for Regression Seonghyeon Hwang Steven Euijong Whang UQCV 37 8 0 07 Jun 2021
Bilevel Optimization: Convergence Analysis and Enhanced Design Kaiyi Ji Junjie Yang Yingbin Liang 152 256 0 15 Oct 2020
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 103 1,724 0 26 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 330 19,824 0 23 Oct 2019
Analyzing Learned Molecular Representations for Property Prediction Kevin Kaichuang Yang Kyle Swanson Wengong Jin Connor W. Coley Philipp Eiden ... Andrew Palmer Volker Settels Tommi Jaakkola K. Jensen Regina Barzilay 89 1,305 0 02 Apr 2019
An Integrated Transfer Learning and Multitask Learning Approach for Pharmacokinetic Parameter Prediction Zhuyifan Ye Yilong Yang Xiaoshan Li Dongsheng Cao D. Ouyang 43 75 0 21 Dec 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 113 2,474 0 14 Mar 2018
Crowdsourcing Multiple Choice Science Questions Johannes Welbl Nelson F. Liu Matt Gardner AI4Ed 57 493 0 19 Jul 2017
XGBoost: A Scalable Tree Boosting System Tianqi Chen Carlos Guestrin 474 37,815 0 09 Mar 2016
Hyperparameter optimization with approximate gradient Fabian Pedregosa 90 449 0 07 Feb 2016