Title
On the Robustness of Reward Models for Language Model Alignment Jiwoo Hong Noah Lee Eunki Kim Guijin Son Woojin Chung Aman Gupta Shao Tang James Thorne 29 0 0 12 May 2025
Understanding Stragglers in Large Model Training Using What-if Analysis Jinkun Lin Ziheng Jiang Zuquan Song Sida Zhao Menghan Yu ... Shuguang Wang Yanghua Peng Xin Liu Aurojit Panda Jinyang Li 27 0 0 09 May 2025
ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling Xiao Wang Jong Youl Choi Takuya Kurihaya Isaac Lyngaas Hong-Jun Yoon ... Dali Wang Peter Thornton Prasanna Balaprakash M. Ashfaq Dan Lu 28 0 0 07 May 2025
Galvatron: An Automatic Distributed System for Efficient Foundation Model Training Xinyi Liu Y. Wang Shenhan Zhu Fangcheng Fu Qingshuo Liu Guangming Lin Tengjiao Wang GNN 140 0 0 30 Apr 2025
MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core Dennis Liu Zijie Yan Xin Yao Tong Liu V. Korthikanti ... Jiajie Yao Chandler Zhou David Wu Xipeng Li J. Yang MoE 70 0 0 21 Apr 2025
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model Team Seawead Ceyuan Yang Zhijie Lin Yang Zhao Shanchuan Lin ... Zuquan Song Zhenheng Yang Jiashi Feng Jianchao Yang Lu Jiang DiffM 93 1 0 11 Apr 2025
AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation Tuhin Chakrabarty Philippe Laban C. Wu 32 1 0 10 Apr 2025
TAGC: Optimizing Gradient Communication in Distributed Transformer Training Igor Polyakov Alexey Dukhanov Egor Spirin 43 0 0 08 Apr 2025
SEA-LION: Southeast Asian Languages in One Network Raymond Ng Thanh Ngan Nguyen Yuli Huang Ngee Chia Tai Wai Yi Leong ... David Ong Tat-Wee B. Liu William-Chandra Tjhi Erik Cambria Leslie Teo 36 12 0 08 Apr 2025
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training Yijie Zheng Bangjun Xiao Lei Shi Xiaoyang Li Faming Wu Tianyu Li Xuefeng Xiao Wenjie Qu Yue Wang Shouda Liu MLLM MoE 67 1 0 31 Mar 2025
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control Nvidia Hassan Abu Alhaija Jose M. Alvarez Maciej Bala Tiffany Cai ... Yuchong Ye Xiaodong Yang Boxin Wang Fangyin Wei Yu Zeng VGen 95 2 0 18 Mar 2025
Florenz: Scaling Laws for Systematic Generalization in Vision-Language Models Julian Spravil Sebastian Houben Sven Behnke VLM 75 0 0 12 Mar 2025
Seesaw: High-throughput LLM Inference via Model Re-sharding Qidong Su Wei Zhao X. Li Muralidhar Andoorveedu Chenhao Jiang Zhanda Zhu Kevin Song Christina Giannoula Gennady Pekhimenko LRM 77 0 0 09 Mar 2025
EuroBERT: Scaling Multilingual Encoders for European Languages Nicolas Boizard Hippolyte Gisserot-Boukhlef Duarte M. Alves André F. T. Martins Ayoub Hammal ... Maxime Peyrard Nuno M. Guerreiro Patrick Fernandes Ricardo Rei Pierre Colombo 125 1 0 07 Mar 2025
Mixtera: A Data Plane for Foundation Model Training Maximilian Böther Xiaozhe Yao Tolga Kerimoglu Ana Klimovic Viktor Gsteiger Ana Klimovic MoE 101 0 0 27 Feb 2025
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation Zhiyuan Liu Yanchen Luo Han Huang Enzhi Zhang Sihang Li Fan Zhang Yaorui Shi Xuben Wang Kenji Kawaguchi Tat-Seng Chua 102 3 0 18 Feb 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 102 15 0 17 Feb 2025
GoRA: Gradient-driven Adaptive Low Rank Adaptation Haonan He Peng Ye Yuchen Ren Yuan Yuan Lei Chen AI4TS AI4CE 172 0 0 13 Feb 2025
Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputers Siddharth Singh Prajwal Singhania Aditya K. Ranjan John Kirchenbauer Jonas Geiping ... Abhimanyu Hans Manli Shu Aditya Tomar Tom Goldstein A. Bhatele 102 2 0 12 Feb 2025
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation Hamed Firooz Maziar Sanjabi Adrian Englhardt Aman Gupta Ben Levine ... Xiaoling Zhai Ya Xu Yu Wang Yun Dai Yun Dai ALM 47 3 0 27 Jan 2025
Characterization of GPU TEE Overheads in Distributed Data Parallel ML Training Jonghytun Lee Yongqin Wang Rachit Rajat M. Annavaram 53 0 0 20 Jan 2025
LUSIFER: Language Universal Space Integration for Enhanced Multilingual Embeddings with Large Language Models Hieu Man Nghia Trung Ngo Viet Dac Lai Ryan A. Rossi Franck Dernoncourt T. Nguyen 160 0 0 01 Jan 2025
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 147 0 0 30 Dec 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 200 3 0 20 Nov 2024
Mixture of Parrots: Experts improve memorization more than reasoning Samy Jelassi Clara Mohri David Brandfonbrener Alex Gu Nikhil Vyas Nikhil Anand David Alvarez-Melis Yuanzhi Li Sham Kakade Eran Malach MoE 33 4 0 24 Oct 2024
Liger Kernel: Efficient Triton Kernels for LLM Training Pin-Lun Hsu Yun Dai Vignesh Kothapalli Qingquan Song Shao Tang Siyu Zhu Steven Shimizu Shivam Sahni Haowen Ning Yanning Chen 50 26 0 14 Oct 2024
Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions Zhihao He Hang Yu Zi Gong Shizhan Liu J. Li Weiyao Lin VLM 38 1 0 09 Oct 2024
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 147 0 0 07 Oct 2024
Training Language Models on Synthetic Edit Sequences Improves Code Synthesis Ulyana Piterbarg Lerrel Pinto Rob Fergus SyDa 37 2 0 03 Oct 2024
SSR: Alignment-Aware Modality Connector for Speech Language Models Weiting Tan Hirofumi Inaguma Ning Dong Paden Tomasello Xutai Ma 32 3 0 30 Sep 2024
Flash STU: Fast Spectral Transform Units Y. Isabel Liu Windsor Nguyen Yagiz Devre Evan Dogariu Anirudha Majumdar Elad Hazan AI4TS 72 1 0 16 Sep 2024
Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer Jinghan Yao Sam Ade Jacobs Masahiro Tanaka Olatunji Ruwase Hari Subramoni D. Panda 33 2 0 30 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Yu Qiao Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 67 48 0 05 Aug 2024
MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts Xi Lin Akshat Shrivastava Liang Luo Srinivasan Iyer Mike Lewis Gargi Gosh Luke Zettlemoyer Armen Aghajanyan MoE 43 20 0 31 Jul 2024
Small Molecule Optimization with Large Language Models Philipp Guevorguian Menua Bedrosian Tigran Fahradyan Gayane Chilingaryan Hrant Khachatrian Armen Aghajanyan 40 1 0 26 Jul 2024
DistML.js: Installation-free Distributed Deep Learning Framework for Web Browsers Masatoshi Hidaka Tomohiro Hashimoto Yuto Nishizawa Tatsuya Harada 24 2 0 01 Jul 2024
ProTrain: Efficient LLM Training via Memory-Aware Techniques Hanmei Yang Jin Zhou Yao Fu Xiaoqun Wang Ramine Roane Hui Guan Tongping Liu VLM 33 0 0 12 Jun 2024
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Peize Sun Yi Jiang Shoufa Chen Shilong Zhang Bingyue Peng Ping Luo Zehuan Yuan VLM 66 227 0 10 Jun 2024
Performance-Aligned LLMs for Generating Fast Code Daniel Nichols Pranav Polasam Harshitha Menon Aniruddha Marathe T. Gamblin A. Bhatele 35 8 0 29 Apr 2024
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data Sachin Mehta Maxwell Horton Fartash Faghri Mohammad Hossein Sekhavat Mahyar Najibi Mehrdad Farajtabar Oncel Tuzel Mohammad Rastegari VLM CLIP 44 6 0 24 Apr 2024
Linear Attention Sequence Parallelism Weigao Sun Zhen Qin Dong Li Xuyang Shen Yu Qiao Yiran Zhong 73 2 0 03 Apr 2024
Partitioned Neural Network Training via Synthetic Intermediate Labels C. V. Karadag Nezih Topaloglu 34 1 0 17 Mar 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 65 2 0 27 Feb 2024
Investigating the Effectiveness of HyperTuning via Gisting Jason Phang 46 0 0 26 Feb 2024
LLM4SecHW: Leveraging Domain Specific Large Language Model for Hardware Debugging Weimin Fu Kaichen Yang R. Dutta Xiaolong Guo Gang Qu 27 23 0 28 Jan 2024
DEAP: Design Space Exploration for DNN Accelerator Parallelism Ekansh Agrawal Xiangyu Sam Xu 26 1 0 24 Dec 2023
FlexModel: A Framework for Interpretability of Distributed Large Language Models Matthew Choi Muhammad Adil Asif John Willes David Emerson AI4CE ALM 27 1 0 05 Dec 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 53 11 0 23 Nov 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 34 93 0 27 Sep 2023
Benchmarks for Detecting Measurement Tampering Fabien Roger Ryan Greenblatt Max Nadeau Buck Shlegeris Nate Thomas 28 2 0 29 Aug 2023