Title
Differentiable Solver Search for Fast Diffusion Sampling Shuai Wang Zexian Li Qipeng Zhang Tianhui Song Xubin Li T. Ge Bo Zheng Limin Wang DiffM 103 0 0 27 May 2025
Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam Tianjin Huang Haotian Hu Zhenyu Zhang Gaojie Jin Xianrui Li ... Tianlong Chen Lu Liu Qingsong Wen Zhangyang Wang Shiwei Liu MQ 96 1 0 24 Feb 2025
TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data Leonardo Berti Gjergji Kasneci AI4TS 180 1 0 12 Feb 2025
Spectral-factorized Positive-definite Curvature Learning for NN Training Wu Lin Felix Dangel Runa Eschenhagen Juhan Bae Richard E. Turner Roger B. Grosse 138 0 0 10 Feb 2025
Distributed Sign Momentum with Local Steps for Training Transformers Shuhua Yu Ding Zhou Cong Xie An Xu Zhi-Li Zhang Xin Liu S. Kar 114 0 0 26 Nov 2024
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 214 9 0 25 Nov 2024
NeuralDEM -- Real-time Simulation of Industrial Particulate Flows Benedikt Alkin Tobias Kronlachner Samuele Papa Stefan Pirker Thomas Lichtenegger Johannes Brandstetter PINN AI4CE 109 3 1 14 Nov 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yaojie Lu Kurt Keutzer Jianfei Chen Song Han MQ 129 10 0 25 Oct 2024
Faster Acceleration for Steepest Descent Site Bai Brian Bullins ODL 77 1 0 28 Sep 2024
SOAP: Improving and Stabilizing Shampoo using Adam Nikhil Vyas Depen Morwani Rosie Zhao Itai Shapira David Brandfonbrener Lucas Janson Sham Kakade Sham Kakade 142 38 0 17 Sep 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 146 5 0 30 Jul 2024
Deconstructing What Makes a Good Optimizer for Language Models Rosie Zhao Depen Morwani David Brandfonbrener Nikhil Vyas Sham Kakade 103 25 0 10 Jul 2024
Latent Space Imaging Matheus Souza Yidan Zheng Kaizhang Kang Yogeshwar Nath Mishra Qiang Fu Wolfgang Heidrich 109 0 0 09 Jul 2024
Can Learned Optimization Make Reinforcement Learning Less Difficult? Alexander David Goldie Chris Xiaoxuan Lu Matthew Jackson Shimon Whiteson Jakob N. Foerster 111 4 0 09 Jul 2024
BADM: Batch ADMM for Deep Learning Ouya Wang Shenglong Zhou Geoffrey Ye Li ODL 107 1 0 30 Jun 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 164 3 0 26 May 2024
Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators Benedikt Alkin Andreas Fürst Simon Schmid Lukas Gruber Markus Holzleitner Johannes Brandstetter PINN AI4CE 200 13 0 19 Feb 2024
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources Zhikai Li Xiaoxuan Liu Banghua Zhu Zhen Dong Qingyi Gu Kurt Keutzer MQ 85 7 0 11 Oct 2023
Unified Functional Hashing in Automatic Machine Learning Ryan Gillard S. Jonany Yingjie Miao Michael Munn Connal de Souza Jonathan Dungay Chen Liang David R. So Quoc V. Le Esteban Real 87 2 0 10 Feb 2023
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 106 60 0 17 Nov 2022
Efficient Non-Parametric Optimizer Search for Diverse Tasks Ruochen Wang Yuanhao Xiong Minhao Cheng Cho-Jui Hsieh 86 5 0 27 Sep 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 196 3,963 0 26 Jul 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 461 6,067 0 23 May 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 169 1,307 0 04 May 2022
TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets Chengrun Yang Gabriel Bender Hanxiao Liu Pieter-Jan Kindermans Madeleine Udell Yifeng Lu Quoc V. Le Da Huang LMTD 54 8 0 15 Apr 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 126 231 0 21 Feb 2022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts Nan Du Yanping Huang Andrew M. Dai Simon Tong Dmitry Lepikhin ... Kun Zhang Quoc V. Le Yonghui Wu Zhiwen Chen Claire Cui ALM MoE 227 825 0 13 Dec 2021
LiT: Zero-Shot Transfer with Locked-image text Tuning Xiaohua Zhai Tianlin Li Basil Mustafa Andreas Steiner Daniel Keysers Alexander Kolesnikov Lucas Beyer VLM 108 560 0 15 Nov 2021
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving Ruochen Wang Xiangning Chen Minhao Cheng Xiaocheng Tang Cho-Jui Hsieh 95 13 0 18 Aug 2021
Rethinking Architecture Selection in Differentiable NAS Ruochen Wang Minhao Cheng Xiangning Chen Xiaocheng Tang Cho-Jui Hsieh 71 177 0 10 Aug 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 126 1,208 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 141 1,094 0 08 Jun 2021
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations Xiangning Chen Cho-Jui Hsieh Boqing Gong ViT 90 328 0 03 Jun 2021
Diffusion Models Beat GANs on Image Synthesis Prafulla Dhariwal Alex Nichol 265 7,938 0 11 May 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
PyGlove: Symbolic Programming for Automated Machine Learning Daiyi Peng Xuanyi Dong Esteban Real Mingxing Tan Yifeng Lu Hanxiao Liu Gabriel Bender Adam Kraft Chen Liang Quoc V. Le 55 31 0 21 Jan 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,430 0 22 Oct 2020
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients Juntang Zhuang Tommy M. Tang Yifan Ding S. Tatikonda Nicha Dvornek X. Papademetris James S. Duncan ODL 176 518 0 15 Oct 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 289 7,469 0 06 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,358 0 03 Oct 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 350 1,754 0 29 Jun 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 691 18,310 0 19 Jun 2020
DrNAS: Dirichlet Neural Architecture Search Xiangning Chen Ruochen Wang Minhao Cheng Xiaocheng Tang Cho-Jui Hsieh OOD 66 102 0 18 Jun 2020
Array Programming with NumPy Charles R. Harris K. Millman S. Walt R. Gommers Pauli Virtanen ... Tyler Reddy Warren Weckesser Hameer Abbasi C. Gohlke T. Oliphant 156 14,995 0 18 Jun 2020
Are we done with ImageNet? Lucas Beyer Olivier J. Hénaff Alexander Kolesnikov Xiaohua Zhai Aaron van den Oord VLM 131 405 0 12 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 873 42,379 0 28 May 2020
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch Esteban Real Chen Liang David R. So Quoc V. Le 77 225 0 06 Mar 2020
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization Xiangning Chen Cho-Jui Hsieh 81 204 0 12 Feb 2020
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 175 1,839 0 26 Nov 2019
Compressive Transformers for Long-Range Sequence Modelling Jack W. Rae Anna Potapenko Siddhant M. Jayakumar Timothy Lillicrap RALM VLM KELM 73 652 0 13 Nov 2019