Mixed Precision Training

10 October 2017

Paulius Micikevicius

Sharan Narang

Boris Ginsburg

Papers citing "Mixed Precision Training"

50 / 380 papers shown

Title
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting Kai Zhang Sai Bi Hao Tan Yuanbo Xiangli Nanxuan Zhao Kalyan Sunkavalli Zexiang Xu 3DGS 39 124 0 30 Apr 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent Pingzhi Li Junyu Liu Hanrui Wang Tianlong Chen 86 1 0 30 Apr 2024
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data Sachin Mehta Maxwell Horton Fartash Faghri Mohammad Hossein Sekhavat Mahyar Najibi Mehrdad Farajtabar Oncel Tuzel Mohammad Rastegari VLM CLIP 44 6 0 24 Apr 2024
A Multi-Level Framework for Accelerating Training Transformer Models Longwei Zou Han Zhang Yangdong Deng AI4CE 40 1 0 07 Apr 2024
Accurate Block Quantization in LLMs with Outliers Nikita Trukhanov I. Soloveychik MQ 28 4 0 29 Mar 2024
DITTO: Dual and Integrated Latent Topologies for Implicit 3D Reconstruction Jaehyeok Shim Kyungdon Joo 3DPC 3DV 48 1 0 08 Mar 2024
Better Schedules for Low Precision Training of Deep Neural Networks Cameron R. Wolfe Anastasios Kyrillidis 47 1 0 04 Mar 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 65 2 0 27 Feb 2024
Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot Fabien Baradel M. Armando Salma Galaaoui Romain Brégier Philippe Weinzaepfel Grégory Rogez Thomas Lucas 3DH 39 18 0 22 Feb 2024
Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation S. Landgraf Markus Hillemann Theodor Kapler Markus Ulrich UQCV 28 8 0 16 Feb 2024
Flora: Low-Rank Adapters Are Secretly Gradient Compressors Yongchang Hao Yanshuai Cao Lili Mou 16 40 0 05 Feb 2024
Nomic Embed: Training a Reproducible Long Context Text Embedder Zach Nussbaum John X. Morris Brandon Duderstadt Andriy Mulyar 27 96 0 02 Feb 2024
Speeding up and reducing memory usage for scientific machine learning via mixed precision Joel Hayford Jacob Goldman-Wetzler Eric Wang Lu Lu 49 8 0 30 Jan 2024
RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture M. A. D. L. Balaguer Vinamra Benara Renato Luiz de Freitas Cunha Roberto de M. Estevao Filho Todd Hendry ... Morris Sharp B. Silva Swati Sharma Vijay Aski Ranveer Chandra FaML 38 82 0 16 Jan 2024
Knowledge Translation: A New Pathway for Model Compression Wujie Sun Defang Chen Jiawei Chen Yan Feng Chun-Yen Chen Can Wang 25 0 0 11 Jan 2024
Enhancing Contrastive Learning with Efficient Combinatorial Positive Pairing Jaeill Kim Duhun Hwang Eunjung Lee Jangwon Suh Jimyeong Kim Wonjong Rhee 33 0 0 11 Jan 2024
Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition Vahid Noroozi Somshubra Majumdar Ankur Kumar Jagadeesh Balam Boris Ginsburg 27 10 0 27 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 29 1 0 18 Dec 2023
LLM360: Towards Fully Transparent Open-Source LLMs Zhengzhong Liu Aurick Qiao W. Neiswanger Hongyi Wang Bowen Tan ... Zhiting Hu Mark Schulze Preslav Nakov Timothy Baldwin Eric P. Xing 49 70 0 11 Dec 2023
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC Wu Lin Felix Dangel Runa Eschenhagen Kirill Neklyudov Agustinus Kristiadi Richard Turner Alireza Makhzani 22 3 0 09 Dec 2023
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness Xiaoyun Xu Shujian Yu Jingzheng Wu S. Picek AAML 35 0 0 08 Dec 2023
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames Shuming Liu Chen-Da Liu-Zhang Chen Zhao Guohao Li 36 25 0 28 Nov 2023
LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Language Aunabil Chakma Masum Hasan 44 3 0 21 Nov 2023
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model Jiahao Li Hao Tan Kai Zhang Zexiang Xu Fujun Luan Yinghao Xu Yicong Hong Kalyan Sunkavalli Greg Shakhnarovich Sai Bi 59 254 0 10 Nov 2023
ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences Yuanhe Tian Ruyi Gan Yan Song Jiaxing Zhang Yongdong Zhang AI4MH AI4CE LM&MA 27 31 0 10 Nov 2023
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources Zhikai Li Xiaoxuan Liu Banghua Zhu Zhen Dong Qingyi Gu Kurt Keutzer MQ 32 7 0 11 Oct 2023
AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ Jonas Belouadi Anne Lauscher Steffen Eger 21 28 0 30 Sep 2023
Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI Dustin Wright Christian Igel Gabrielle Samuel Raghavendra Selvan 32 15 0 05 Sep 2023
kTrans: Knowledge-Aware Transformer for Binary Code Embedding Wenyu Zhu Hao Wang Yuchen Zhou Jiaming Wang Zihan Sha Zeyu Gao Chao Zhang 32 10 0 24 Aug 2023
Towards General Text Embeddings with Multi-stage Contrastive Learning Zehan Li Xin Zhang Yanzhao Zhang Dingkun Long Pengjun Xie Meishan Zhang 59 351 0 07 Aug 2023
UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming Hao Lin Ke Wu Jie Li Jun Yu Li Wu-Jun Li 39 1 0 31 Jul 2023
U-CE: Uncertainty-aware Cross-Entropy for Semantic Segmentation S. Landgraf Markus Hillemann Kira Wursthorn Markus Ulrich SSeg UQCV 26 6 0 19 Jul 2023
Accelerating Distributed ML Training via Selective Synchronization S. Tyagi Martin Swany FedML 32 3 0 16 Jul 2023
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models Jean Kaddour Oscar Key Piotr Nawrot Pasquale Minervini Matt J. Kusner 22 41 0 12 Jul 2023
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels Bang-ju Yang Fenglin Liu Zheng Li Qingyu Yin Chenyu You Bing Yin Yuexian Zou VLM 36 5 0 05 Jul 2023
Breaking On-device Training Memory Wall: A Systematic Survey Shitian Li Chunlin Tian Kahou Tam Ruirui Ma Li Li 23 2 0 17 Jun 2023
Full Parameter Fine-tuning for Large Language Models with Limited Resources Kai Lv Yuqing Yang Tengxiao Liu Qi-jie Gao Qipeng Guo Xipeng Qiu 47 127 0 16 Jun 2023
MobileNMT: Enabling Translation in 15MB and 30ms Ye Lin Xiaohui Wang Zhexi Zhang Mingxuan Wang Tong Xiao Jingbo Zhu MQ 30 1 0 07 Jun 2023
NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks Jean-Michel Attendu Jean-Philippe Corbeil 35 15 0 05 Jun 2023
A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics Hong-Yu Zhou Yizhou Yu Chengdi Wang Shu Zhen Zhang Yuanxu Gao Jia-Yu Pan Jun Shao Guangming Lu Kang Zhang Weimin Li MedIm 19 150 0 01 Jun 2023
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning Baohao Liao Shaomu Tan Christof Monz KELM 23 29 0 01 Jun 2023
Thought Cloning: Learning to Think while Acting by Imitating Human Thinking Shengran Hu Jeff Clune LM&Ro OffRL LRM AI4CE 35 27 0 01 Jun 2023
SSSegmenation: An Open Source Supervised Semantic Segmentation Toolbox Based on PyTorch Zhenchao Jin VLM 43 4 0 26 May 2023
CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model Shuai Zhao Xiaohan Wang Linchao Zhu Yezhou Yang CLIP VLM 23 25 0 23 May 2023
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Efficient ConvBN Blocks for Transfer Learning and Beyond Kaichao You Guo Qin Anchang Bao Mengsi Cao Ping-Chia Huang Jiulong Shan Mingsheng Long 31 1 0 19 May 2023
mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra Chenhao Shuai Chaohua Shi Lu Gan Hongqing Liu 30 8 0 18 May 2023
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation Ye Lin Shuhan Zhou Yanyang Li Anxiang Ma Tong Xiao Jingbo Zhu 38 0 0 10 May 2023
ArgU: A Controllable Factual Argument Generator Sougata Saha Rohini Srihari 22 13 0 09 May 2023
TASTY: A Transformer based Approach to Space and Time complexity K. Moudgalya Ankit Ramakrishnan Vamsikrishna Chemudupati Xinghai Lu 16 3 0 06 May 2023