Mixed Precision Training

10 October 2017

Paulius Micikevicius

Sharan Narang

Boris Ginsburg

Papers citing "Mixed Precision Training"

50 / 380 papers shown

Title
FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation Ahmad Shawahna S. M. Sait A. El-Maleh Irfan Ahmad MQ 20 6 0 22 Mar 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 36 200 0 14 Mar 2022
FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support Seock-Hwan Noh Jahyun Koo Seunghyun Lee Jongse Park Jaeha Kung AI4CE 32 17 0 13 Mar 2022
A Mixed Quantization Network for Computationally Efficient Mobile Inverse Tone Mapping Juan Borrego-Carazo Mete Ozay Frederik Laboyrie Paul Wisbey MQ 21 0 0 12 Mar 2022
Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval Luyu Gao Xueguang Ma Jimmy J. Lin Jamie Callan 34 76 0 11 Mar 2022
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection Hao Zhang Feng Li Shilong Liu Lei Zhang Hang Su Jun Zhu L. Ni H. Shum ViT 59 1,375 0 07 Mar 2022
Carbon Footprint of Selecting and Training Deep Learning Models for Medical Image Analysis Raghavendra Selvan N. Bhagwat Lasse F. Wolff Anthony Benjamin Kanding Erik Dam 36 24 0 04 Mar 2022
Debiased Batch Normalization via Gaussian Process for Generalizable Person Re-Identification Jiawei Liu Zhipeng Huang Liang-Sheng Li Kecheng Zheng Zhengjun Zha 28 26 0 03 Mar 2022
Engineering the Neural Automatic Passenger Counter Nico Jahn Michael Siebert 13 2 0 02 Mar 2022
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 19 7 0 28 Feb 2022
GNN-Surrogate: A Hierarchical and Adaptive Graph Neural Network for Parameter Space Exploration of Unstructured-Mesh Ocean Simulations Neng Shi Jiayi Xu Skylar W. Wurster Hanqi Guo J. Woodring L. V. Roekel Han-Wei Shen AI4TS AI4CE 34 32 0 18 Feb 2022
ST-MoE: Designing Stable and Transferable Sparse Expert Models Barret Zoph Irwan Bello Sameer Kumar Nan Du Yanping Huang J. Dean Noam M. Shazeer W. Fedus MoE 24 181 0 17 Feb 2022
Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines Alexander Isenko R. Mayer Jeffrey Jedele Hans-Arno Jacobsen 19 23 0 17 Feb 2022
pNLP-Mixer: an Efficient all-MLP Architecture for Language Francesco Fusco Damian Pascual Peter W. J. Staar Diego Antognini 37 29 0 09 Feb 2022
Backdoor Defense via Decoupling the Training Process Kunzhe Huang Yiming Li Baoyuan Wu Zhan Qin Kui Ren AAML FedML 27 187 0 05 Feb 2022
Accelerating DNN Training with Structured Data Gradient Pruning Bradley McDanel Helia Dinh J. Magallanes 17 7 0 01 Feb 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... M. Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 90 730 0 28 Jan 2022
Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation Zhuoyuan Mao Chenhui Chu Sadao Kurohashi 17 6 0 20 Jan 2022
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding Thomas Müller Alex Evans Christoph Schied A. Keller 81 3,869 0 16 Jan 2022
SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps Carlos Rodriguez-Pardo Elena Garces GAN 12 24 0 13 Jan 2022
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation Han Zhang Weichong Yin Yewei Fang Lanxin Li Boqiang Duan Zhihua Wu Yu Sun Hao Tian Hua Wu Haifeng Wang 27 58 0 31 Dec 2021
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 96 3,483 0 20 Dec 2021
Efficient Large Scale Language Modeling with Mixtures of Experts Mikel Artetxe Shruti Bhosale Naman Goyal Todor Mihaylov Myle Ott ... Jeff Wang Luke Zettlemoyer Mona T. Diab Zornitsa Kozareva Ves Stoyanov MoE 61 188 0 20 Dec 2021
AdaViT: Adaptive Tokens for Efficient Vision Transformer Hongxu Yin Arash Vahdat J. Álvarez Arun Mallya Jan Kautz Pavlo Molchanov ViT 35 314 0 14 Dec 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 31 879 0 22 Nov 2021
Mesa: A Memory-saving Training Framework for Transformers Zizheng Pan Peng Chen Haoyu He Jing Liu Jianfei Cai Bohan Zhuang 31 20 0 22 Nov 2021
COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression Sian Jin Chengming Zhang Xintong Jiang Yunhe Feng Hui Guan Guanpeng Li Shuaiwen Leon Song Dingwen Tao 27 23 0 18 Nov 2021
Scaling Law for Recommendation Models: Towards General-purpose User Representations Kyuyong Shin Hanock Kwak KyungHyun Kim Max Nihlén Ramström Jisu Jeong Jung-Woo Ha S. Kim ELM 36 38 0 15 Nov 2021
FILIP: Fine-grained Interactive Language-Image Pre-Training Lewei Yao Runhu Huang Lu Hou Guansong Lu Minzhe Niu Hang Xu Xiaodan Liang Zhenguo Li Xin Jiang Chunjing Xu VLM CLIP 30 615 0 09 Nov 2021
Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training Minguk Kang Woohyeon Shim Minsu Cho Jaesik Park GAN 39 108 0 01 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 30 14 0 01 Nov 2021
MetaICL: Learning to Learn In Context Sewon Min M. Lewis Luke Zettlemoyer Hannaneh Hajishirzi LRM 61 467 0 29 Oct 2021
BitTrain: Sparse Bitmap Compression for Memory-Efficient Training on the Edge Abdelrahman I. Hosny Marina Neseem Sherief Reda MQ 35 4 0 29 Oct 2021
Whole Brain Segmentation with Full Volume Neural Network Yeshu Li Jianwei Cui Yilun Sheng Xiao Liang Jingdong Wang E. Chang Yan Xu 32 11 0 29 Oct 2021
AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning Siddharth Singh A. Bhatele GNN 34 14 0 25 Oct 2021
Time-domain Ad-hoc Array Speech Enhancement Using a Triple-path Network Ashutosh Pandey Buye Xu Anurag Kumar Jacob Donley P. Calamia DeLiang Wang 29 4 0 22 Oct 2021
TPARN: Triple-path Attentive Recurrent Network for Time-domain Multichannel Speech Enhancement Ashutosh Pandey Buye Xu Anurag Kumar Jacob Donley P. Calamia DeLiang Wang KELM 19 40 0 20 Oct 2021
NormFormer: Improved Transformer Pretraining with Extra Normalization Sam Shleifer Jason Weston Myle Ott AI4CE 33 74 0 18 Oct 2021
TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models S. Chatterjee Arnab Das Chirag Mandal Budhaditya Mukhopadhyay Manish Vipinraj Aniruddh Shukla R. Rao Chompunuch Sarasaen Oliver Speck A. Nürnberger MedIm 37 14 0 16 Oct 2021
ByteTrack: Multi-Object Tracking by Associating Every Detection Box Yifu Zhang Pei Sun Yi-Xin Jiang Dongdong Yu Fucheng Weng Zehuan Yuan Ping Luo Wenyu Liu Xinggang Wang VOT 107 1,330 0 13 Oct 2021
NoisyActions2M: A Multimedia Dataset for Video Understanding from Noisy Labels Mohit Sharma Rajkumar Patra Harshali Desai Shruti Vyas Yogesh S Rawat R. Shah VGen NoLa 16 3 0 13 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 347 0 13 Oct 2021
Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning Chongjian Ge Youwei Liang Yibing Song Jianbo Jiao Jue Wang Ping Luo ViT 24 36 0 11 Oct 2021
Optimized U-Net for Brain Tumor Segmentation Michal Futrega Alexandre Milesi Michal Marcinkiewicz Pablo Ribalta SSeg 32 91 0 07 Oct 2021
8-bit Optimizers via Block-wise Quantization Tim Dettmers M. Lewis Sam Shleifer Luke Zettlemoyer MQ 34 273 0 06 Oct 2021
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation Md. Akmal Haidar Nithin Anchuri Mehdi Rezagholizadeh Abbas Ghaddar Philippe Langlais Pascal Poupart 31 22 0 21 Sep 2021
Multilingual Translation via Grafting Pre-trained Language Models Zewei Sun Mingxuan Wang Lei Li AI4CE 191 22 0 11 Sep 2021
PPT: Pre-trained Prompt Tuning for Few-shot Learning Yuxian Gu Xu Han Zhiyuan Liu Minlie Huang VLM 54 403 0 09 Sep 2021
A Protection Method of Trained CNN Model Using Feature Maps Transformed With Secret Key From Unauthorized Access Maungmaung Aprilpyone Hitoshi Kiya 16 5 0 01 Sep 2021
EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTa Taewoon Kim Piek Vossen 25 98 0 26 Aug 2021