Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

15 August 2013

Aaron Courville

Papers citing "Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation"

50 / 1,517 papers shown

Title
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 203 1,199 0 30 Jun 2020
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization Sang Michael Xie Tengyu Ma Percy Liang 135 15 0 29 Jun 2020
Lattice Representation Learning Luis A. Lastras 42 1 0 24 Jun 2020
The Depth-to-Width Interplay in Self-Attention Yoav Levine Noam Wies Or Sharir Hofit Bata Amnon Shashua 137 46 0 22 Jun 2020
IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression Rianne van den Berg A. Gritsenko Mostafa Dehghani C. Sønderby Tim Salimans 92 61 0 22 Jun 2020
Generating Annotated High-Fidelity Images Containing Multiple Coherent Objects Bryan G. Cardenas Devanshu Arya D. K. Gupta DiffM 108 6 0 22 Jun 2020
Modeling Lost Information in Lossy Image Compression Yaolong Wang Mingqing Xiao Chang-Shu Liu Shuxin Zheng Tie-Yan Liu 69 23 0 22 Jun 2020
DisARM: An Antithetic Gradient Estimator for Binary Latent Variables Zhe Dong A. Mnih George Tucker DRL 71 34 0 18 Jun 2020
Universally Quantized Neural Compression E. Agustsson Lucas Theis MQ 132 90 0 17 Jun 2020
Generative Semantic Hashing Enhanced via Boltzmann Machines Lin Zheng Qinliang Su Dinghan Shen Changyou Chen 55 6 0 16 Jun 2020
Towards Understanding the Effect of Leak in Spiking Neural Networks Sayeed Shafayet Chowdhury Chankyu Lee Kaushik Roy 60 59 0 15 Jun 2020
Self-supervised Learning: Generative or Contrastive Xiao Liu Fanjin Zhang Zhenyu Hou Zhaoyu Wang Li Mian Jing Zhang Jie Tang SSL 223 1,650 0 15 Jun 2020
Meta Approach to Data Augmentation Optimization Ryuichiro Hataya Jan Zdenek Kazuki Yoshizoe Hideki Nakayama 84 35 0 14 Jun 2020
CoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs Zhen Dong Dequan Wang Qijing Huang Yizhao Gao Yaohui Cai Tian Li Bichen Wu Kurt Keutzer J. Wawrzynek ObjD 59 1 0 12 Jun 2020
Dynamic Model Pruning with Feedback Tao R. Lin Sebastian U. Stich Luis Barba Daniil Dmitriev Martin Jaggi 167 204 0 12 Jun 2020
Ensemble Distillation for Robust Model Fusion in Federated Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi FedML 149 1,063 0 12 Jun 2020
Surrogate gradients for analog neuromorphic computing Benjamin Cramer Sebastian Billaudelle Simeon Kanya Aron Leibfried Andreas Grubl ... Korbinian Schreiber Yannik Stradmann Johannes Weis Johannes Schemmel Friedemann Zenke 79 110 0 12 Jun 2020
Reintroducing Straight-Through Estimators as Principled Methods for Stochastic Binary Networks Alexander Shekhovtsov Dmitry Molchanov MQ 85 16 0 11 Jun 2020
Data Augmentation for Graph Neural Networks Tong Zhao Yozen Liu Leonardo Neves Oliver J. Woodford Meng Jiang Neil Shah GNN 166 419 0 11 Jun 2020
DNF-Net: A Neural Architecture for Tabular Data A. Abutbul G. Elidan L. Katzir Ran El-Yaniv LMTD AI4CE 55 29 0 11 Jun 2020
Improving Inference for Neural Image Compression Yibo Yang Robert Bamler Stephan Mandt 97 123 0 07 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 160 100 0 05 Jun 2020
Real-time Human Activity Recognition Using Conditionally Parametrized Convolutions on Mobile and Wearable Devices Xin-Hua Cheng Lefei Zhang Yin Tang Yue Liu Hao Wu Jun He CVBM 3DH HAI 73 55 0 05 Jun 2020
Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks Alexander Shekhovtsov V. Yanush B. Flach MQ 80 11 0 04 Jun 2020
Weight Pruning via Adaptive Sparsity Loss George Retsinas Athena Elafrou G. Goumas Petros Maragos 64 10 0 04 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 1.2K 42,750 0 28 May 2020
In search of isoglosses: continuous and discrete language embeddings in Slavic historical phonology C. Cathcart Florian Wandl 47 6 0 27 May 2020
Generating Semantically Valid Adversarial Questions for TableQA Yi Zhu Yiwei Zhou Menglin Xia AAML 63 5 0 26 May 2020
Fast differentiable DNA and protein sequence optimization for molecular design Johannes Linder Georg Seelig 66 60 0 22 May 2020
Pairwise Supervised Hashing with Bernoulli Variational Auto-Encoder and Self-Control Gradient Estimator Siamak Zamani Dadaneh Shahin Boluki Mingzhang Yin Mingyuan Zhou Xiaoning Qian BDL DRL 41 22 0 21 May 2020
Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems Chang Zhou Jianxin Ma Jianwei Zhang Jingren Zhou Hongxia Yang 159 148 0 20 May 2020
Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge Benjamin van Niekerk Leanne Nortje Herman Kamper 123 117 0 19 May 2020
COVI White Paper H. Alsdurf Edmond Belliveau Yoshua Bengio T. Deleu Prateek Gupta ... Abhinav Sharma B. Struck Jian Tang Martin Weiss Y. Yu 74 29 0 18 May 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh Thomas Wolf Alexander M. Rush 107 489 0 15 May 2020
Bayesian Bits: Unifying Quantization and Pruning M. V. Baalen Christos Louizos Markus Nagel Rana Ali Amjad Ying Wang Tijmen Blankevoort Max Welling MQ 97 116 0 14 May 2020
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers Junjie Liu Zhe Xu Runbin Shi R. Cheung Hayden Kwok-Hay So 62 122 0 14 May 2020
Invertible Image Rescaling Mingqing Xiao Shuxin Zheng Chang-Shu Liu Yaolong Wang Di He Guolin Ke Jiang Bian Zhouchen Lin Tie-Yan Liu SupR 95 241 0 12 May 2020
Lossy Compression with Distortion Constrained Optimization T. V. Rozendaal Guillaume Sautière Taco S. Cohen 68 13 0 08 May 2020
Efficient Exact Verification of Binarized Neural Networks Kai Jia Martin Rinard AAML MQ 48 59 0 07 May 2020
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation Lifu Tu Richard Yuanzhe Pang Sam Wiseman Kevin Gimpel 103 53 0 02 May 2020
Hide-and-Seek: A Template for Explainable AI Thanos Tagaris A. Stafylopatis 26 6 0 30 Apr 2020
On the Spontaneous Emergence of Discrete and Compositional Signals Nur Lan Emmanuel Chemla Shane Steinert-Threlkeld LRM 75 8 0 30 Apr 2020
Generative Adversarial Networks (GANs Survey): Challenges, Solutions, and Future Directions Divya Saxena Jiannong Cao AAML AI4CE 158 307 0 30 Apr 2020
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking Nicola De Cao Michael Schlichtkrull Wilker Aziz Ivan Titov 76 92 0 30 Apr 2020
Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing A. Mehonic Abu Sebastian Bipin Rajendran Osvaldo Simeone Eleni Vasilaki A. Kenyon 62 212 0 30 Apr 2020
Polygonal Building Segmentation by Frame Field Learning N. Girard Dmitriy Smirnov Justin Solomon Y. Tarabalka 107 26 0 30 Apr 2020
Towards Unsupervised Language Understanding and Generation by Joint Dual Learning Shang-Yu Su Chao-Wei Huang Yun-Nung Chen 89 26 0 30 Apr 2020
Faster Depth-Adaptive Transformers Yijin Liu Fandong Meng Jie Zhou Jinan Xu Jinan Xu 57 2 0 27 Apr 2020
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models Mengjie Zhao Tao R. Lin Fei Mi Martin Jaggi Hinrich Schütze 77 121 0 26 Apr 2020
The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget Anirudh Goyal Yoshua Bengio M. Botvinick Sergey Levine 70 24 0 24 Apr 2020