Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

15 August 2013

Aaron Courville

Papers citing "Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation"

50 / 1,869 papers shown

Title
Pruning Large Language Models with Semi-Structural Adaptive Sparse Training Weiyu Huang Yuezhou Hu Guohao Jian Jun Zhu Jianfei Chen 35 5 0 30 Jul 2024
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey Jiangfei Duan Shuo Zhang Zerui Wang Lijuan Jiang Wenwen Qu ... Dahua Lin Yonggang Wen Xin Jin Tianwei Zhang Peng Sun 73 8 0 29 Jul 2024
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction Feijie Wu Xingchen Wang Yaqing Wang Tianci Liu Lu Su Jing Gao FedML 51 3 0 28 Jul 2024
Mixed Non-linear Quantization for Vision Transformers Gihwan Kim Jemin Lee Sihyeong Park Yongin Kwon Hyungshin Kim MQ 40 0 0 26 Jul 2024
Pixel Embedding: Fully Quantized Convolutional Neural Network with Differentiable Lookup Table Hiroyuki Tokunaga Joel Nicholls Daria Vazhenina Atsunori Kanemura MQ 18 1 0 23 Jul 2024
Revisiting Score Function Estimators for $k$ -Subset Sampling Klas Wijk Ricardo Vinuesa Hossein Azizpour TDI 32 1 0 22 Jul 2024
Decomposition of Neural Discrete Representations for Large-Scale 3D Mapping Minseong Park Suhan Woo Euntai Kim 3DV 33 0 0 22 Jul 2024
Differentiable Product Quantization for Memory Efficient Camera Relocalization Zakaria Laskar Iaroslav Melekhov Assia Benbihi Shuzhe Wang Arno Solin 53 2 0 22 Jul 2024
Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders Senthooran Rajamanoharan Tom Lieberum Nicolas Sonnerat Arthur Conmy Vikrant Varma János Kramár Neel Nanda 16 77 0 19 Jul 2024
Accurate Mapping of RNNs on Neuromorphic Hardware with Adaptive Spiking Neurons Gauthier Boeshertz Giacomo Indiveri M. Nair Alpha Renner 41 2 0 18 Jul 2024
Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors Matt Gorbett Hossein Shirazi Indrakshi Ray MQ 48 0 0 16 Jul 2024
Exploring Quantization for Efficient Pre-Training of Transformer Language Models Kamran Chitsaz Quentin Fournier Gonccalo Mordido Sarath Chandar MQ 49 3 0 16 Jul 2024
NITRO-D: Native Integer-only Training of Deep Convolutional Neural Networks Alberto Pirillo Luca Colombo Manuel Roveri MQ 37 0 0 16 Jul 2024
Scaling Diffusion Transformers to 16 Billion Parameters Zhengcong Fei Mingyuan Fan Changqian Yu Debang Li Junshi Huang DiffM MoE 65 16 0 16 Jul 2024
Q-Sparse: All Large Language Models can be Fully Sparsely-Activated Hongyu Wang Shuming Ma Ruiping Wang Furu Wei MoE 38 11 0 15 Jul 2024
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 48 1 0 13 Jul 2024
Trainable Highly-expressive Activation Functions Irit Chelly Shahaf E. Finder Shira Ifergane Oren Freifeld 31 4 0 10 Jul 2024
Dynamic Encoder Size Based on Data-Driven Layer-wise Pruning for Speech Recognition Jingjing Xu Wei Zhou Zijian Yang Eugen Beck Ralf Schlueter 41 1 0 10 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Yonghong Tian Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Ping Luo MQ 50 26 0 10 Jul 2024
FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation Liqun Ma Mingjie Sun Zhiqiang Shen 31 7 0 09 Jul 2024
Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules Zhuocheng Gong Ang Lv Jian Guan Junxi Yan Wei Wu Huishuai Zhang Minlie Huang Dongyan Zhao Rui Yan MoE 52 6 0 09 Jul 2024
OvSW: Overcoming Silent Weights for Accurate Binary Neural Networks Jingyang Xiang Zuohui Chen Siqi Li Qing Wu Yong-Jin Liu 31 1 0 07 Jul 2024
Scalable Variational Causal Discovery Unconstrained by Acyclicity Nu Hoang Bao Duong Thin Nguyen CML 58 0 0 06 Jul 2024
Balance of Number of Embedding and their Dimensions in Vector Quantization Hang Chen Sankepally Sainath Reddy Ziwei Chen Dianbo Liu 49 1 0 06 Jul 2024
Resource-Efficient Speech Quality Prediction through Quantization Aware Training and Binary Activation Maps Mattias Nilsson Riccardo Miccini Clément Laroche Tobias Piechowiak Friedemann Zenke MQ 34 0 0 05 Jul 2024
ISQuant: apply squant to the real deployment Dezan Zhao MQ 27 0 0 05 Jul 2024
Learning Interpretable Differentiable Logic Networks Chang Yue N. Jha NAI AI4CE 29 0 0 04 Jul 2024
Query-Guided Self-Supervised Summarization of Nursing Notes Ya Gao H. Moen S. Koivusalo M. Koskinen Pekka Marttinen 44 1 0 04 Jul 2024
Functional Faithfulness in the Wild: Circuit Discovery with Differentiable Computation Graph Pruning Lei Yu Jingcheng Niu Zining Zhu Gerald Penn 38 6 0 04 Jul 2024
Fisher-aware Quantization for DETR Detectors with Critical-category Objectives Huanrui Yang Yafeng Huang Zhen Dong Denis A. Gudovskiy Tomoyuki Okuno Yohei Nakata Yuan Du Kurt Keutzer Shanghang Zhang MQ 51 0 0 03 Jul 2024
Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point Bokun Wang Axel Berg D. A. E. Acar Chuteng Zhou FedML MQ 48 0 0 02 Jul 2024
Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects Yan Ge Wu Wenjie Chen Yuheng Pan Kaisen Lu Xudong Zhou Zixiang Wang Yuhan Wang Ruocheng Yan Junchi 35 14 0 30 Jun 2024
Kolmogorov-Smirnov GAN Maciej Falkiewicz Naoya Takeishi Alexandros Kalousis GAN 32 0 0 28 Jun 2024
Directly Training Temporal Spiking Neural Network with Sparse Surrogate Gradient Yang Li Feifei Zhao Dongcheng Zhao Yi Zeng 50 2 0 28 Jun 2024
Efficient World Models with Context-Aware Tokenization Vincent Micheli Eloi Alonso François Fleuret OffRL VLM 34 5 0 27 Jun 2024
OutlierTune: Efficient Channel-Wise Quantization for Large Language Models Jinguang Wang Yuexi Yin Haifeng Sun Qi Qi Jingyu Wang Zirui Zhuang Tingting Yang Jianxin Liao 46 2 0 27 Jun 2024
Neural Texture Block Compression S. Fujieda Takahiro Harada 34 0 0 27 Jun 2024
ViT-1.58b: Mobile Vision Transformers in the 1-bit Era Zhengqing Yuan Rong Zhou Hongyi Wang Lifang He Yanfang Ye Lichao Sun MQ 27 8 0 26 Jun 2024
Multimodal foundation world models for generalist embodied agents Pietro Mazzaglia Tim Verbelen Bart Dhoedt Rameswar Panda Sai Rajeswar OffRL LM&Ro 50 1 0 26 Jun 2024
Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers Lei Chen Yuan Meng Chen Tang Xinzhu Ma Jingyan Jiang Xin Wang Zhi Wang Wenwu Zhu MQ 31 23 0 25 Jun 2024
Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers Chao Lou Zixia Jia Zilong Zheng Kewei Tu ODL 35 19 0 24 Jun 2024
SimSMoE: Solving Representational Collapse via Similarity Measure Giang Do Hung Le T. Tran MoE 49 1 0 22 Jun 2024
BrowNNe: Brownian Nonlocal Neurons & Activation Functions Sriram Nagaraj Truman Hickok 31 0 0 21 Jun 2024
Older and Wiser: The Marriage of Device Aging and Intellectual Property Protection of Deep Neural Networks Ning Lin Shaocong Wang Yue Zhang Yangu He Kwunhang Wong Arindam Basu Dashan Shang Xiaoming Chen Zhongrui Wang AAML 33 0 0 21 Jun 2024
MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset Kim Sung-Bin Lee Chae-Yeon Gihun Son Oh Hyun-Bin Janghoon Ju Suekyeong Nam Tae-Hyun Oh 36 11 0 20 Jun 2024
HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment Yongqiang Chen Quanming Yao Juzheng Zhang James Cheng Yatao Bian 36 4 0 20 Jun 2024
Learned Compression of Encoding Distributions Mateen Ulhaq Ivan V. Bajić 20 1 0 18 Jun 2024
Bayesian-LoRA: LoRA based Parameter Efficient Fine-Tuning using Optimal Quantization levels and Rank Values trough Differentiable Bayesian Gates Cristian Meo Ksenia Sycheva Anirudh Goyal Justin Dauwels MQ 29 4 0 18 Jun 2024
TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation Haohao Qu Wenqi Fan Zihuai Zhao Qing Li 28 16 0 15 Jun 2024
Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling Siwei Zhang Xi Chen Yun Xiong Xixi Wu Yao Zhang Yongrui Fu Yinglong Zhao Jiawei Zhang 30 8 0 14 Jun 2024