Adaptive Computation Modules: Granular Conditional Computation For
Efficient Inference

Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference

15 December 2023

Bartosz Wójcik

Karol Pustelnik

Pasquale Minervini

Simone Scardapane

Papers citing "Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference"

9 / 9 papers shown

Title
Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts Youngseog Chung Dhruv Malik J. Schneider Yuanzhi Li Aarti Singh MoE 36 1 0 02 Sep 2024
Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning Alessio Devoto Federico Alvetreti Jary Pomponi P. Lorenzo Pasquale Minervini Simone Scardapane 51 2 0 16 Aug 2024
Adaptive Semantic Token Selection for AI-native Goal-oriented Communications Alessio Devoto Simone Petruzzi Jary Pomponi P. Lorenzo Simone Scardapane 39 2 0 25 Apr 2024
Conditional computation in neural networks: principles and research trends Simone Scardapane Alessandro Baiocchi Alessio Devoto V. Marsocci Pasquale Minervini Jary Pomponi 34 1 0 12 Mar 2024
From Sparse to Soft Mixtures of Experts J. Puigcerver C. Riquelme Basil Mustafa N. Houlsby MoE 121 114 0 02 Aug 2023
Mixture of Attention Heads: Selecting Attention Heads Per Token Xiaofeng Zhang Yikang Shen Zeyu Huang Jie Zhou Wenge Rong Zhang Xiong MoE 99 42 0 11 Oct 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 160 327 0 18 Feb 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 141 684 0 31 Jan 2021
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,198 0 01 Sep 2014