A Generic Shared Attention Mechanism for Various Backbone Neural Networks

27 October 2022

Papers citing "A Generic Shared Attention Mechanism for Various Backbone Neural Networks"

22 / 22 papers shown

Title
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection Zhongzhan Huang Pan Zhou Shuicheng Yan Liang Lin 61 27 0 20 Oct 2023
ASR: Attention-alike Structural Re-parameterization Shan Zhong Zhongzhan Huang Wushao Wen Jinghui Qin Liang Lin 70 6 0 13 Apr 2023
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 75 6 0 16 Dec 2022
All are Worth Words: A ViT Backbone for Diffusion Models Fan Bao Shen Nie Kaiwen Xue Yue Cao Chongxuan Li Hang Su Jun Zhu VLM 66 343 0 25 Sep 2022
The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network Zhongzhan Huang Senwei Liang Mingfu Liang Wei He Haizhao Yang Liang Lin 54 9 0 16 Jul 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 49 124 0 14 Apr 2022
RePaint: Inpainting using Denoising Diffusion Probabilistic Models Andreas Lugmayr Martin Danelljan Andrés Romero Feng Yu Radu Timofte Luc Van Gool DiffM 310 1,385 0 24 Jan 2022
Evolving Attention with Residual Convolutions Yujing Wang Yaming Yang Jiangang Bai Mingliang Zhang Jing Bai Jiahao Yu Ce Zhang Gao Huang Yunhai Tong ViT 68 33 0 20 Feb 2021
Deep Neural Networks using a Single Neuron: Folded-in-Time Architecture using Feedback-Modulated Delay Loops Florian Stelzer André Röhm Raul Vicente Ingo Fischer University of Tartu AI4CE 50 46 0 19 Nov 2020
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond Yue Cao Jiarui Xu Stephen Lin Fangyun Wei Han Hu ISeg 68 1,561 0 25 Apr 2019
Towards Universal Object Detection by Domain Attention Xudong Wang Zhaowei Cai Dashan Gao Nuno Vasconcelos OOD 58 195 0 09 Apr 2019
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks Jie Hu Li Shen Samuel Albanie Gang Sun Andrea Vedaldi 66 573 0 29 Oct 2018
CBAM: Convolutional Block Attention Module Sanghyun Woo Jongchan Park Joon-Young Lee In So Kweon 187 16,337 0 17 Jul 2018
BAM: Bottleneck Attention Module Jongchan Park Sanghyun Woo Joon-Young Lee In So Kweon 56 1,035 0 17 Jul 2018
Massive Exploration of Neural Machine Translation Architectures D. Britz Anna Goldie Minh-Thang Luong Quoc V. Le 56 516 0 11 Mar 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 442 10,281 0 16 Nov 2016
Matching Networks for One Shot Learning Oriol Vinyals Charles Blundell Timothy Lillicrap Koray Kavukcuoglu Daan Wierstra VLM 299 7,299 0 13 Jun 2016
Identity Mappings in Deep Residual Networks Kaiming He Xinming Zhang Shaoqing Ren Jian Sun 295 10,149 0 16 Mar 2016
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 323 7,951 0 17 Aug 2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 286 10,034 0 10 Feb 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.2K 39,383 0 01 Sep 2014
Recurrent Models of Visual Attention Volodymyr Mnih N. Heess Alex Graves Koray Kavukcuoglu VLM 113 3,645 0 24 Jun 2014