v1v2 (latest)

ASR: Attention-alike Structural Re-parameterization

13 April 2023

Papers citing "ASR: Attention-alike Structural Re-parameterization"

50 / 59 papers shown

Title
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection Zhongzhan Huang Pan Zhou Shuicheng Yan Liang Lin 77 27 0 20 Oct 2023
Understanding Self-attention Mechanism via Dynamical System Perspective Zhongzhan Huang Mingfu Liang Jinghui Qin Shan Zhong Liang Lin 63 15 0 19 Aug 2023
LSAS: Lightweight Sub-attention Strategy for Alleviating Attention Bias Problem Shan Zhong Wushao Wen Jinghui Qin Qiang Chen Zhongzhan Huang 85 8 0 09 May 2023
AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarios Zhongzhan Huang Mingfu Liang Liang Lin Liang Lin 76 5 0 05 Feb 2023
Mix-Pooling Strategy for Attention Mechanism Shan Zhong Wushao Wen Jinghui Qin 69 3 0 22 Aug 2022
The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network Zhongzhan Huang Senwei Liang Mingfu Liang Wei He Haizhao Yang Liang Lin 65 9 0 16 Jul 2022
Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment Takeshi Kojima Yutaka Matsuo Yusuke Iwasawa 88 28 0 28 Jun 2022
MDMLP: Image Classification from Scratch on Small Datasets with MLP Tianxu Lv Chongyang Bai Chaojie Wang 61 6 0 28 May 2022
RepSR: Training Efficient VGG-style Super-Resolution Networks with Structural Re-Parameterization and Batch Normalization Xintao Wang Chao Dong Ying Shan 83 48 0 11 May 2022
DyRep: Bootstrapping Training with Dynamic Re-parameterization Tao Huang Shan You Bohan Zhang Yuxuan Du Fei Wang Chao Qian Chang Xu 70 27 0 24 Mar 2022
MetaFormer Is Actually What You Need for Vision Weihao Yu Mi Luo Pan Zhou Chenyang Si Yichen Zhou Xinchao Wang Jiashi Feng Shuicheng Yan 165 911 0 22 Nov 2021
CS-Rep: Making Speaker Verification Networks Embracing Re-parameterization Ruiteng Zhang Jianguo Wei Wenhuan Lu Lin Zhang Y. Ji Junhai Xu Xugang Lu 42 10 0 26 Oct 2021
RepNAS: Searching for Efficient Re-parameterizing Blocks Mingyang Zhang Xinyi Yu Jingtao Rong L. Ou 61 20 0 08 Sep 2021
Blending Pruning Criteria for Convolutional Neural Networks Wei He Zhongzhan Huang Mingfu Liang Senwei Liang Haizhao Yang CVBM 46 15 0 11 Jul 2021
ResMLP: Feedforward networks for image classification with data-efficient training Hugo Touvron Piotr Bojanowski Mathilde Caron Matthieu Cord Alaaeldin El-Nouby ... Gautier Izacard Armand Joulin Gabriel Synnaeve Jakob Verbeek Hervé Jégou VLM 77 664 0 07 May 2021
RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition Xiaohan Ding Chunlong Xia Xinming Zhang Xiaojie Chu Jungong Han Guiguang Ding 57 94 0 05 May 2021
Diverse Branch Block: Building a Convolution as an Inception-like Unit Xiaohan Ding Xinming Zhang Jungong Han Guiguang Ding AI4CE 66 288 0 24 Mar 2021
Coordinate Attention for Efficient Mobile Network Design Qibin Hou Daquan Zhou Jiashi Feng 83 3,063 0 04 Mar 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 133 1,939 0 28 Jan 2021
RepVGG: Making VGG-style ConvNets Great Again Xiaohan Ding Xinming Zhang Ningning Ma Jungong Han Guiguang Ding Jian Sun 284 1,599 0 11 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 387 6,768 0 23 Dec 2020
FcaNet: Frequency Channel Attention Networks Zequn Qin Pengyi Zhang Leilei Gan Xi Li 87 708 0 22 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
DO-Conv: Depthwise Over-parameterized Convolutional Layer Jinming Cao Yangyan Li Mingchao Sun Ying-Cong Chen Dani Lischinski Daniel Cohen-Or Baoquan Chen Changhe Tu OOD 66 172 0 22 Jun 2020
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning Zhongzhan Huang Wenqi Shao Xinjiang Wang Liang Lin Ping Luo 47 54 0 24 Apr 2020
ResNeSt: Split-Attention Networks Hang Zhang Chongruo Wu Zhongyue Zhang Yi Zhu Yanghua Peng ... Tong He Jonas W. Mueller R. Manmatha Mu Li Alex Smola 104 1,477 0 19 Apr 2020
Efficient Differentiable Neural Architecture Search with Meta Kernels Shoufa Chen Yunpeng Chen Shuicheng Yan Jiashi Feng 53 19 0 10 Dec 2019
Dynamical System Inspired Adaptive Time Stepping Controller for Residual Network Families Yibo Yang Jianlong Wu Hongyang Li Xia Li Tiancheng Shen Zhouchen Lin OOD 41 21 0 23 Nov 2019
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks Qilong Wang Banggu Wu Peng Fei Zhu P. Li W. Zuo Q. Hu 140 4,020 0 08 Oct 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 74 58 0 12 Aug 2019
ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks Xiaohan Ding Yuchen Guo Guiguang Ding Jiawei Han 68 674 0 11 Aug 2019
DIANet: Dense-and-Implicit Attention Network Zhongzhan Huang Senwei Liang Mingfu Liang Haizhao Yang CVBM 66 57 0 25 May 2019
Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks Xiang Li Xiaolin Hu Jian Yang 48 196 0 23 May 2019
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond Yue Cao Jiarui Xu Stephen Lin Fangyun Wei Han Hu ISeg 80 1,571 0 25 Apr 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 132 1,014 0 22 Apr 2019
An Empirical Study of Spatial Attention Mechanisms in Deep Networks Xizhou Zhu Dazhi Cheng Zheng Zhang Stephen Lin Jifeng Dai 79 414 0 11 Apr 2019
SRM : A Style-based Recalibration Module for Convolutional Neural Networks HyunJae Lee Hyo-Eun Kim Hyeonseob Nam 55 225 0 26 Mar 2019
Selective Kernel Networks Xiang Li Wenhai Wang Xiaolin Hu Jian Yang 94 2,035 0 15 Mar 2019
Global Second-order Pooling Convolutional Networks Zilin Gao Jiangtao Xie Qilong Wang P. Li 69 335 0 29 Nov 2018
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks Shuxuan Guo J. Álvarez Mathieu Salzmann 69 80 0 26 Nov 2018
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks Jie Hu Li Shen Samuel Albanie Gang Sun Andrea Vedaldi 73 576 0 29 Oct 2018
Dual Attention Network for Scene Segmentation J. Fu Qingbin Liu Haijie Tian Yong Li Yongjun Bao Zhiwei Fang Hanqing Lu SSeg 322 5,108 0 09 Sep 2018
Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks Abhijit Guha Roy Nassir Navab Christian Wachinger SSeg 107 380 0 23 Aug 2018
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 177 4,990 0 30 Jul 2018
CBAM: Convolutional Block Attention Module Sanghyun Woo Jongchan Park Joon-Young Lee In So Kweon 224 16,553 0 17 Jul 2018
BAM: Bottleneck Attention Module Jongchan Park Sanghyun Woo Joon-Young Lee In So Kweon 83 1,042 0 17 Jul 2018
Group Normalization Yuxin Wu Kaiming He 231 3,660 0 22 Mar 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 289 8,906 0 21 Nov 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 424 26,500 0 05 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 704 131,652 0 12 Jun 2017