On the Relationship between Self-Attention and Convolutional Layers

8 November 2019

Papers citing "On the Relationship between Self-Attention and Convolutional Layers"

50 / 269 papers shown

Title
Function Fitting Based on Kolmogorov-Arnold Theorem and Kernel Functions Jianpeng Liu Qizhi Pan 40 0 0 29 Mar 2025
Quantifying Interpretability in CLIP Models with Concept Consistency Avinash Madasu Vasudev Lal Phillip Howard VLM 69 0 0 14 Mar 2025
Vision-LSTM: xLSTM as Generic Vision Backbone Benedikt Alkin M. Beck Korbinian Poppel Sepp Hochreiter Johannes Brandstetter VLM 64 43 0 24 Feb 2025
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis Amir Hosein Fadaei M. Dehaqani 45 0 0 11 Feb 2025
Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi Ella Koresh Ronit D. Gross Yuval Meir Yarden Tzach Tal Halevi Ido Kanter ViT 49 0 0 22 Jan 2025
Approximation Rate of the Transformer Architecture for Sequence Modeling Hao Jiang Qianxiao Li 48 9 0 03 Jan 2025
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers Mingliang Xu Yuyao Zhou Yuxin Zhang Shen Li Yong Li Rongrong Ji Zhanpeng Zeng Rongrong Ji MQ 94 0 0 31 Dec 2024
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 104 15 0 03 Dec 2024
Probe-Me-Not: Protecting Pre-trained Encoders from Malicious Probing Ruyi Ding Tong Zhou Lili Su A. A. Ding Xiaolin Xu Yunsi Fei AAML 69 1 0 19 Nov 2024
GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise Moseli Motsóehli Kyungim Baek 34 1 0 08 Nov 2024
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective Qishuai Wen Chun-Guang Li ViT 37 0 0 05 Nov 2024
Is Smoothness the Key to Robustness? A Comparison of Attention and Convolution Models Using a Novel Metric Baiyuan Chen MLT 28 0 0 23 Oct 2024
Artificial Human Intelligence: The role of Humans in the Development of Next Generation AI Suayb S. Arslan 31 2 0 24 Sep 2024
Quantifying and Enabling the Interpretability of CLIP-like Models Avinash Madasu Yossi Gandelsman Vasudev Lal Phillip Howard VLM 56 2 0 10 Sep 2024
Decompose the model: Mechanistic interpretability in image models with Generalized Integrated Gradients (GIG) Yearim Kim Sangyu Han Sangbum Han Nojun Kwak 55 0 0 03 Sep 2024
Research on Personalized Compression Algorithm for Pre-trained Models Based on Homomorphic Entropy Increase Yicong Li Xing Guo Haohua Du 35 0 0 16 Aug 2024
Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks Jianfei Li Han Feng Ding-Xuan Zhou 32 1 0 10 Aug 2024
3D Geometric Shape Assembly via Efficient Point Cloud Matching Nahyuk Lee Juhong Min Junha Lee Seungwook Kim Kanghee Lee Jaesik Park Minsu Cho 44 4 0 15 Jul 2024
Toto: Time Series Optimized Transformer for Observability Ben Cohen E. Khwaja Kan Wang Charles Masson Elise Ramé Youssef Doubli Othmane Abou-Amal AI4TS 43 3 0 10 Jul 2024
PosMLP-Video: Spatial and Temporal Relative Position Encoding for Efficient Video Recognition Y. Hao Diansong Zhou Zhicai Wang Chong-Wah Ngo Meng Wang ViT 40 4 0 03 Jul 2024
LPViT: Low-Power Semi-structured Pruning for Vision Transformers Kaixin Xu Zhe Wang Chunyun Chen Xue Geng Jie Lin Xulei Yang Min-man Wu Min Wu Xiaoli Li Weisi Lin ViT VLM 51 7 0 02 Jul 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
Grounding Continuous Representations in Geometry: Equivariant Neural Fields David R. Wessels David M. Knigge Samuele Papa Riccardo Valperga Sharvaree P. Vadgama E. Gavves Erik J. Bekkers 55 7 0 09 Jun 2024
Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularity Benjamin Shih Ahmad Peyvan Zhongqiang Zhang George Karniadakis AI4CE 46 11 0 29 May 2024
CHANI: Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration Sophie Jaffard Samuel Vaiter Patricia Reynaud-Bouret 75 0 0 29 May 2024
Automated Deep Learning for Load Forecasting Julie Keisler Sandra Claudel Gilles Cabriel Margaux Brégère AI4TS 40 3 0 14 May 2024
Computer-Aided Diagnosis of Thoracic Diseases in Chest X-rays using hybrid CNN-Transformer Architecture Sonit Singh MedIm ViT 29 1 0 18 Apr 2024
Rethinking Self-training for Semi-supervised Landmark Detection: A Selection-free Approach Haibo Jin Haoxuan Che Hao Chen 51 0 0 06 Apr 2024
Vision Transformers in Domain Adaptation and Generalization: A Study of Robustness Shadi Alijani Jamil Fayyad H. Najjaran OOD 35 1 0 05 Apr 2024
Learning Correlation Structures for Vision Transformers Manjin Kim Paul Hongsuck Seo Cordelia Schmid Minsu Cho ViT 40 7 0 05 Apr 2024
Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images Rita Pucci Vincent J. Kalkman Dan Stowell 23 2 0 04 Apr 2024
Structured Initialization for Attention in Vision Transformers Jianqiao Zheng Xueqian Li Simon Lucey ViT 26 1 0 01 Apr 2024
SpiralMLP: A Lightweight Vision MLP Architecture Haojie Mu Burhan Ul Tayyab Nicholas Chua 43 0 0 31 Mar 2024
A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints Tran Anh Tuan Nguyen Viet Dung Tran Ngoc Thang 39 3 0 04 Feb 2024
Convolutional Initialization for Data-Efficient Vision Transformers Jianqiao Zheng Xueqian Li Simon Lucey 43 2 0 23 Jan 2024
Understanding Video Transformers via Universal Concept Discovery M. Kowal Achal Dave Rares Andrei Ambrus Adrien Gaidon Konstantinos G. Derpanis P. Tokmakov ViT 37 8 0 19 Jan 2024
Integrating Human Vision Perception in Vision Transformers for Classifying Waste Items Akshat Shrivastava Tapan K. Gandhi 27 1 0 19 Dec 2023
Disentangling Linear Mode-Connectivity Gul Sena Altintas Gregor Bachmann Lorenzo Noci Thomas Hofmann 31 6 0 15 Dec 2023
DiT-Head: High-Resolution Talking Head Synthesis using Diffusion Transformers Aaron Mir Eduardo Alonso Esther Mondragón DiffM 38 2 0 11 Dec 2023
Advancing Vision Transformers with Group-Mix Attention Chongjian Ge Xiaohan Ding Zhan Tong Li Yuan Jiangliu Wang Yibing Song Ping Luo 112 16 0 26 Nov 2023
Explainability of Vision Transformers: A Comprehensive Review and New Perspectives Rojina Kashefi Leili Barekatain Mohammad Sabokrou Fatemeh Aghaeipoor ViT 45 9 0 12 Nov 2023
High-Performance Transformers for Table Structure Recognition Need Early Convolutions Sheng-Hsuan Peng Seongmin Lee Xiaojing Wang Rajarajeswari Balasubramaniyan Duen Horng Chau ViT LMTD 24 3 0 09 Nov 2023
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns Deli Yu Teng Xi Jianwei Li Baopu Li Gang Zhang Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang ViT 31 0 0 11 Oct 2023
Multi-spectral Entropy Constrained Neural Compression of Solar Imagery Ali Zafari Atefeh Khoshkhahtinat P. Mehta Nasser M. Nasrabadi Barbara J. Thompson M. Kirk D. D. Silva 24 0 0 19 Sep 2023
Interpret Vision Transformers as ConvNets with Dynamic Convolutions Chong Zhou Chen Change Loy Bo Dai ViT 32 1 0 19 Sep 2023
Toward a Deeper Understanding: RetNet Viewed through Convolution Chenghao Li Chaoning Zhang ViT 37 7 0 11 Sep 2023
A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking Lorenzo Papa Paolo Russo Irene Amerini Luping Zhou 33 43 0 05 Sep 2023
JPEG Quantized Coefficient Recovery via DCT Domain Spatial-Frequential Transformer Ming-Yan Ouyang Zhenzhong Chen MQ 33 1 0 17 Aug 2023
SCSC: Spatial Cross-scale Convolution Module to Strengthen both CNNs and Transformers Xijun Wang Xiaojie Chu Chunrui Han Xiangyu Zhang ViT 23 1 0 14 Aug 2023
Robustifying Point Cloud Networks by Refocusing Meir Yossef Levi Guy Gilboa 3DPC 32 4 0 10 Aug 2023