Training data-efficient image transformers & distillation through attention

23 December 2020

Alexandre Sablayrolles

Hervé Jégou

ViT

ArXiv PDF HTML

Papers citing "Training data-efficient image transformers & distillation through attention"

50 / 1,132 papers shown

Title
Knowledge-Informed Deep Learning for Irrigation Type Mapping from Remote Sensing Oishee Bintey Hoque Nibir Chandra Mandal Abhijin Adiga S. Swarup S. Nouwakpo Amanda Wilson M. Marathe 31 0 0 13 May 2025
A Reproduction Study: The Kernel PCA Interpretation of Self-Attention Fails Under Scrutiny Karahan Sarıtaş Çağatay Yıldız 31 0 0 12 May 2025
L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformers S. Casarin Sergio Escalera O. Lanz 32 0 0 12 May 2025
Brain Hematoma Marker Recognition Using Multitask Learning: SwinTransformer and Swin-Unet Kodai Hirata Tsuyoshi Okita ViT 43 0 0 09 May 2025
Achieving 3D Attention via Triplet Squeeze and Excitation Block Maan Alhazmi Abdulrahman Altahhan 30 0 0 09 May 2025
From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection Moritz Vandenhirtz Julia E. Vogt 38 0 0 09 May 2025
Pro2SAM: Mask Prompt to SAM with Grid Points for Weakly Supervised Object Localization Xi Yang Songsong Duan Nannan Wang Xinbo Gao WSOL 78 0 0 08 May 2025
Balancing Accuracy, Calibration, and Efficiency in Active Learning with Vision Transformers Under Label Noise Moseli Motsóehli Hope Mogale Kyungim Baek 38 0 0 07 May 2025
SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer Young-Hu Park R.-H. Park Hyung-Min Park 49 0 0 07 May 2025
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs Lukas Meiner Jens Mehnert A. P. Condurache MQ 42 0 0 06 May 2025
Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction Changjun Li Runqing Jiang Zhuo Song Pengpeng Yu Ye Zhang Yulan Guo MQ 56 0 0 01 May 2025
Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems Alireza Furutanpey Carmen Walser Philipp Raith P. Frangoudis Schahram Dustdar GNN NAI 85 0 0 28 Apr 2025
GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretability Sehyeong Jo Gangjae Jang Haesol Park 32 0 0 28 Apr 2025
Enhancing Surgical Documentation through Multimodal Visual-Temporal Transformers and Generative AI Hugo Georgenthum Cristian Cosentino Fabrizio Marozzo Pietro Liò MedIm 144 0 0 28 Apr 2025
MERA: Multimodal and Multiscale Self-Explanatory Model with Considerably Reduced Annotation for Lung Nodule Diagnosis Jiahao Lu Chong Yin Silvia Ingala Kenny Erleben M. Nielsen S. Darkner 51 0 0 27 Apr 2025
WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada Braeden Sherritt Isar Nejadgholi Marzieh Amini VLM 44 0 0 17 Apr 2025
How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions Aditya Prakash Benjamin Lundell Dmitry Andreychuk David Forsyth Saurabh Gupta H. Sawhney 36 0 0 16 Apr 2025
Search is All You Need for Few-shot Anomaly Detection Qishan Wang Jia Guo Shuyong Gao H. Wang Li Xiong J. Hu Hanqi Guo Wenqiang Zhang 53 0 0 16 Apr 2025
Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking You Wu Xucheng Wang Xiangyang Yang Mengyuan Liu Dan Zeng Hengzhou Ye Shuiwang Li 34 0 0 12 Apr 2025
Evolved Hierarchical Masking for Self-Supervised Learning Zhanzhou Feng Shiliang Zhang 47 0 0 12 Apr 2025
Low-Bit Integerization of Vision Transformers using Operand Reodering for Efficient Hardware Ching-Yi Lin Sahil Shah MQ 64 0 0 11 Apr 2025
Heart Failure Prediction using Modal Decomposition and Masked Autoencoders for Scarce Echocardiography Databases Andrés Bell-Navas M. Villalba-Orero Enrique Lara Pezzi J. Garicano-Mena S. L. Clainche 53 0 0 10 Apr 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 51 0 0 03 Apr 2025
HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning Hao Wang Shuo Zhang Biao Leng ViT 82 0 0 03 Apr 2025
Mamba-3D as Masked Autoencoders for Accurate and Data-Efficient Analysis of Medical Ultrasound Videos Jiaheng Zhou Yanfeng Zhou Wei Fang Yuxing Tang Le Lu Ge Yang Mamba 199 0 0 26 Mar 2025
Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding Jinlong Li Cristiano Saltori Fabio Poiesi N. Sebe 162 0 0 20 Mar 2025
8-Calves Image dataset Xuyang Fang S. Hannuna Neill D. F. Campbell 118 0 0 17 Mar 2025
Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimization Y. Zhang Yingzhe Xu Junyu Shi L. Zhang Shengshan Hu Minghui Li Yanjun Zhang AAML 51 1 0 17 Mar 2025
Self-Supervised Pretraining for Fine-Grained Plankton Recognition Joona Kareinen T. Eerola K. Kraft L. Lensu S. Suikkanen H. Kalviainen SSL 165 0 0 14 Mar 2025
Accelerate 3D Object Detection Models via Zero-Shot Attention Key Pruning Lizhen Xu Xiuxiu Bai Xiaojun Jia Jianwu Fang Shanmin Pang 61 0 0 13 Mar 2025
ReSi: A Comprehensive Benchmark for Representational Similarity Measures Max Klabunde Tassilo Wald Tobias Schumacher Klaus H. Maier-Hein Markus Strohmaier Adriana Iamnitchi AI4TS VLM 76 5 0 13 Mar 2025
Breaking the Limits of Quantization-Aware Defenses: QADT-R for Robustness Against Patch-Based Adversarial Attacks in QNNs Amira Guesmi B. Ouni Muhammad Shafique MQ AAML 36 0 0 10 Mar 2025
Spectral State Space Model for Rotation-Invariant Visual Representation Learning Sahar Dastani Ali Bahri Moslem Yazdanpanah Mehrdad Noori David Osowiechi ... Farzad Beizaee Milad Cheraghalikhani A. Mondal H. Lombaert Christian Desrosiers 55 0 0 09 Mar 2025
Nexar Dashcam Collision Prediction Dataset and Challenge Daniel C. Moura Shizhan Zhu Orly Zvitia 66 0 0 05 Mar 2025
Transformer Meets Twicing: Harnessing Unattended Residual Information Laziz U. Abdullaev Tan M. Nguyen 41 2 0 02 Mar 2025
Split Adaptation for Pre-trained Vision Transformers Lixu Wang Bingqi Shang Y. Li Payal Mohapatra Wei Dong Xiao-Xu Wang Qi Zhu ViT 43 0 0 01 Mar 2025
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 76 0 0 28 Feb 2025
Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer Yufei Guo Xiaode Liu Y. Chen Weihang Peng Yuhan Zhang Zhe Ma MQ 43 0 0 28 Feb 2025
OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels Meng Lou Yizhou Yu 115 1 0 27 Feb 2025
Examining the Threat Landscape: Foundation Models and Model Stealing Ankita Raj Deepankar Varma Chetan Arora AAML 70 1 0 25 Feb 2025
Vision-LSTM: xLSTM as Generic Vision Backbone Benedikt Alkin M. Beck Korbinian Poppel Sepp Hochreiter Johannes Brandstetter VLM 61 43 0 24 Feb 2025
Graph Perceiver IO: A General Architecture for Graph Structured Data Seyun Bae Hoyoon Byun Changdae Oh Yoon-Sik Cho Kyungwoo Song GNN 98 2 0 24 Feb 2025
Disentangling Visual Transformers: Patch-level Interpretability for Image Classification Guillaume Jeanneret Loïc Simon F. Jurie ViT 49 0 0 24 Feb 2025
EvoP: Robust LLM Inference via Evolutionary Pruning Shangyu Wu Hongchao Du Ying Xiong Shuai Chen Tei-Wei Kuo Nan Guan Chun Jason Xue 34 1 0 19 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 54 0 0 17 Feb 2025
Simplifying DINO via Coding Rate Regularization Ziyang Wu Jingyuan Zhang Druv Pai X. Wang Chandan Singh Jianwei Yang Jianfeng Gao Yi-An Ma 162 1 0 17 Feb 2025
ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Models Ehsan Zeraatkar Salah Faroughi Jelena Tešić 111 0 0 10 Feb 2025
Kolmogorov-Arnold Fourier Networks Jusheng Zhang Yijia Fan Kaitong Cai Keze Wang 65 0 0 09 Feb 2025
Exploring Visual Embedding Spaces Induced by Vision Transformers for Online Auto Parts Marketplaces Cameron Armijo Pablo Rivas 41 0 0 09 Feb 2025
All-in-One Image Compression and Restoration Huimin Zeng Jiacheng Li Ziqiang Zheng Zhiwei Xiong 88 1 0 05 Feb 2025