Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers

9 June 2021

Ishan Misra Florian Metze

Christoph Feichtenhofer

Andrea Vedaldi

João F. Henriques

ArXiv PDF HTML

Papers citing "Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers"

50 / 189 papers shown

Title
Video Action Recognition with Attentive Semantic Units Yifei Chen Dapeng Chen Ruijin Liu Hao Li Wei Peng 21 11 0 17 Mar 2023
EgoViT: Pyramid Video Transformer for Egocentric Action Recognition Chen-Ming Pan Zhiqi Zhang Senem Velipasalar Yi Tian Xu ViT 20 1 0 15 Mar 2023
Reversible Vision Transformers K. Mangalam Haoqi Fan Yanghao Li Chaoxiong Wu Bo Xiong Christoph Feichtenhofer Jitendra Malik ViT 11 45 0 09 Feb 2023
CMAE-V: Contrastive Masked Autoencoders for Video Action Recognition Cheng Lu Xiaojie Jin Zhicheng Huang Qibin Hou Mingg-Ming Cheng Jiashi Feng 37 8 0 15 Jan 2023
EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding Shuhan Tan Tushar Nagarajan Kristen Grauman 26 21 0 05 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 36 7 0 05 Jan 2023
Efficient Movie Scene Detection using State-Space Transformers Md. Mohaiminul Islam Mahmudul Hasan Kishan Athrey Tony Braskich Gedas Bertasius ViT 44 44 0 29 Dec 2022
Audiovisual Masked Autoencoders Mariana-Iuliana Georgescu Eduardo Fonseca Radu Tudor Ionescu Mario Lucic Cordelia Schmid Anurag Arnab SSL 37 43 0 09 Dec 2022
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Lu Yuan Yu-Gang Jiang VGen 32 87 0 08 Dec 2022
PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data Roei Herzig Ofir Abramovich Elad Ben-Avraham Assaf Arbelle Leonid Karlinsky Ariel Shamir Trevor Darrell Amir Globerson 41 16 0 08 Dec 2022
Fine-tuned CLIP Models are Efficient Video Learners H. Rasheed Muhammad Uzair Khattak Muhammad Maaz Salman Khan Fahad Shahbaz Khan CLIP VLM 34 150 0 06 Dec 2022
Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning A. Piergiovanni Weicheng Kuo A. Angelova ViT 36 54 0 06 Dec 2022
Semantic-Aware Local-Global Vision Transformer Jiatong Zhang Zengwei Yao Fanglin Chen Guangming Lu Wenjie Pei ViT 25 0 0 27 Nov 2022
Interaction Region Visual Transformer for Egocentric Action Anticipation Debaditya Roy Ramanathan Rajendiran Basura Fernando 40 15 0 25 Nov 2022
Video Test-Time Adaptation for Action Recognition Wei Lin M. Jehanzeb Mirza Mateusz Koziñski Horst Possegger Hilde Kuehne Horst Bischof TTA 47 31 0 24 Nov 2022
EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens Sun-Kyoo Hwang Jaehong Yoon Youngwan Lee Sung Ju Hwang 31 6 0 19 Nov 2022
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang Limin Wang Yu Qiao ViT 30 107 0 17 Nov 2022
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges Guo Chen Sen Xing Zhe Chen Yi Wang Kunchang Li ... Hongjie Zhang Tong Lu Yali Wang Liming Wang Yu Qiao 41 46 0 17 Nov 2022
AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders W. G. C. Bandara Naman Patel A. Gholami Mehdi Nikkhah M. Agrawal Vishal M. Patel 25 39 0 16 Nov 2022
SCOTCH and SODA: A Transformer Video Shadow Detection Framework Lihao Liu Jean Prost Lei Zhu Nicolas Papadakis Pietro Lio Carola-Bibiane Schönlieb Angelica I Aviles-Rivero 21 23 0 13 Nov 2022
PatchBlender: A Motion Prior for Video Transformers Gabriele Prato Yale Song Janarthanan Rajendran R. Devon Hjelm Neel Joshi Sarath Chandar ViT 27 0 0 11 Nov 2022
Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networks M. Kowal Mennatullah Siam Md. Amirul Islam Neil D. B. Bruce Richard P. Wildes Konstantinos G. Derpanis FAtt 22 4 0 03 Nov 2022
Temporal Action Segmentation: An Analysis of Modern Techniques Guodong Ding Fadime Sener Angela Yao 47 74 0 19 Oct 2022
Linear Video Transformer with Feature Fixation Kaiyue Lu Zexia Liu Jianyuan Wang Weixuan Sun Zhen Qin ... Xuyang Shen Huizhong Deng Xiaodong Han Yuchao Dai Yiran Zhong 30 4 0 15 Oct 2022
Masked Motion Encoding for Self-Supervised Video Representation Learning Xinyu Sun Peihao Chen Liang-Chieh Chen Chan Li Thomas H. Li Mingkui Tan Chuang Gan 27 29 0 12 Oct 2022
It Takes Two: Masked Appearance-Motion Modeling for Self-supervised Video Transformer Pre-training Yuxin Song Min Yang Wenhao Wu Dongliang He Fu Li Jingdong Wang ViT 97 8 0 11 Oct 2022
Turbo Training with Token Dropout Tengda Han Weidi Xie Andrew Zisserman ViT 34 10 0 10 Oct 2022
Self-supervised Video Representation Learning with Motion-Aware Masked Autoencoders Haosen Yang Deng Huang Bin Wen Jiannan Wu H. Yao Yi-Xin Jiang Xiatian Zhu Zehuan Yuan 43 19 0 09 Oct 2022
Alignment-guided Temporal Attention for Video Action Recognition Yizhou Zhao Zhenyang Li Xun Guo Yan Lu 20 14 0 30 Sep 2022
Rethinking Resolution in the Context of Efficient Video Recognition Chuofan Ma Qiushan Guo Yi-Xin Jiang Zehuan Yuan Ping Luo Xiaojuan Qi 68 12 0 26 Sep 2022
Video Mobile-Former: Video Recognition with Efficient Global Spatial-temporal Modeling Rui Wang Zuxuan Wu Dongdong Chen Yinpeng Chen Xiyang Dai Mengchen Liu Luowei Zhou Lu Yuan Yu-Gang Jiang ViT 43 4 0 25 Aug 2022
Efficient Attention-free Video Shift Transformers Adrian Bulat Brais Martínez Georgios Tzimiropoulos ViT 29 1 0 23 Aug 2022
Frozen CLIP Models are Efficient Video Learners Ziyi Lin Shijie Geng Renrui Zhang Peng Gao Gerard de Melo Xiaogang Wang Jifeng Dai Yu Qiao Hongsheng Li CLIP VLM 16 200 0 06 Aug 2022
Expanding Language-Image Pretrained Models for General Video Recognition Bolin Ni Houwen Peng Minghao Chen Songyang Zhang Gaofeng Meng Jianlong Fu Shiming Xiang Haibin Ling VLM CLIP ViT 40 313 0 04 Aug 2022
Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition Wangmeng Xiang Chong Li Biao Wang Xihan Wei Xiangpei Hua Lei Zhang ViT 30 27 0 27 Jul 2022
MAR: Masked Autoencoders for Efficient Action Recognition Zhiwu Qing Shiwei Zhang Ziyuan Huang Xiang Wang Yuehuang Wang Yiliang Lv Changxin Gao Nong Sang 32 42 0 24 Jul 2022
Is an Object-Centric Video Representation Beneficial for Transfer? Chuhan Zhang Ankush Gupta Andrew Zisserman ViT 37 27 0 20 Jul 2022
Time Is MattEr: Temporal Self-supervision for Video Transformers Sukmin Yun Jaehyung Kim Dongyoon Han Hwanjun Song Jung-Woo Ha Jinwoo Shin ViT 19 12 0 19 Jul 2022
TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation Chengxu Liu Huan Yang Jianlong Fu Xueming Qian ViT 31 15 0 19 Jul 2022
XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model Ho Kei Cheng A. Schwing VLM VOS 24 388 0 14 Jul 2022
Earthformer: Exploring Space-Time Transformers for Earth System Forecasting Zhihan Gao Xingjian Shi Hao Wang Yi Zhu Yuyang Wang Mu Li Dit-Yan Yeung AI4TS 42 150 0 12 Jul 2022
1st Place Solution to the EPIC-Kitchens Action Anticipation Challenge 2022 Zeyu Jiang Changxing Ding EgoV 16 1 0 10 Jul 2022
Distance Matters in Human-Object Interaction Detection Guangzhi Wang Yangyang Guo Yongkang Wong Mohan S. Kankanhalli 24 13 0 05 Jul 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 27 191 0 27 Jun 2022
One-stage Action Detection Transformer Lijun Li Lian Zhuo Bangyin Zhang ViT 32 0 0 21 Jun 2022
Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens Elad Ben-Avraham Roei Herzig K. Mangalam Amir Bar Anna Rohrbach Leonid Karlinsky Trevor Darrell Amir Globerson 19 0 0 13 Jun 2022
MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing Zhaofan Qiu Ting Yao Chong-Wah Ngo Tao Mei ViT 37 15 0 13 Jun 2022
A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Information M. Kowal Mennatullah Siam Md. Amirul Islam Neil D. B. Bruce Richard P. Wildes Konstantinos G. Derpanis 23 25 0 06 Jun 2022
A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications Fei Wu Qingzhong Wang Jian Bian Haoyi Xiong Ning Ding Feixiang Lu Junqing Cheng Dejing Dou AI4TS 28 52 0 02 Jun 2022
Future Transformer for Long-term Action Anticipation Dayoung Gong Joonseok Lee Manjin Kim S. Ha Minsu Cho AI4TS 16 61 0 27 May 2022