TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding

29 October 2023

Shicheng Li

Papers citing "TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding"

31 / 31 papers shown

Title
TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos Linli Yao Y. Li Y. X. Wei Lei Li Shuhuai Ren ... Sida Li Lingpeng Kong Qi Liu Wenjie Qu Xu Sun 41 1 0 24 Apr 2025
REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding Sakib Reza Xiyun Song Heather Yu Zongfang Lin Mohsen Moghaddam Mario Sznaier 29 0 0 07 Apr 2025
DynamicVis: An Efficient and General Visual Foundation Model for Remote Sensing Image Understanding Keyan Chen Chenyang Liu Bowen Chen Wenyuan Li Zhengxia Zou Zhenwei Shi 50 2 0 20 Mar 2025
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding Shehreen Azad Vibhav Vineet Y. S. Rawat VLM 139 1 0 11 Mar 2025
ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understanding Xiao Wang Qingyi Si Jianlong Wu Shiyu Zhu Zheng Lin Liqiang Nie VLM 82 6 0 29 Dec 2024
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
Video Token Merging for Long-form Video Understanding Seon-Ho Lee Jue Wang Zhikang Zhang D. Fan Xinyu Li 48 5 0 31 Oct 2024
xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs Michael S Ryoo Honglu Zhou Shrikant B. Kendre Can Qin Le Xue Manli Shu Silvio Savarese Ran Xu Caiming Xiong Juan Carlos Niebles VGen 43 13 0 21 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 84 26 0 04 Oct 2024
TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval Leqi Shen Tianxiang Hao Tao He Sicheng Zhao Pengzhang Liu Yongjun Bao Guiguang Ding Guiguang Ding 135 7 0 02 Sep 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 46 5 0 31 Jul 2024
Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment Hao Fei Shengqiong Wu Meishan Zhang M. Zhang Tat-Seng Chua Shuicheng Yan AI4TS 47 40 0 27 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 50 302 0 31 May 2024
DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models Linli Yao Lei Li Shuhuai Ren Lean Wang Yuanxin Liu Xu Sun Lu Hou 35 28 0 31 May 2024
Leveraging Temporal Contextualization for Video Action Recognition Minji Kim Dongyoon Han Taekyung Kim Bohyung Han 51 2 0 15 Apr 2024
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding Bo He Hengduo Li Young Kyun Jang Menglin Jia Xuefei Cao Ashish Shah Abhinav Shrivastava Ser-Nam Lim MLLM 83 88 0 08 Apr 2024
TempCompass: Do Video LLMs Really Understand Videos? Yuanxin Liu Shicheng Li Yi Liu Yuxiang Wang Shuhuai Ren Lei Li Sishuo Chen Xu Sun Lu Hou VLM 41 98 0 01 Mar 2024
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding Shuhuai Ren Linli Yao Shicheng Li Xu Sun Lu Hou VLM MLLM 25 174 0 04 Dec 2023
A Video is Worth 10,000 Words: Training and Benchmarking with Diverse Captions for Better Long Video Retrieval M. Gwilliam Michael Cogswell Meng Ye Karan Sikka Abhinav Shrivastava Ajay Divakaran 3DV 15 1 1 30 Nov 2023
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models Shicheng Li Lei Li Shuhuai Ren Yuanxin Liu Yi Liu Rundong Gao Xu Sun Lu Hou 36 29 0 29 Nov 2023
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding Peng Jin Ryuichi Takanobu Caiwan Zhang Xiaochun Cao Li-ming Yuan MLLM 36 223 0 14 Nov 2023
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training Qinghao Ye Guohai Xu Ming Yan Haiyang Xu Qi Qian Ji Zhang Fei Huang VLM AI4TS 173 69 0 30 Dec 2022
A CLIP-Hitchhiker's Guide to Long Video Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman CLIP 129 62 0 17 May 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 192 499 0 22 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 308 7,443 0 11 Nov 2021
Token Pooling in Vision Transformers D. Marin Jen-Hao Rick Chang Anurag Ranjan Anish K. Prabhu Mohammad Rastegari Oncel Tuzel ViT 76 66 0 08 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 558 0 28 Sep 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 320 780 0 18 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 293 1,084 0 17 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,982 0 09 Feb 2021