v1v2 (latest)

CLIP-VIS: Adapting CLIP for Open-Vocabulary Video Instance Segmentation

19 March 2024

Wenqi Zhu

Jiale Cao

Jin Xie

ArXiv (abs)PDF HTML Github (43★)

Papers citing "CLIP-VIS: Adapting CLIP for Open-Vocabulary Video Instance Segmentation"

46 / 46 papers shown

Title
SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation Bin Xie Jiale Cao Jin Xie Fahad Shahbaz Khan Yanwei Pang VLM 95 47 0 27 Nov 2023
Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP Qihang Yu Ju He XueQing Deng Xiaohui Shen Liang-Chieh Chen VLM CLIP 84 150 0 04 Aug 2023
CTVIS: Consistent Training for Online Video Instance Segmentation Kaining Ying Qing Zhong Wei Mao Zhenhua Wang Hao Chen Lin Yuanbo Wu Yifan Liu Chengxiang Fan Yunzhi Zhuge Chunhua Shen 84 40 0 24 Jul 2023
Towards Open-Vocabulary Video Instance Segmentation Haochen Wang Cilin Yan Shuailong Wang Xiaolong Jiang XU Tang Yao Hu Weidi Xie E. Gavves VOS VLM 67 34 0 04 Apr 2023
FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation Jie Qin Jie Wu Pengxiang Yan Ming Li Ren Yuxi ... Yitong Wang Rui Wang Shilei Wen X. Pan Xingang Wang SSeg VLM 80 94 0 30 Mar 2023
DynaMask: Dynamic Mask Selection for Instance Segmentation Ruihuang Li Chenhang He Shuai Li Yabin Zhang Lei Zhang ISeg 48 18 0 14 Mar 2023
Look Before You Match: Instance Understanding Matters in Video Object Segmentation Junke Wang Dongdong Chen Zuxuan Wu Chong Luo Chuanxin Tang Xiyang Dai Yucheng Zhao Yujia Xie Lu Yuan Yu-Gang Jiang VOS 97 41 0 13 Dec 2022
ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation Ziqi Zhou Bowen Zhang Yinjie Lei Lingqiao Liu Yifan Liu VLM 76 175 0 07 Dec 2022
MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors Yuang Zhang Tiancai Wang Xiangyu Zhang VOT 72 138 0 17 Nov 2022
Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection Yanxin Long Jianhua Han Runhu Huang Xu Hang Yi Zhu Chunjing Xu Xiaodan Liang VLM ObjD 74 19 0 02 Nov 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 200 3,493 0 16 Oct 2022
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP Feng Liang Bichen Wu Xiaoliang Dai Kunpeng Li Yinan Zhao Hang Zhang Peizhao Zhang Peter Vajda Diana Marculescu CLIP VLM 105 457 0 09 Oct 2022
BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video A. Athar Jonathon Luiten P. Voigtlaender Tarasha Khurana Achal Dave Bastian Leibe Deva Ramanan VOS VLM 87 60 0 25 Sep 2022
Open-Vocabulary Universal Image Segmentation with MaskCLIP Zheng Ding Jieke Wang Zhuowen Tu CLIP ISeg VLM 91 90 0 18 Aug 2022
MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Training De-An Huang Zhiding Yu Anima Anandkumar VLM 98 81 0 03 Aug 2022
In Defense of Online Models for Video Instance Segmentation Junfeng Wu Qihao Liu Yi Jiang S. Bai Alan Yuille Xiang Bai 76 111 0 21 Jul 2022
VITA: Video Instance Segmentation via Object Token Association Miran Heo Sukjun Hwang Seoung Wug Oh Joon-Young Lee Seon Joo Kim VOS 76 92 0 09 Jun 2022
Temporally Efficient Vision Transformer for Video Instance Segmentation Shusheng Yang Xinggang Wang Yu Li Yuxin Fang Jiemin Fang Wenyu Liu Xun Zhao Ying Shan ViT 57 67 0 18 Apr 2022
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model Yu Du Fangyun Wei Zihe Zhang Miaojing Shi Yue Gao Guoqi Li VPVLM VLM 81 334 0 28 Mar 2022
Efficient Video Instance Segmentation via Tracklet Query and Proposal Jialian Wu Sudhir Yarram Hui Liang Tian Lan Junsong Yuan J. Eledath Gérard Medioni 70 37 0 03 Mar 2022
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 186 5,213 0 10 Jan 2022
Language-driven Semantic Segmentation Boyi Li Kilian Q. Weinberger Serge Belongie V. Koltun René Ranftl VLM 124 625 0 10 Jan 2022
Detecting Twenty-thousand Classes using Image-level Supervision Xingyi Zhou Rohit Girdhar Armand Joulin Phillip Krahenbuhl Ishan Misra CLIP VLM 106 617 0 07 Jan 2022
Decoupling Zero-Shot Semantic Segmentation Jian Ding Nan Xue Guisong Xia Dengxin Dai VLM 106 195 0 15 Dec 2021
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 131 1,067 0 07 Dec 2021
Masked-attention Mask Transformer for Universal Image Segmentation Bowen Cheng Ishan Misra Alex Schwing Alexander Kirillov Rohit Girdhar ISeg 257 2,379 0 02 Dec 2021
FILIP: Fine-grained Interactive Language-Image Pre-Training Lewei Yao Runhu Huang Lu Hou Guansong Lu Minzhe Niu Hang Xu Xiaodan Liang Zhenguo Li Xin Jiang Chunjing Xu VLM CLIP 108 642 0 09 Nov 2021
Crossover Learning for Fast Online Video Instance Segmentation Shusheng Yang Yuxin Fang Xinggang Wang Yu Li Chen Fang Ying Shan Bin Feng Wenyu Liu 94 105 0 13 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 459 3,893 0 11 Feb 2021
Occluded Video Instance Segmentation: A Benchmark Jiyang Qi Yan Gao Yao Hu Xinggang Wang Xiaoyu Liu Xiang Bai Serge Belongie Alan Yuille Philip Torr S. Bai VOS VLM 70 140 0 02 Feb 2021
CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation Yang Fu Linjie Yang Ding Liu Thomas S. Huang Humphrey Shi VOS 68 72 0 07 Dec 2020
End-to-End Video Instance Segmentation with Transformers Yuqing Wang Zhaoliang Xu Xinlong Wang Chunhua Shen Baoshan Cheng Hao Shen Huaxia Xia ViT 82 691 0 30 Nov 2020
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation Jiale Cao Rao Muhammad Anwer Hisham Cholakkal Fahad Shahbaz Khan Yanwei Pang Ling Shao ISeg 56 171 0 29 Jul 2020
TAO: A Large-Scale Benchmark for Tracking Any Object Achal Dave Tarasha Khurana P. Tokmakov Cordelia Schmid Deva Ramanan 73 180 0 20 May 2020
Conditional Convolutions for Instance Segmentation Zhi Tian Chunhua Shen Hao Chen ISeg 246 613 0 12 Mar 2020
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 250 2,488 0 20 Aug 2019
LVIS: A Dataset for Large Vocabulary Instance Segmentation Agrim Gupta Piotr Dollár Ross B. Girshick ISeg VLM 105 1,379 0 08 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 243 3,695 0 06 Aug 2019
Video Instance Segmentation Linjie Yang Yuchen Fan N. Xu VOS ISeg 85 509 0 12 May 2019
YOLACT: Real-time Instance Segmentation Daniel Bolya Chong Zhou Fanyi Xiao Yong Jae Lee SSeg VLM ISeg 131 1,681 0 04 Apr 2019
Deep Affinity Network for Multiple Object Tracking Shijie Sun Naveed Akhtar Huansheng Song Ajmal Mian M. Shah VOT 48 330 0 28 Oct 2018
Decoupled Weight Decay Regularization I. Loshchilov Frank Hutter OffRL 151 2,151 0 14 Nov 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 366 27,244 0 20 Mar 2017
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation Fausto Milletari Nassir Navab Seyed-Ahmad Ahmadi 240 8,716 0 15 Jun 2016
Simple Online and Realtime Tracking Alex Bewley Zongyuan Ge Lionel Ott F. Ramos B. Upcroft VOT 86 3,104 0 02 Feb 2016
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 432 43,814 0 01 May 2014