Visual Prompting via Image Inpainting

Neural Information Processing Systems (NeurIPS), 2022

1 September 2022

ArXiv (abs)PDF HTML HuggingFace (1 upvotes)

Papers citing "Visual Prompting via Image Inpainting"

50 / 180 papers shown

Title
VRP-SAM: SAM with Visual Reference PromptComputer Vision and Pattern Recognition (CVPR), 2024 Yanpeng Sun Jiahui Chen Shan Zhang Xinyu Zhang Qiang Chen Qiang Chen Gang Zhang Errui Ding Jingdong Wang Zechao Li 293 68 0 27 Feb 2024
A Simple Framework Uniting Visual In-context Learning with Masked Image Modeling to Improve Ultrasound Segmentation Yuyue Zhou B. Felfeliyan Shrimanti Ghosh Jessica Knight Fatima Alves-Pereira Christopher Keen Jessica Küpper A. Hareendranathan Jacob L. Jaremko 131 0 0 22 Feb 2024
Tumor segmentation on whole slide images: training or prompting? Huaqian Wu C. B. Martin Kevin Bouaou Cédric Clouchoux VLM MedIm 59 3 0 21 Feb 2024
Data-efficient Large Vision Models through Sequential Autoregression Jianyuan Guo Zhiwei Hao Chengcheng Wang Yehui Tang Han Wu Han Hu Kai Han Chang Xu VLM 156 12 0 07 Feb 2024
Can MLLMs Perform Text-to-Image In-Context Learning? Yuchen Zeng Wonjun Kang Yicong Chen Hyung Il Koo Kangwook Lee MLLM 167 14 0 02 Feb 2024
OMG-Seg: Is One Model Good Enough For All Segmentation? Xiangtai Li Haobo Yuan Wei Li Henghui Ding Size Wu Wenwei Zhang Yining Li Kai Chen Chen Change Loy VLM MLLM ViT 190 99 0 18 Jan 2024
A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting Wouter Van Gansbeke Bert De Brabandere DiffM 196 14 0 18 Jan 2024
Edit One for All: Interactive Batch Image Editing Thao Nguyen Utkarsh Ojha Yuheng Li Haotian Liu Yong Jae Lee DiffM 149 4 0 18 Jan 2024
Low-Resource Vision Challenges for Foundation ModelsComputer Vision and Pattern Recognition (CVPR), 2024 Yunhua Zhang Hazel Doughty Cees G. M. Snoek VLM 192 12 0 09 Jan 2024
Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively Haobo Yuan Xiangtai Li Chong Zhou Yining Li Kai Chen Chen Change Loy VLM 197 79 0 05 Jan 2024
3DAxiesPrompts: Unleashing the 3D Spatial Task Capabilities of GPT-4V Dingning Liu Xiaomeng Dong Renrui Zhang Xu Luo Shiyang Feng Xiaoshui Huang Yongshun Gong Zhihui Wang 128 17 0 15 Dec 2023
Adaptive Human Trajectory Prediction via Latent Corridors Neerja Thakkar K. Mangalam Andrea V. Bajcsy Jitendra Malik 200 7 0 11 Dec 2023
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 169 101 0 11 Dec 2023
Flexible visual prompts for in-context learning in computer vision Thomas Foster Ioana Croitoru Robert Dorfman Christoffer Edlund Thomas Varsavsky Jon Almazán VLM VOS 113 2 0 11 Dec 2023
From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in VideosIEEE Transactions on Affective Computing (TAC), 2023 Yin Chen Jia Li Shiguang Shan Meng Wang Richang Hong 133 55 0 09 Dec 2023
NeRFiller: Completing Scenes via Generative 3D Inpainting Ethan Weber Aleksander Holyñski Varun Jampani Saurabh Saxena Noah Snavely Abhishek Kar Angjoo Kanazawa 190 51 0 07 Dec 2023
Context Diffusion: In-Context Aware Image GenerationEuropean Conference on Computer Vision (ECCV), 2023 Ivona Najdenkoska Animesh Sinha Abhimanyu Dubey Dhruv Mahajan Vignesh Ramanathan Filip Radenovic DiffM 164 14 0 06 Dec 2023
UPOCR: Towards Unified Pixel-Level OCR InterfaceInternational Conference on Machine Learning (ICML), 2023 Dezhi Peng Zhenhua Yang Jiaxin Zhang Chongyu Liu Yongxin Shi Kai Ding Fengjun Guo Lianwen Jin 197 13 0 05 Dec 2023
Towards More Unified In-context Visual UnderstandingComputer Vision and Pattern Recognition (CVPR), 2023 Dianmo Sheng DongDong Chen Zhentao Tan Qiankun Liu Qi Chu Jianmin Bao Tao Gong Bin Liu Shengwei Xu Nenghai Yu MLLM VLM 144 12 0 05 Dec 2023
IMProv: Inpainting-based Multimodal Prompting for Computer Vision Tasks Jiarui Xu Yossi Gandelsman Amir Bar Jianwei Yang Jianfeng Gao Trevor Darrell Xiaolong Wang VLM 80 5 0 04 Dec 2023
Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Models Bingshuai Liu Chenyang Lyu Zijun Min Zhanyu Wang Jinsong Su Longyue Wang LRM 183 8 0 04 Dec 2023
Regressor-Segmenter Mutual Prompt Learning for Crowd CountingComputer Vision and Pattern Recognition (CVPR), 2023 Mingyue Guo Li Yuan Zhaoyi Yan Binghui Chen Yaowei Wang QiXiang Ye 176 14 0 04 Dec 2023
Improving In-Context Learning in Diffusion Models with Visual Context-Modulated Prompts Tianqi Chen Yongfei Liu Zhendong Wang Jianbo Yuan Quanzeng You Hongxia Yang Mingyuan Zhou VLM 115 6 0 03 Dec 2023
Sequential Modeling Enables Scalable Learning for Large Vision ModelsComputer Vision and Pattern Recognition (CVPR), 2023 Yutong Bai Xinyang Geng K. Mangalam Amir Bar Alan Yuille Trevor Darrell Jitendra Malik Alexei A. Efros MLLM VLM 239 212 0 01 Dec 2023
InstructSeq: Unifying Vision Tasks with Instruction-conditioned Multi-modal Sequence Generation Rongyao Fang Shilin Yan Zhaoyang Huang Jingqiu Zhou Hao Tian Jifeng Dai Hongsheng Li MLLM 144 16 0 30 Nov 2023
SEGIC: Unleashing the Emergent Correspondence for In-Context SegmentationEuropean Conference on Computer Vision (ECCV), 2023 Lingchen Meng Shiyi Lan Hengduo Li Jose M. Alvarez Zuxuan Wu Yu-Gang Jiang VLM ISeg MLLM 157 13 0 24 Nov 2023
MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learningComputer Vision and Pattern Recognition (CVPR), 2023 Yixin Liu Chenrui Fan Yutong Dai Xun Chen Pan Zhou Lichao Sun DiffM 277 32 0 22 Nov 2023
Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World ModelsNeural Information Processing Systems (NeurIPS), 2023 Yeongbin Kim Gautam Singh Junyeong Park Çağlar Gülçehre Sungjin Ahn OCL VLM 162 7 0 15 Nov 2023
EviPrompt: A Training-Free Evidential Prompt Generation Method for Segment Anything Model in Medical Images Yinsong Xu Jiaqi Tang Aidong Men Qingchao Chen VLM MedIm 151 8 0 10 Nov 2023
Instruct Me More! Random Prompting for Visual In-Context LearningIEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2023 Jiahao Zhang Bowen Wang Liangzhi Li Yuta Nakashima Hajime Nagahara VLM 115 26 0 07 Nov 2023
ExPT: Synthetic Pretraining for Few-Shot Experimental DesignNeural Information Processing Systems (NeurIPS), 2023 Tung Nguyen Sudhanshu Agrawal Aditya Grover 218 22 0 30 Oct 2023
Quantifying Privacy Risks of Prompts in Visual Prompt LearningUSENIX Security Symposium (USENIX Security), 2023 Yixin Wu Rui Wen Michael Backes Pascal Berrang Mathias Humbert Yun Shen Yang Zhang AAML VPVLM 153 11 0 18 Oct 2023
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V Jianwei Yang Hao Zhang Feng Li Xueyan Zou Chun-yue Li Jianfeng Gao MLLM VLM 303 255 0 17 Oct 2023
Context-Aware Meta-LearningInternational Conference on Learning Representations (ICLR), 2023 Christopher Fifty Dennis Duan Ronald G. Junkins Ehsan Amid Jurij Leskovec Christopher Ré Sebastian Thrun LRM VLM MLLM 188 24 0 17 Oct 2023
Unifying Image Processing as Visual Prompting Question AnsweringInternational Conference on Machine Learning (ICML), 2023 Yihao Liu Xiangyu Chen Xianzheng Ma Xintao Wang Jiantao Zhou Yu Qiao Chao Dong MLLM 134 28 0 16 Oct 2023
SAIR: Learning Semantic-aware Implicit Representation Canyu Zhang Xiaoguang Li Qing Guo Song Wang 112 4 0 13 Oct 2023
AutoVP: An Automated Visual Prompting Framework and BenchmarkInternational Conference on Learning Representations (ICLR), 2023 Hsi-Ai Tsao Lei Hsiung Pin-Yu Chen Sijia Liu Tsung-Yi Ho VLM 180 27 0 12 Oct 2023
InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision GeneralistsInternational Conference on Learning Representations (ICLR), 2023 Yulu Gan Sungwoo Park Alexander Schubert Anthony Philippakis Ahmed Alaa VLM 191 26 0 30 Sep 2023
Visual In-Context Learning for Few-Shot Eczema SegmentationAnnual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2023 Monitirtha Dey S. K. Bhandari Venugopal Vasudevan 63 3 0 28 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient ChannelsInternational Journal of Computer Vision (IJCV), 2023 Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 354 18 0 15 Sep 2023
InstructDiffusion: A Generalist Modeling Interface for Vision TasksComputer Vision and Pattern Recognition (CVPR), 2023 Zigang Geng Binxin Yang Tiankai Hang Chen Li Shuyang Gu ... Jianmin Bao Zheng Zhang Han Hu DongDong Chen Baining Guo DiffM VLM 195 146 0 07 Sep 2023
Knowledge-Aware Prompt Tuning for Generalizable Vision-Language ModelsIEEE International Conference on Computer Vision (ICCV), 2023 Baoshuo Kan Teng Wang Sibo Wei Xiantong Zhen Weili Guan Feng Zheng VPVLM VLM 194 40 0 22 Aug 2023
Improving Adversarial Robustness of Masked Autoencoders via Test-time Frequency-domain PromptingIEEE International Conference on Computer Vision (ICCV), 2023 Qidong Huang Xiaoyi Dong DongDong Chen Yinpeng Chen Lu Yuan Gang Hua Weiming Zhang Neng H. Yu AAML 168 11 0 20 Aug 2023
EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world UnderstandingEuropean Conference on Computer Vision (ECCV), 2023 Jiazhou Zhou Xueye Zheng Yuanhuiyi Lyu Lin Wang VLM 249 22 0 06 Aug 2023
ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image ManipulationNeural Information Processing Systems (NeurIPS), 2023 Yasheng Sun Yifan Yang Houwen Peng Yifei Shen Yuqing Yang Hang-Rui Hu Lili Qiu Hideki Koike DiffM LM&Ro 137 51 0 02 Aug 2023
Visual Instruction Inversion: Image Editing via Visual Prompting Thao Nguyen Yuheng Li Utkarsh Ojha Yong Jae Lee DiffM 83 30 0 26 Jul 2023
ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration Jiaqi Ma Tianheng Cheng Guoli Wang Qian Zhang Xinggang Wang Guang Dai DiffM VLM 136 62 0 23 Jun 2023
Explore In-Context Learning for 3D Point Cloud UnderstandingNeural Information Processing Systems (NeurIPS), 2023 Zhongbin Fang Xiangtai Li Xia Li J. M. Buhmann Chen Change Loy Mengyuan Liu 3DPC 137 34 0 14 Jun 2023
Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model Xinyu Zhang Jiaxian Guo Paul D. Yoo Yutaka Matsuo Yusuke Iwasawa DiffM 162 24 0 13 Jun 2023
Leveraging Large Language Models for Scalable Vector Graphics-Driven Image UnderstandingIEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2023 Mu Cai Zeyi Huang Yuheng Li Utkarsh Ojha Haohan Wang Yong Jae Lee VLM 93 4 0 09 Jun 2023