Robustness Analysis of Video-Language Models Against Visual and Language Perturbations

5 July 2022

Madeline Chantry Schiappa

Papers citing "Robustness Analysis of Video-Language Models Against Visual and Language Perturbations"

50 / 52 papers shown

Title
Robust Fairness Vision-Language Learning for Medical Image Analysis Sparsh Bansal Mingyang Wu Xin Wang S. Hu VLM 75 0 0 06 May 2025
Defending Multimodal Fusion Models against Single-Source Adversaries Karren D. Yang Wan-Yi Lin M. Barman Filipe Condessa Zico Kolter AAML 21 31 0 25 Jun 2022
How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness? Xinhsuai Dong Anh Tuan Luu Min Lin Shuicheng Yan Hanwang Zhang SILM AAML 48 57 0 22 Dec 2021
Measure and Improve Robustness in NLP Models: A Survey Xuezhi Wang Haohan Wang Diyi Yang 204 134 0 15 Dec 2021
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation Kaustubh D. Dhole Varun Gangal Sebastian Gehrmann Aadesh Gupta Zhenhao Li ... Tianbao Xie Usama Yaseen Michael A. Yee Jing Zhang Yue Zhang 182 86 0 06 Dec 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 291 567 0 28 Sep 2021
Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks F. Nesti Giulio Rossolini Saasha Nair Alessandro Biondi Giorgio Buttazzo AAML 52 75 0 13 Aug 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 49 162 0 15 Jul 2021
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization Jiaao Chen Dinghan Shen Weizhu Chen Diyi Yang BDL 41 47 0 31 May 2021
VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Prahal Arora Masoumeh Aminzadeh Christoph Feichtenhofer Florian Metze Luke Zettlemoyer 41 132 0 20 May 2021
A Survey of Data Augmentation Approaches for NLP Steven Y. Feng Varun Gangal Jason W. Wei Sarath Chandar Soroush Vosoughi Teruko Mitamura Eduard H. Hovy AIMat 79 813 0 07 May 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 274 581 0 22 Apr 2021
Can audio-visual integration strengthen robustness under multimodal attacks? Yapeng Tian Chenliang Xu AAML 73 37 0 05 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 112 1,154 0 01 Apr 2021
Understanding Robustness of Transformers for Image Classification Srinadh Bhojanapalli Ayan Chakrabarti Daniel Glasner Daliang Li Thomas Unterthiner Andreas Veit ViT 47 382 0 26 Mar 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 327 2,016 0 09 Feb 2021
Semantics Altering Modifications for Evaluating Comprehension in Machine Reading Viktor Schlegel Goran Nenadic Riza Batista-Navarro 37 18 0 07 Dec 2020
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 66 171 0 01 Nov 2020
Learning to Recognize Dialect Features Dorottya Demszky D. Sharma J. Clark Vinodkumar Prabhakaran Jacob Eisenstein 150 38 0 23 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 371 40,217 0 22 Oct 2020
Support-set bottlenecks for video-text representation learning Mandela Patrick Po-Yao (Bernie) Huang Yuki M. Asano Florian Metze Alexander G. Hauptmann João Henriques Andrea Vedaldi 66 246 0 06 Oct 2020
Learning to Discretely Compose Reasoning Module Networks for Video Captioning Ganchao Tan Daqing Liu Meng Wang Zhengjun Zha LRM 65 73 0 17 Jul 2020
Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori Achal Dave Vaishaal Shankar Nicholas Carlini Benjamin Recht Ludwig Schmidt OOD 91 541 0 01 Jul 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo D. Song Jacob Steinhardt Justin Gilmer OOD 254 1,715 0 29 Jun 2020
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 47 142 0 16 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 498 41,106 0 28 May 2020
The Effect of Natural Distribution Shift on Question Answering Models John Miller K. Krauth Benjamin Recht Ludwig Schmidt OOD 67 143 0 29 Apr 2020
MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification Jiaao Chen Zichao Yang Diyi Yang VLM 55 358 0 25 Apr 2020
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation Huaishao Luo Lei Ji Botian Shi Haoyang Huang Nan Duan Tianrui Li Jason Li Xilin Chen Ming Zhou VLM 76 442 0 15 Feb 2020
A simple way to make neural networks robust against diverse image corruptions E. Rusak Lukas Schott Roland S. Zimmermann Julian Bitterwolf Oliver Bringmann Matthias Bethge Wieland Brendel 37 64 0 16 Jan 2020
End-to-End Learning of Visual Representations from Uncurated Instructional Videos Antoine Miech Jean-Baptiste Alayrac Lucas Smaira Ivan Laptev Josef Sivic Andrew Zisserman VGen SSL 100 710 0 13 Dec 2019
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty Dan Hendrycks Norman Mu E. D. Cubuk Barret Zoph Justin Gilmer Balaji Lakshminarayanan OOD UQCV 88 1,293 0 05 Dec 2019
Natural Adversarial Examples Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt D. Song OODD 169 1,454 0 16 Jul 2019
A Fourier Perspective on Model Robustness in Computer Vision Dong Yin Raphael Gontijo-Lopes Jonathon Shlens E. D. Cubuk Justin Gilmer OOD 59 494 0 21 Jun 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 91 1,192 0 07 Jun 2019
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation Raphael Gontijo-Lopes Dong Yin Ben Poole Justin Gilmer E. D. Cubuk AAML 110 205 0 06 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 578 4,735 0 13 May 2019
A deep learning framework for quality assessment and restoration in video endoscopy Sharib Ali Felix Y Zhou A. Bailey B. Braden J. East Xin Lu J. Rittscher MedIm 41 116 0 15 Apr 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 105 3,399 0 28 Mar 2019
Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning Xiaobai Ma Katherine Driggs-Campbell Mykel J. Kochenderfer AAML 39 48 0 08 Mar 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 75 1,693 0 13 Feb 2019
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting Maria De-Arteaga Alexey Romanov Hanna M. Wallach J. Chayes C. Borgs Alexandra Chouldechova S. Geyik K. Kenthapadi Adam Tauman Kalai 116 449 0 27 Jan 2019
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 82 2,647 0 29 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 951 93,936 0 11 Oct 2018
Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate Marcelo O. R. Prates Pedro H. C. Avelar Luís C. Lamb 41 345 0 06 Sep 2018
A Joint Sequence Fusion Model for Video Question Answering and Retrieval Youngjae Yu Jongseok Kim Gunhee Kim 63 343 0 07 Aug 2018
Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models D. Su Huan Zhang Hongge Chen Jinfeng Yi Pin-Yu Chen Yupeng Gao VLM 87 390 0 05 Aug 2018
On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks Egor Lakomkin M. Zamani C. Weber S. Magg S. Wermter 39 53 0 06 Apr 2018
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification Saining Xie Chen Sun Jonathan Huang Zhuowen Tu Kevin Patrick Murphy 3DH 133 1,317 0 13 Dec 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 227 11,962 0 19 Jun 2017