Debiasing Vision-Language Models via Biased Prompts

31 January 2023

Ching-Yao Chuang

Varun Jampani

Yuanzhen Li

Antonio Torralba

Stefanie Jegelka

VLM

ArXiv PDF HTML

Papers citing "Debiasing Vision-Language Models via Biased Prompts"

27 / 27 papers shown

Title
Is Free Self-Alignment Possible? Dyah Adila Changho Shin Yijing Zhang Frederic Sala MoMe 141 2 0 24 Feb 2025
Refining Skewed Perceptions in Vision-Language Models through Visual Representations Haocheng Dai Sarang Joshi VLM 102 0 0 03 Jan 2025
Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning Zhifang Zhang Shuo He Bingquan Shen Lei Feng Lei Feng AAML 103 0 0 29 Dec 2024
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model Huan Ma Yan Zhu Changqing Zhang Peilin Zhao Baoyuan Wu Long-Kai Huang Qinghua Hu Bing Wu VLM 86 2 0 01 Mar 2024
Balancing Act: Distribution-Guided Debiasing in Diffusion Models Rishubh Parihar Abhijnya Bhat Abhipsa Basu Saswat Mallick Jogendra Nath Kundu R. V. Babu 92 18 0 28 Feb 2024
Fine-tuning language models to find agreement among humans with diverse preferences Michiel A. Bakker Martin Chadwick Hannah R. Sheahan Michael Henry Tessler Lucy Campbell-Gillingham ... Nat McAleese Amelia Glaese John Aslanides M. Botvinick Christopher Summerfield ALM 100 230 0 28 Nov 2022
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 81 127 0 20 Oct 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 348 6,830 0 13 Apr 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 76 333 0 06 Apr 2022
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning Hugo Elias Berg S. Hall Yash Bhalgat Wonsuk Yang Hannah Rose Kirk Aleksandar Shtedritski Max Bain VLM 67 101 0 22 Mar 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar Aditi Raghunathan Robbie Jones Tengyu Ma Percy Liang OODD 109 669 0 21 Feb 2022
Ethical and social risks of harm from Language Models Laura Weidinger John F. J. Mellor Maribeth Rauh Conor Griffin J. Uesato ... Lisa Anne Hendricks William S. Isaac Sean Legassick G. Irving Iason Gabriel PILM 95 1,025 0 08 Dec 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 248 1,035 0 09 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 117 721 0 04 Sep 2021
Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications Sandhini Agarwal Gretchen Krueger Jack Clark Alec Radford Jong Wook Kim Miles Brundage 52 142 0 05 Aug 2021
Measuring Fairness in Generative Models Christopher T. H. Teo Ngai-Man Cheung SyDa 34 26 0 16 Jul 2021
Fair Attribute Classification through Latent Space De-biasing V. V. Ramaswamy Sunnie S. Y. Kim Olga Russakovsky 54 163 0 02 Dec 2020
Towards Debiasing Sentence Representations Paul Pu Liang Irene Li Emily Zheng Y. Lim Ruslan Salakhutdinov Louis-Philippe Morency 70 238 0 16 Jul 2020
Language (Technology) is Power: A Critical Survey of "Bias" in NLP Su Lin Blodgett Solon Barocas Hal Daumé Hanna M. Wallach 152 1,234 0 28 May 2020
Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning Mei Wang Weihong Deng CVBM 83 231 0 25 Nov 2019
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization Shiori Sagawa Pang Wei Koh Tatsunori B. Hashimoto Percy Liang OOD 85 1,236 0 20 Nov 2019
Fair Generative Modeling via Weak Supervision Kristy Choi Aditya Grover Trisha Singh Rui Shu Stefano Ermon 88 134 0 26 Oct 2019
Mitigating Gender Bias in Natural Language Processing: Literature Review Tony Sun Andrew Gaut Shirlyn Tang Yuxin Huang Mai Elsherief Jieyu Zhao Diba Mirza E. Belding-Royer Kai-Wei Chang William Yang Wang AI4CE 105 555 0 21 Jun 2019
Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search S. Geyik Stuart Ambler K. Kenthapadi 90 381 0 30 Apr 2019
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings Mohsan S. Alvi Andrew Zisserman C. Nellåker FaML 106 248 0 06 Sep 2018
Learning Adversarially Fair and Transferable Representations David Madras Elliot Creager T. Pitassi R. Zemel FaML 366 681 0 17 Feb 2018
Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes Nikhil Garg L. Schiebinger Dan Jurafsky James Zou AI4TS 69 959 0 22 Nov 2017