The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models

1 July 2021

Papers citing "The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models"

50 / 50 papers shown

Title
VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis Xinyuan Yan Xiwei Xuan Jorge Henrique Piazentin Ono Jiajing Guo V. Mohanty Shekar Arvind Kumar Liang Gou Bei Wang Liu Ren 42 0 0 06 May 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
Subgroup Performance Analysis in Hidden Stratifications Alceu Bissoto Trung-Dung Hoang Tim Flühmann Susu Sun Christian F. Baumgartner Lisa M. Koch OOD 60 0 0 13 Mar 2025
Detecting Systematic Weaknesses in Vision Models along Predefined Human-Understandable Dimensions Sujan Sai Gannamaneni Rohil Prakash Rao Michael Mock Maram Akila Stefan Wrobel AAML 192 0 0 17 Feb 2025
Efficient Bias Mitigation Without Privileged Information Mateo Espinosa Zarlenga Swami Sankaranarayanan Jerone T. A. Andrews Z. Shams M. Jamnik Alice Xiang 42 3 0 26 Sep 2024
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing Chenyang Yang Yining Hong Grace A. Lewis Tongshuang Wu Christian Kastner 38 1 0 14 Sep 2024
LADDER: Language Driven Slice Discovery and Error Rectification Shantanu Ghosh Rayan Syed Chenyu Wang Clare B. Poynton Kayhan Batmanghelich 42 0 0 31 Jul 2024
Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods Vincent Olesen Nina Weng Aasa Feragen Eike Petersen 39 0 0 17 Jun 2024
Site-specific Deterministic Temperature and Humidity Forecasts with Explainable and Reliable Machine Learning Mengmeng Han Tennessee Leeuwenburg Brad Murphy 21 1 0 04 Apr 2024
Lifelong Benchmarks: Efficient Model Evaluation in an Era of Rapid Progress Ameya Prabhu Vishaal Udandarao Philip Torr Matthias Bethge Adel Bibi Samuel Albanie 47 3 0 29 Feb 2024
A hierarchical decomposition for explaining ML performance discrepancies Jean Feng Harvineet Singh Fan Xia Adarsh Subbaswamy Alexej Gossmann CML 32 0 0 22 Feb 2024
Learning Exceptional Subgroups by End-to-End Maximizing KL-divergence Sascha Xu Nils Philipp Walter Janis Kalofolias Jilles Vreeken 40 1 0 20 Feb 2024
A Survey on Safe Multi-Modal Learning System Tianyi Zhao Liangliang Zhang Yao Ma Lu Cheng 60 9 0 08 Feb 2024
Clarify: Improving Model Robustness With Natural Language Corrections Yoonho Lee Michelle S. Lam Helena Vasconcelos Michael S. Bernstein Chelsea Finn 35 6 0 06 Feb 2024
AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding Xiwei Xuan Jorge Henrique Piazentin Ono Liang Gou Kwan-Liu Ma Liu Ren 56 1 0 12 Jan 2024
Prompt-driven Latent Domain Generalization for Medical Image Classification Siyuan Yan Chi Liu Zhen Yu Lie Ju Dwarikanath Mahapatra B. Betz‐Stablein Victoria Mar Monika Janda Peter Soyer Zongyuan Ge OOD VLM MedIm 35 7 0 05 Jan 2024
Language-assisted Vision Model Debugger: A Sample-Free Approach to Finding and Fixing Bugs Chaoquan Jiang Jinqiang Wang Rui Hu Jitao Sang 29 0 0 09 Dec 2023
Error Discovery by Clustering Influence Embeddings Fulton Wang Julius Adebayo Sarah Tan Diego Garcia-Olano Narine Kokhlikyan 21 3 0 07 Dec 2023
Describing Differences in Image Sets with Natural Language Lisa Dunlap Yuhui Zhang Xiaohan Wang Ruiqi Zhong Trevor Darrell Jacob Steinhardt Joseph E. Gonzalez Serena Yeung-Levy CoGe VLM 32 30 0 05 Dec 2023
TLDR: Text Based Last-layer Retraining for Debiasing Image Classifiers Juhyeon Park Seokhyeon Jeong Taesup Moon 41 1 0 30 Nov 2023
Probing clustering in neural network representations Thao Nguyen Simon Kornblith 32 1 0 14 Nov 2023
Effective Human-AI Teams via Learned Natural Language Rules and Onboarding Hussein Mozannar Jimin J Lee Dennis L. Wei P. Sattigeri Subhro Das David Sontag 41 11 0 02 Nov 2023
PRIME: Prioritizing Interpretability in Failure Mode Extraction Keivan Rezaei Mehrdad Saberi Mazda Moayeri S. Feizi 25 8 0 29 Sep 2023
VLSlice: Interactive Vision-and-Language Slice Discovery Eric Slyman Minsuk Kahng Stefan Lee VLM 25 9 0 13 Sep 2023
Assessing Systematic Weaknesses of DNNs using Counterfactuals Sujan Sai Gannamaneni Michael Mock Maram Akila AAML 38 1 0 03 Aug 2023
Where Does My Model Underperform? A Human Evaluation of Slice Discovery Algorithms Nari Johnson Ángel Alexander Cabrera Gregory Plumb Ameet Talwalkar 44 13 0 13 Jun 2023
Efficient Failure Pattern Identification of Predictive Algorithms Bao Nguyen Viet Anh Nguyen 30 0 0 01 Jun 2023
Targeted Data Generation: Finding and Fixing Model Weaknesses Zexue He Marco Tulio Ribeiro Fereshte Khani 29 13 0 28 May 2023
Capturing Humans' Mental Models of AI: An Item Response Theory Approach Markelle Kelly Aakriti Kumar Padhraic Smyth M. Steyvers 37 13 0 15 May 2023
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures Eugenia Iofinova Alexandra Peste Dan Alistarh 28 9 0 25 Apr 2023
Angler: Helping Machine Translation Practitioners Prioritize Model Improvements Samantha Robertson Zijie J. Wang Dominik Moritz Mary Beth Kery Fred Hohman 38 15 0 12 Apr 2023
Mitigating Source Bias for Fairer Weak Supervision Changho Shin Sonia Cromp Dyah Adila Frederic Sala 29 2 0 30 Mar 2023
ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysis Yongsu Ahn Y. Lin Panpan Xu Zeng Dai AAML HAI 21 8 0 16 Mar 2023
Identification of Systematic Errors of Image Classifiers on Rare Subgroups J. H. Metzen Robin Hutmacher N. G. Hua Valentyn Boreiko Dan Zhang AAML VLM 61 19 0 09 Mar 2023
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning Ángel Alexander Cabrera Erica Fu Donald Bertucci Kenneth Holstein Ameet Talwalkar Jason I. Hong Adam Perer 25 48 0 09 Feb 2023
Diagnosing and Rectifying Vision Models using Language Yuhui Zhang Jeff Z. HaoChen Shih-Cheng Huang Kuan-Chieh Jackson Wang James Zou Serena Yeung 39 45 0 08 Feb 2023
Discovering and Mitigating Visual Biases through Keyword Explanation Younghyun Kim Sangwoo Mo Minkyu Kim Kyungmin Lee Jaeho Lee Jinwoo Shin 40 33 0 26 Jan 2023
A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others Zhiheng Li Ivan Evtimov Albert Gordo C. Hazirbas Tal Hassner Cristian Canton Ferrer Chenliang Xu Mark Ibrahim 39 71 0 09 Dec 2022
Adaptive Testing of Computer Vision Models Irena Gao Gabriel Ilharco Scott M. Lundberg Marco Tulio Ribeiro VLM 17 42 0 06 Dec 2022
AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization Bhargavi Paranjape Pradeep Dasigi Vivek Srikumar Luke Zettlemoyer Hannaneh Hajishirzi 36 7 0 02 Dec 2022
Identifying Spurious Correlations and Correcting them with an Explanation-based Learning Misgina Tsighe Hagos Kathleen M. Curran Brian Mac Namee 26 10 0 15 Nov 2022
Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing Wenyue Hua Lifeng Jin Linfeng Song Haitao Mi Yongfeng Zhang Dong Yu 32 1 0 08 Nov 2022
"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts Haoran Zhang Harvineet Singh Marzyeh Ghassemi Shalmali Joshi 29 20 0 19 Oct 2022
SEAL : Interactive Tool for Systematic Error Analysis and Labeling Nazneen Rajani Weixin Liang Lingjiao Chen Margaret Mitchell James Zou 48 16 0 11 Oct 2022
Towards a More Rigorous Science of Blindspot Discovery in Image Classification Models Gregory Plumb Nari Johnson Ángel Alexander Cabrera Ameet Talwalkar 40 5 0 08 Jul 2022
Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning Mayee F. Chen Daniel Y. Fu A. Narayan Michael Zhang Zhao Song Kayvon Fatahalian Christopher Ré SSL 32 47 0 15 Apr 2022
Domino: Discovering Systematic Errors with Cross-Modal Embeddings Sabri Eyuboglu M. Varma Khaled Kamal Saab Jean-Benoit Delbrouck Christopher Lee-Messer Jared A. Dunnmon James Zou Christopher Ré 38 143 0 24 Mar 2022
How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task Urja Khurana Eric T. Nalisnick Antske Fokkens MoMe 35 6 0 18 Nov 2021
Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations Abubakar Abid Mert Yuksekgonul James Zou CML 34 64 0 24 Jun 2021
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 296 31,280 0 16 Jan 2013