ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset

17 August 2022

Qun Liu

Huamin Qu

ArXiv (abs)PDF HTML

Papers citing "ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset"

41 / 41 papers shown

Title
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation Alisa Liu Swabha Swayamdipta Noah A. Smith Yejin Choi 165 221 0 16 Jan 2022
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework Mengjie Zhao Fei Mi Yasheng Wang Minglei Li Xin Jiang Qun Liu Hinrich Schütze RALM 85 11 0 14 Dec 2021
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models Wei Ping Chejian Xu Shuohang Wang Zhe Gan Yu Cheng Jianfeng Gao Ahmed Hassan Awadallah Yangqiu Song VLM ELM AAML 69 225 0 04 Nov 2021
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension Anna Rogers Matt Gardner Isabelle Augenstein 124 167 0 27 Jul 2021
M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis Xingbo Wang Jianben He Zhihua Jin Muqiao Yang Yong Wang Huamin Qu 102 80 0 17 Jul 2021
Why Machine Reading Comprehension Models Learn Shortcuts? Yuxuan Lai Chen Zhang Yansong Feng Quzhe Huang Dongyan Zhao SyDa 67 53 0 02 Jun 2021
Generating Datasets with Pretrained Language Models Timo Schick Hinrich Schütze 156 235 0 15 Apr 2021
SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoning Roshanak Mirzaee Hossein Rajaby Faghihi Qiang Ning Parisa Kordjmashidi 51 83 0 12 Apr 2021
Dynabench: Rethinking Benchmarking in NLP Douwe Kiela Max Bartolo Yixin Nie Divyansh Kaushik Atticus Geiger ... Pontus Stenetorp Robin Jia Joey Tianyi Zhou Christopher Potts Adina Williams 208 410 0 07 Apr 2021
What Will it Take to Fix Benchmarking in Natural Language Understanding? Samuel R. Bowman George E. Dahl ELM ALM 55 164 0 05 Apr 2021
Data and its (dis)contents: A survey of dataset development and use in machine learning research Amandalynne Paullada Inioluwa Deborah Raji Emily M. Bender Emily L. Denton A. Hanna 124 525 0 09 Dec 2020
GNNLens: A Visual Analytics Approach for Prediction Error Diagnosis of Graph Neural Networks Zhihua Jin Yong Wang Qianwen Wang Yao Ming Tengfei Ma Huamin Qu HAI 68 34 0 22 Nov 2020
OpenAttack: An Open-source Textual Adversarial Attack Toolkit Guoyang Zeng Fanchao Qi Qianrui Zhou Ting Zhang Zixian Ma Bairu Hou Yuan Zang Zhiyuan Liu Maosong Sun AAML 174 124 0 19 Sep 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 61 84 0 03 Sep 2020
A Survey of Visual Analytics Techniques for Machine Learning Jun Yuan Changjian Chen Weikai Yang Mengchen Liu Jiazhi Xia Shixia Liu 88 222 0 21 Aug 2020
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models Ian Tenney James Wexler Jasmijn Bastings Tolga Bolukbasi Andy Coenen ... Ellen Jiang Mahima Pushkarna Carey Radebaugh Emily Reif Ann Yuan VLM 121 196 0 12 Aug 2020
DQI: Measuring Data Quality in NLP Swaroop Mishra Anjana Arunkumar Bhavdeep Singh Sachdeva Chris Bryan Chitta Baral 126 32 0 02 May 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 218 2,061 0 16 Apr 2020
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 117 223 0 10 Feb 2020
OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples Changjian Chen Jun Yuan Yafeng Lu Yang Liu Hang Su Songtao Yuan Shixia Liu OODD 55 65 0 08 Feb 2020
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation Jian Guan Fei Huang Zhihao Zhao Xiaoyan Zhu Minlie Huang LRM SyDa 70 249 0 15 Jan 2020
Word-level Textual Adversarial Attacking as Combinatorial Optimization Yuan Zang Fanchao Qi Chenghao Yang Zhiyuan Liu Meng Zhang Qun Liu Maosong Sun AAML 82 81 0 27 Oct 2019
A Visual Analytics Framework for Adversarial Text Generation Brandon Laughlin C. Collins K. Sankaranarayanan K. El-Khatib AAML 37 10 0 24 Sep 2019
Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment Di Jin Zhijing Jin Qiufeng Wang Peter Szolovits SILM AAML 191 1,088 0 27 Jul 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 686 24,557 0 26 Jul 2019
Probing Neural Network Comprehension of Natural Language Arguments Timothy Niven Hung-Yu kao AAML 96 455 0 17 Jul 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,326 0 02 May 2019
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis Hu Xu Bing-Quan Liu Lei Shu Philip S. Yu 59 700 0 03 Apr 2019
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence Chi Sun Luyao Huang Xipeng Qiu 72 613 0 22 Mar 2019
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference R. Thomas McCoy Ellie Pavlick Tal Linzen 143 1,244 0 04 Feb 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,229 0 11 Oct 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 242 1,413 0 31 May 2018
Hypothesis Only Baselines in Natural Language Inference Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme 241 580 0 02 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,200 0 20 Apr 2018
Visual Analytics for Explainable Deep Learning Jaegul Choo Shixia Liu HAI XAI 64 237 0 07 Apr 2018
Datasheets for Datasets Timnit Gebru Jamie Morgenstern Briana Vecchione Jennifer Wortman Vaughan Hanna M. Wallach Hal Daumé Kate Crawford 287 2,201 0 23 Mar 2018
Annotation Artifacts in Natural Language Inference Data Suchin Gururangan Swabha Swayamdipta Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith 155 1,180 0 06 Mar 2018
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction Leland McInnes John Healy James Melville 202 9,479 0 09 Feb 2018
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples Kimin Lee Honglak Lee Kibok Lee Jinwoo Shin OODD 128 882 0 26 Nov 2017
Understanding Hidden Memories of Recurrent Neural Networks Yao Ming Shaozu Cao Ruixiang Zhang Zerui Li Yuanzhe Chen Yangqiu Song Huamin Qu HAI 40 201 0 30 Oct 2017
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 842 5,841 0 05 Dec 2016