Data and its (dis)contents: A survey of dataset development and use in machine learning research

9 December 2020

Amandalynne Paullada

Inioluwa Deborah Raji

Papers citing "Data and its (dis)contents: A survey of dataset development and use in machine learning research"

50 / 78 papers shown

Title
Toward an Evaluation Science for Generative AI Systems Laura Weidinger Deb Raji Hanna M. Wallach Margaret Mitchell Angelina Wang Olawale Salaudeen Rishi Bommasani Sayash Kapoor Deep Ganguli Sanmi Koyejo EGVM ELM 67 4 0 07 Mar 2025
Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation Maria Eriksson Erasmo Purificato Arman Noroozian Joao Vinagre Guillaume Chaslot Emilia Gomez David Fernandez Llorca ELM 139 1 0 10 Feb 2025
Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals Qingyang Wu Ying Xu Tingsong Xiao Yunze Xiao Yitong Li ... Yichi Zhang Shanghai Zhong Yuwei Zhang Wei Lu Yifan Yang 78 2 0 17 Jan 2025
Authenticated Delegation and Authorized AI Agents Tobin South Samuele Marro Thomas Hardjono Robert Mahari Cedric Deslandes Whitney Dazza Greenwood Alan Chan Alex Pentland 52 3 0 17 Jan 2025
To which reference class do you belong? Measuring racial fairness of reference classes with normative modeling S. Rutherford T. Wolfers Charlotte J. Fraza Nathaniel G. Harrnet Christian F. Beckmann H. Ruhé A. Marquand CML 56 3 0 26 Jul 2024
Is That Rain? Understanding Effects on Visual Odometry Performance for Autonomous UAVs and Efficient DNN-based Rain Classification at the Edge Andrea Albanese Yanran Wang Davide Brunelli David E. Boyle 34 1 0 17 Jul 2024
Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers Harald Semmelrock Tony Ross-Hellauer Simone Kopeinik Dieter Theiler Armin Haberl Stefan Thalmann Dominik Kowald 65 6 0 20 Jun 2024
CowScreeningDB: A public benchmark dataset for lameness detection in dairy cows Shahid Ismail Moisés Díaz Cristina Carmona-Duarte Jose Manuel Vilar M. A. Ferrer-Ballester 18 1 0 24 May 2024
Navigating LLM Ethics: Advancements, Challenges, and Future Directions Junfeng Jiao S. Afroogh Yiming Xu Connor Phillips AILaw 65 19 0 14 May 2024
TIGQA:An Expert Annotated Question Answering Dataset in Tigrinya Hailay Teklehaymanot Dren Fazlija Niloy Ganguly Gourab K. Patro Wolfgang Nejdl 34 0 0 26 Apr 2024
Hallucination is Inevitable: An Innate Limitation of Large Language Models Ziwei Xu Sanjay Jain Mohan S. Kankanhalli HILM LRM 71 212 0 22 Jan 2024
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance Christoph Kern Stephanie Eckman Jacob Beck Rob Chew Bolei Ma Frauke Kreuter 24 9 0 23 Nov 2023
Prototype-based Dataset Comparison Nanne van Noord 31 6 0 05 Sep 2023
AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia Rida Qadri Renee Shelby Cynthia L. Bennett Emily Denton 26 67 0 19 May 2023
A benchmark for computational analysis of animal behavior, using animal-borne tags Benjamin Hoffman M. Cusimano V. Baglione D. Canestrari D. Chevallier ... O. Vainio A. Vehkaoja Ken Yoda Katie Zacarian A. Friedlaender 25 7 0 18 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 110 1,148 0 17 May 2023
The MiniPile Challenge for Data-Efficient Language Models Jean Kaddour MoE ALM 24 40 0 17 Apr 2023
An investigation of licensing of datasets for machine learning based on the GQM model Junyu Chen Norihiro Yoshida Hiroaki Takada 33 2 0 24 Mar 2023
A Bag-of-Prototypes Representation for Dataset-Level Applications Wei-Chih Tu Weijian Deng Tom Gedeon Liang Zheng 38 9 0 23 Mar 2023
Overwriting Pretrained Bias with Finetuning Data Angelina Wang Olga Russakovsky 26 29 0 10 Mar 2023
Auditing large language models: a three-layered approach Jakob Mokander Jonas Schuett Hannah Rose Kirk Luciano Floridi AILaw MLAU 48 194 0 16 Feb 2023
Bipol: Multi-axes Evaluation of Bias with Explainability in Benchmark Datasets Tosin P. Adewumi Isabella Sodergren Lama Alkhaled Sana Sabah Sabry F. Liwicki Marcus Liwicki 35 4 0 28 Jan 2023
Accuracy and Fidelity Comparison of Luna and DALL-E 2 Diffusion-Based Image Generation Systems Michael Cahyadi M. Rafi William Shan Jurike V. Moniaga Henry Lucky 35 4 0 05 Jan 2023
Muse: Text-To-Image Generation via Masked Generative Transformers Huiwen Chang Han Zhang Jarred Barber AJ Maschinot José Lezama ... Kevin Patrick Murphy William T. Freeman Michael Rubinstein Yuanzhen Li Dilip Krishnan DiffM 197 519 0 02 Jan 2023
Evaluation for Change Rishi Bommasani ELM 40 0 0 20 Dec 2022
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources Xinyan Velocity Yu Akari Asai Trina Chatterjee Junjie Hu Eunsol Choi 20 21 0 28 Nov 2022
The Grind for Good Data: Understanding ML Practitioners' Struggles and Aspirations in Making Good Data Inha Cha Juhyun Oh Cheul Young Park Jiyoon Han Hwalsuk Lee 29 2 0 28 Nov 2022
Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation Sérgio Jesus José P. Pombal Duarte M. Alves André F. Cruz Pedro Saleiro Rita P. Ribeiro João Gama P. Bizarro 40 32 0 24 Nov 2022
A Blockchain Protocol for Human-in-the-Loop AI N. Dehouche R. Blythman 18 0 0 20 Nov 2022
NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research J. Bornschein Alexandre Galashov Ross Hemsley Amal Rannen-Triki Yutian Chen ... Angeliki Lazaridou Yee Whye Teh Andrei A. Rusu Razvan Pascanu MarcÁurelio Ranzato OOD VLM AI4TS 39 16 0 15 Nov 2022
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale Federico Bianchi Pratyusha Kalluri Esin Durmus Faisal Ladhak Myra Cheng Debora Nozza Tatsunori Hashimoto Dan Jurafsky James Zou Aylin Caliskan DiffM VLM 36 288 0 07 Nov 2022
State-of-the-art Models for Object Detection in Various Fields of Application S. A. G. Naqvi Syed Shahnawaz Ali ObjD OOD 35 0 0 01 Nov 2022
Men Also Do Laundry: Multi-Attribute Bias Amplification Dora Zhao Jerone T. A. Andrews Alice Xiang FaML 41 20 0 21 Oct 2022
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction Renee Shelby Shalaleh Rismani Kathryn Henne AJung Moon Negar Rostamzadeh ... N'Mah Yilla-Akbari Jess Gallegos A. Smart Emilio Garcia Gurleen Virk 36 188 0 11 Oct 2022
Attention is All They Need: Exploring the Media Archaeology of the Computer Vision Research Paper Sam Goree G. Appleby David J. Crandall Norman Su 29 2 0 22 Sep 2022
Active-Passive SimStereo -- Benchmarking the Cross-Generalization Capabilities of Deep Learning-based Stereo Methods Laurent Valentin Jospin A. Antony Lian Xu Hamid Laga F. Boussaïd Bennamoun 26 4 0 17 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 30 109 0 31 Aug 2022
ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset Zhihua Jin Xingbo Wang Furui Cheng Chunhui Sun Qun Liu Huamin Qu 32 9 0 17 Aug 2022
Development and Validation of ML-DQA -- a Machine Learning Data Quality Assurance Framework for Healthcare M. Sendak Gaurav Sirdeshmukh Timothy N. Ochoa Hayley Premo Linda Tang ... M. Nichols Bradley Heintze William S Knechtle W. Ratliff S. Balu 11 6 0 04 Aug 2022
Labeling instructions matter in biomedical image analysis Tim Radsch Annika Reinke V. Weru M. Tizabi Nicholas Schreck A. Emre Kavur Bunyamin Pekdemir T. Ross A. Kopp-Schneider Lena Maier-Hein 25 53 0 20 Jul 2022
Leakage and the Reproducibility Crisis in ML-based Science Sayash Kapoor Arvind Narayanan 25 177 0 14 Jul 2022
Natural Backdoor Datasets Emily Wenger Roma Bhattacharjee A. Bhagoji Josephine Passananti Emilio Andere Haitao Zheng Ben Y. Zhao AAML 33 4 0 21 Jun 2022
Certifying Data-Bias Robustness in Linear Regression Anna P. Meyer Aws Albarghouthi Loris Dántoni 29 3 0 07 Jun 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 66 5,778 0 23 May 2022
When does dough become a bagel? Analyzing the remaining mistakes on ImageNet Vijay Vasudevan Benjamin Caine Raphael Gontijo-Lopes Sara Fridovich-Keil Rebecca Roelofs VLM UQCV 46 57 0 09 May 2022
Can Information Behaviour Inform Machine Learning? M. Ridley AI4CE 21 0 0 01 May 2022
Handling and Presenting Harmful Text in NLP Research Hannah Rose Kirk Abeba Birhane Bertie Vidgen Leon Derczynski 15 47 0 29 Apr 2022
You Are What You Write: Preserving Privacy in the Era of Large Language Models Richard Plant V. Giuffrida Dimitra Gkatzia PILM 23 19 0 20 Apr 2022
Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review E. Olaniyi Dong Chen Yuzhen Lu Ya-Yu Huang 21 38 0 10 Apr 2022
A Theoretically Grounded Benchmark for Evaluating Machine Commonsense Henrique M. Dinis Santos Ke Shen Alice M. Mulvehill Yasaman Razeghi D. McGuinness Mayank Kejriwal ELM LRM 22 4 0 23 Mar 2022