On Catastrophic Inheritance of Large Foundation Models

2 February 2024

Hao Chen

Bhiksha Raj

Xing Xie

Jindong Wang

AI4CE

ArXiv (abs)PDF HTML

Papers citing "On Catastrophic Inheritance of Large Foundation Models"

50 / 66 papers shown

Title
On Memorization in Diffusion Models Xiangming Gu Chao Du Tianyu Pang Chongxuan Li Min Lin Ye Wang DiffM TDI 335 55 0 21 Feb 2025
Demystifying CLIP Data Hu Xu Saining Xie Xiaoqing Ellen Tan Po-Yao (Bernie) Huang Russell Howes Vasu Sharma Shang-Wen Li Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM CLIP 112 127 0 31 Dec 2024
Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models Xu Han Linghao Jin Xuezhe Ma Xiaofeng Liu AAML 65 3 0 02 Jul 2024
Just Say the Name: Online Continual Learning with Category Names Only via Data Generation Minhyuk Seo Diganta Misra Seongwon Cho Minjae Lee Jonghyun Choi CLL 83 9 0 16 Mar 2024
Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias Sierra Wyllie Ilia Shumailov Nicolas Papernot 73 30 0 12 Mar 2024
A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP) Weijie Tu Weijian Deng Tom Gedeon UQCV VLM 46 35 0 12 Feb 2024
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research Luca Soldaini Rodney Michael Kinney Akshita Bhagia Dustin Schwenk David Atkinson ... Hanna Hajishirzi Iz Beltagy Dirk Groeneveld Jesse Dodge Kyle Lo 92 278 0 31 Jan 2024
The Neglected Tails in Vision-Language Models Shubham Parashar Zhiqiu Lin Tian Liu Xiangjue Dong Yanan Li Deva Ramanan James Caverlee Shu Kong VLM 89 38 0 23 Jan 2024
The Ethics of Interaction: Mitigating Security Threats in LLMs Ashutosh Kumar Shiv Vignesh Murty Sagarika Singh Swathy Ragupathy 78 38 0 22 Jan 2024
Critical Data Size of Language Models from a Grokking Perspective Xuekai Zhu Yao Fu Bowen Zhou Zhouhan Lin 62 18 0 19 Jan 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 100 347 0 11 Jan 2024
Effective pruning of web-scale datasets based on complexity of concept clusters Amro Abbas E. Rusak Kushal Tirumala Wieland Brendel Kamalika Chaudhuri Ari S. Morcos VLM CLIP 56 23 0 09 Jan 2024
Do LLM Agents Exhibit Social Behavior? Yan Leng Yuan Yuan LLMAG 88 36 0 23 Dec 2023
NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes Lizhou Fan Wenyue Hua Lingyao Li Haoyang Ling Yongfeng Zhang LRM 90 57 0 22 Dec 2023
LLM360: Towards Fully Transparent Open-Source LLMs Zhengzhong Liu Aurick Qiao Willie Neiswanger Hongyi Wang Bowen Tan ... Zhiting Hu Mark Schulze Preslav Nakov Timothy Baldwin Eric Xing 115 78 0 11 Dec 2023
The mechanistic basis of data dependence and abrupt learning in an in-context classification task Gautam Reddy 88 66 0 03 Dec 2023
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples Shuo Yang Wei-Lin Chiang Lianmin Zheng Joseph E. Gonzalez Ion Stoica ALM 55 129 0 08 Nov 2023
What's In My Big Data? Yanai Elazar Akshita Bhagia Ian H. Magnusson Abhilasha Ravichander Dustin Schwenk ... Luca Soldaini Sameer Singh Hanna Hajishirzi Noah A. Smith Jesse Dodge 56 95 0 31 Oct 2023
Skywork: A More Open Bilingual Foundation Model Tianwen Wei Liang Zhao Lichang Zhang Bo Zhu Lijie Wang ... Yongyi Peng Xiaojuan Liang Shuicheng Yan Han Fang Yahui Zhou 85 101 0 30 Oct 2023
To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now Yimeng Zhang Jinghan Jia Xin Chen Aochuan Chen Yihua Zhang Jiancheng Liu Ke Ding Sijia Liu DiffM 114 100 0 18 Oct 2023
Large Language Models Are Zero-Shot Time Series Forecasters Nate Gruver Marc Finzi Shikai Qiu Andrew Gordon Wilson AI4TS 91 368 0 11 Oct 2023
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models Ming Jin Shiyu Wang Lintao Ma Zhixuan Chu James Y. Zhang ... Pin-Yu Chen Yuxuan Liang Yuan-Fang Li Shirui Pan Qingsong Wen AI4TS 114 414 0 03 Oct 2023
Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation Yucheng Li 57 35 0 19 Sep 2023
Gender bias and stereotypes in Large Language Models Hadas Kotek Rikker Dockum David Q. Sun 113 236 0 28 Aug 2023
D4: Improving LLM Pretraining via Document De-Duplication and Diversification Kushal Tirumala Daniel Simig Armen Aghajanyan Ari S. Morcos SyDa 52 113 0 23 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 293 1,508 0 27 Jul 2023
Frontier AI Regulation: Managing Emerging Risks to Public Safety Markus Anderljung Joslyn Barnhart Anton Korinek Jade Leung Cullen O'Keefe ... Jonas Schuett Yonadav Shavit Divya Siddarth Robert F. Trager Kevin J. Wolf SILM 86 125 0 06 Jul 2023
Self-Consuming Generative Models Go MAD Sina Alemohammad Josue Casco-Rodriguez Lorenzo Luzi Ahmed Imtiaz Humayun Hossein Babaei Daniel LeJeune Ali Siahkoohi Richard G. Baraniuk WIGM 89 156 0 04 Jul 2023
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity Shayne Longpre Gregory Yauney Emily Reif Katherine Lee Adam Roberts ... Denny Zhou Jason W. Wei Kevin Robinson David M. Mimno Daphne Ippolito 106 166 0 22 May 2023
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing Zhibin Gou Zhihong Shao Yeyun Gong Yelong Shen Yujiu Yang Nan Duan Weizhu Chen KELM LRM 79 394 0 19 May 2023
Unifying Grokking and Double Descent Peter W. Battaglia David Raposo Kelsey 75 32 0 10 Mar 2023
Data Portraits: Recording Foundation Model Training Data Marc Marone Benjamin Van Durme 181 30 0 06 Mar 2023
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective Jindong Wang Xixu Hu Wenxin Hou Hao Chen Runkai Zheng ... Weirong Ye Xiubo Geng Binxing Jiao Yue Zhang Xingxu Xie AI4MH 127 235 0 22 Feb 2023
Does progress on ImageNet transfer to real-world datasets? Alex Fang Simon Kornblith Ludwig Schmidt VLM 68 38 0 11 Jan 2023
Measuring Data Margaret Mitchell A. Luccioni Nathan Lambert Marissa Gerchick Angelina McMillan-Major Ezinwanne Ozoani Nazneen Rajani Tristan Thrush Yacine Jernite Douwe Kiela 76 17 0 09 Dec 2022
Large Language Models Struggle to Learn Long-Tail Knowledge Nikhil Kandpal H. Deng Adam Roberts Eric Wallace Colin Raffel RALM KELM 128 417 0 15 Nov 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 200 3,493 0 16 Oct 2022
Knowledge Unlearning for Mitigating Privacy Risks in Language Models Joel Jang Dongkeun Yoon Sohee Yang Sungmin Cha Moontae Lee Lajanugen Logeswaran Minjoon Seo KELM PILM MU 202 239 0 04 Oct 2022
USB: A Unified Semi-supervised Learning Benchmark for Classification Yidong Wang Hao Chen Yue Fan Wangbin Sun R. Tao ... T. Shinozaki Bernt Schiele Jindong Wang Xingxu Xie Yue Zhang 69 118 0 12 Aug 2022
Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP Thao Nguyen Gabriel Ilharco Mitchell Wortsman Sewoong Oh Ludwig Schmidt CLIP VLM 126 105 0 10 Aug 2022
Beyond neural scaling laws: beating power law scaling via data pruning Ben Sorscher Robert Geirhos Shashank Shekhar Surya Ganguli Ari S. Morcos 97 444 0 29 Jun 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,511 0 15 Jun 2022
Scaling Laws and Interpretability of Learning from Repeated Data Danny Hernandez Tom B. Brown Tom Conerly Nova Dassarma Dawn Drain ... Catherine Olsson Dario Amodei Nicholas Joseph Jared Kaplan Sam McCandlish 75 117 0 21 May 2022
Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP) Alex Fang Gabriel Ilharco Mitchell Wortsman Yu Wan Vaishaal Shankar Achal Dave Ludwig Schmidt VLM OOD 102 146 0 03 May 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 631 0 15 Feb 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 485 15,734 0 20 Dec 2021
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 243 1,442 0 03 Nov 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 135 952 0 08 Oct 2021
Exploring the Limits of Large Scale Pre-training Samira Abnar Mostafa Dehghani Behnam Neyshabur Hanie Sedghi AI4CE 97 119 0 05 Oct 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 360 634 0 14 Jul 2021