Title
Training Production Language Models without Memorizing User Data Swaroop Indra Ramaswamy Om Thakkar Rajiv Mathews Galen Andrew H. B. McMahan Franccoise Beaufays FedML 67 92 0 21 Sep 2020
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation Vitaly Feldman Chiyuan Zhang TDI 193 464 0 09 Aug 2020
Auditing Differentially Private Machine Learning: How Private is Private SGD? Matthew Jagielski Jonathan R. Ullman Alina Oprea FedML 74 245 0 13 Jun 2020
Understanding Unintended Memorization in Federated Learning Om Thakkar Swaroop Indra Ramaswamy Rajiv Mathews Franccoise Beaufays FedML 67 47 0 12 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 820 42,055 0 28 May 2020
Information Leakage in Embedding Models Congzheng Song A. Raghunathan MIACV 57 270 0 31 Mar 2020
Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers Zhuohan Li Eric Wallace Sheng Shen Kevin Lin Kurt Keutzer Dan Klein Joseph E. Gonzalez 102 151 0 26 Feb 2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 121 891 0 10 Feb 2020
Towards a Human-like Open-Domain Chatbot Daniel De Freitas Minh-Thang Luong David R. So Jamie Hall Noah Fiedel ... Zi Yang Apoorv Kulshreshtha Gaurav Nemade Yifeng Lu Quoc V. Le 114 938 0 27 Jan 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 608 4,893 0 23 Jan 2020
Analyzing Information Leakage of Updates to Natural Language Models Santiago Zanella Béguelin Lukas Wutschitz Shruti Tople Victor Rühle Andrew Paverd O. Ohrimenko Boris Köpf Marc Brockschmidt ELM MIACV FedML PILM KELM 53 127 0 17 Dec 2019
Automatic Detection of Generated Text is Easiest when Humans are Fooled Daphne Ippolito Daniel Duckworth Chris Callison-Burch Douglas Eck DeLMO 90 362 0 02 Nov 2019
DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation Yizhe Zhang Siqi Sun Michel Galley Yen-Chun Chen Chris Brockett Xiang Gao Jianfeng Gao Jingjing Liu W. Dolan VLM 189 1,524 0 01 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 257 10,848 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 445 20,298 0 23 Oct 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 331 1,914 0 17 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 576 2,673 0 03 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,528 0 26 Jul 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 130 497 0 12 Jun 2019
Efficient Adaptation of Pretrained Transformers for Abstractive Summarization Andrew Hoang Antoine Bosselut Asli Celikyilmaz Yejin Choi 60 40 0 01 Jun 2019
Defending Against Neural Fake News Rowan Zellers Ari Holtzman Hannah Rashkin Yonatan Bisk Ali Farhadi Franziska Roesner Yejin Choi AAML 125 1,029 0 29 May 2019
Gmail Smart Compose: Real-Time Assisted Writing Mengzhao Chen Benjamin Lee G. Bansal Yuan Cao Shuyuan Zhang ... Yinan Wang Andrew M. Dai Zhiwen Chen Timothy Sohn Yonghui Wu 53 206 0 17 May 2019
The Curious Case of Neural Text Degeneration Ari Holtzman Jan Buys Li Du Maxwell Forbes Yejin Choi 190 3,201 0 22 Apr 2019
Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System? Sorami Hisamoto Matt Post Kevin Duh MIACV SLR 58 106 0 11 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
Machine Learning with Membership Privacy using Adversarial Regularization Milad Nasr Reza Shokri Amir Houmansadr FedML MIACV 52 474 0 16 Jul 2018
Hierarchical Neural Story Generation Angela Fan M. Lewis Yann N. Dauphin DiffM 181 1,623 0 13 May 2018
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks Nicholas Carlini Chang-rui Liu Ulfar Erlingsson Jernej Kos Basel Alomair 144 1,146 0 22 Feb 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 219 11,565 0 15 Feb 2018
Understanding Membership Inferences on Well-Generalized Learning Models Yunhui Long Vincent Bindschaedler Lei Wang Diyue Bu Xiaofeng Wang Haixu Tang Carl A. Gunter Kai Chen MIALM MIACV 55 224 0 13 Feb 2018
Ethical Challenges in Data-Driven Dialogue Systems Peter Henderson Koustuv Sinha Nicolas Angelard-Gontier Nan Rosemary Ke G. Fried Ryan J. Lowe Joelle Pineau 72 172 0 24 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 719 132,199 0 12 Jun 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 345 4,629 0 10 Nov 2016
Membership Inference Attacks against Machine Learning Models Reza Shokri M. Stronati Congzheng Song Vitaly Shmatikov SLR MIALM MIACV 263 4,152 0 18 Oct 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,130 0 01 Jul 2016
A Diversity-Promoting Objective Function for Neural Conversation Models Jiwei Li Michel Galley Chris Brockett Jianfeng Gao W. Dolan 145 2,401 0 11 Oct 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 224 7,755 0 31 Aug 2015
ReCon: Revealing and Controlling PII Leaks in Mobile Network Traffic Jingjing Ren Ashwin Rao Martina Lindorfer A. Legout David Choffnes 67 242 0 01 Jul 2015
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 573 27,311 0 01 Sep 2014
RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response Ulfar Erlingsson Vasyl Pihur Aleksandra Korolova 96 1,992 0 25 Jul 2014
Generating Sequences With Recurrent Neural Networks Alex Graves GAN 155 4,039 0 04 Aug 2013
Learning in a Large Function Space: Privacy-Preserving Mechanisms for SVM Learning Benjamin I. P. Rubinstein Peter L. Bartlett Ling Huang N. Taft 106 295 0 30 Nov 2009