Evaluating representations by the complexity of learning low-loss predictors

15 September 2020

David Brandfonbrener

Papers citing "Evaluating representations by the complexity of learning low-loss predictors"

30 / 30 papers shown

Title
Learning Optimal Representations with the Decodable Information Bottleneck Yann Dubois Douwe Kiela D. Schwab Ramakrishna Vedantam 56 43 0 27 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 675 41,736 0 28 May 2020
Information-Theoretic Probing with Minimum Description Length Elena Voita Ivan Titov 68 274 0 27 Mar 2020
A Theory of Usable Information Under Computational Constraints Yilun Xu Shengjia Zhao Jiaming Song Russell Stewart Stefano Ermon 70 173 0 25 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 331 18,721 0 13 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 532 4,773 0 23 Jan 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 96 303 0 31 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 379 42,299 0 03 Dec 2019
Probing the State of the Art: A Critical Look at Visual Representation Evaluation Cinjon Resnick Zeping Zhan Joan Bruna AI4TS 37 12 0 30 Nov 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 171 12,065 0 13 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 375 20,053 0 23 Oct 2019
Designing and Interpreting Probes with Control Tasks John Hewitt Percy Liang 58 536 0 08 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 524 24,351 0 26 Jul 2019
Measuring Bias in Contextualized Word Representations Keita Kurita Nidhi Vyas Ayush Pareek A. Black Yulia Tsvetkov 98 449 0 18 Jun 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 62 2,647 0 05 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 187 1,467 0 03 Jun 2019
Data-Efficient Image Recognition with Contrastive Predictive Coding Olivier J. Hénaff A. Srinivas J. Fauw Ali Razavi Carl Doersch S. M. Ali Eslami Aaron van den Oord SSL 111 1,427 0 22 May 2019
Identifying and Reducing Gender Bias in Word-Level Language Models Shikha Bordia Samuel R. Bowman FaML 111 326 0 05 Apr 2019
Learning and Evaluating General Linguistic Intelligence Dani Yogatama Cyprien de Masson dÁutume Jerome T. Connor Tomás Kociský Mike Chrzanowski ... Angeliki Lazaridou Wang Ling Lei Yu Chris Dyer Phil Blunsom ELM AI4CE 141 210 0 31 Jan 2019
Formal Limitations on the Measurement of Mutual Information David A. McAllester K. Stratos SSL 63 275 0 10 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.5K 94,511 0 11 Oct 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 290 10,253 0 10 Jul 2018
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 317 892 0 03 May 2018
The Description Length of Deep Learning Models Léonard Blier Yann Ollivier 66 96 0 20 Feb 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 190 11,542 0 15 Feb 2018
Understanding intermediate layers using linear classifier probes Guillaume Alain Yoshua Bengio FAtt 133 941 0 05 Oct 2016
Learning Distributed Representations of Sentences from Unlabelled Data Felix Hill Kyunghyun Cho Anna Korhonen SSL 156 574 0 10 Feb 2016
Skip-Thought Vectors Ryan Kiros Yukun Zhu Ruslan Salakhutdinov R. Zemel Antonio Torralba R. Urtasun Sanja Fidler SSL 198 2,408 0 22 Jun 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.5K 149,842 0 22 Dec 2014
Auto-Encoding Variational Bayes Diederik P. Kingma Max Welling BDL 422 16,944 0 20 Dec 2013