v1v2 (latest)

Denoising Autoregressive Representation Learning

8 March 2024

Papers citing "Denoising Autoregressive Representation Learning"

39 / 39 papers shown

Title
Scalable Pre-training of Large Autoregressive Image Models Alaaeldin El-Nouby Michal Klein Shuangfei Zhai Miguel Angel Bautista Alexander Toshev Vaishaal Shankar J. Susskind Armand Joulin VLM 98 80 0 16 Jan 2024
The Impact of Positional Encoding on Length Generalization in Transformers Amirhossein Kazemnejad Inkit Padhi Karthikeyan N. Ramamurthy Payel Das Siva Reddy 75 205 0 31 May 2023
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder Z. Fu W. Lam Qian Yu Anthony Man-Cho So Shengding Hu Zhiyuan Liu Nigel Collier AuLLM 58 43 0 08 Apr 2023
Simple diffusion: End-to-end diffusion for high resolution images Emiel Hoogeboom Jonathan Heek Tim Salimans 98 267 0 26 Jan 2023
On the Importance of Noise Scheduling for Diffusion Models Ting Chen DiffM 86 158 0 26 Jan 2023
Exploring Plain Vision Transformer Backbones for Object Detection Yanghao Li Hanzi Mao Ross B. Girshick Kaiming He ViT 92 815 0 30 Mar 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 477 7,827 0 11 Nov 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 239 4,004 0 28 Jul 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 131 635 0 18 Jun 2021
Self-Supervised Learning with Kernel Dependence Maximization Yazhe Li Roman Pogodin Danica J. Sutherland Arthur Gretton SSL 76 84 0 15 Jun 2021
Diffusion Models Beat GANs on Image Synthesis Prafulla Dhariwal Alex Nichol 271 7,958 0 11 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 730 6,135 0 29 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 999 29,871 0 26 Feb 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 420 5,000 0 24 Feb 2021
Improved Denoising Diffusion Probabilistic Models Alex Nichol Prafulla Dhariwal DiffM 354 3,728 0 18 Feb 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 682 41,483 0 22 Oct 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 742 18,364 0 19 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 401 6,844 0 13 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 889 42,463 0 28 May 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 390 18,897 0 13 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 639 4,921 0 23 Jan 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 216 12,136 0 13 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 490 20,342 0 23 Oct 2019
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 97 447 0 01 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 260 3,505 0 30 Sep 2019
Large Scale Adversarial Representation Learning Jeff Donahue Karen Simonyan SSL 135 544 0 04 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,451 0 19 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 195 1,479 0 03 Jun 2019
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 351 10,364 0 10 Jul 2018
Unsupervised Representation Learning by Predicting Image Rotations Spyros Gidaris Praveer Singh N. Komodakis OOD SSL DRL 264 3,300 0 21 Mar 2018
MINE: Mutual Information Neural Estimation Mohamed Ishmael Belghazi A. Baratin Sai Rajeswar Sherjil Ozair Yoshua Bengio Aaron Courville R. Devon Hjelm DRL 200 1,285 0 12 Jan 2018
Cascade R-CNN: Delving into High Quality Object Detection Zhaowei Cai Nuno Vasconcelos ObjD 149 4,943 0 03 Dec 2017
Conditional Image Generation with PixelCNN Decoders Aaron van den Oord Nal Kalchbrenner Oriol Vinyals L. Espeholt Alex Graves Koray Kavukcuoglu VLM 217 2,519 0 16 Jun 2016
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles M. Noroozi Paolo Favaro SSL 180 2,986 0 30 Mar 2016
Colorful Image Colorization Richard Y. Zhang Phillip Isola Alexei A. Efros 149 3,534 0 28 Mar 2016
Pixel Recurrent Neural Networks Aaron van den Oord Nal Kalchbrenner Koray Kavukcuoglu SSeg GAN 486 2,579 0 25 Jan 2016
Unsupervised Visual Representation Learning by Context Prediction Carl Doersch Abhinav Gupta Alexei A. Efros DRL SSL 169 2,790 0 19 May 2015
Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Narain Sohl-Dickstein Eric A. Weiss Niru Maheswaranathan Surya Ganguli SyDa DiffM 312 7,031 0 12 Mar 2015
MADE: Masked Autoencoder for Distribution Estimation M. Germain Karol Gregor Iain Murray Hugo Larochelle OOD SyDa UQCV 187 874 0 12 Feb 2015