Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning

22 February 2024

Papers citing "Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning"

38 / 38 papers shown

Title
URLOST: Unsupervised Representation Learning without Stationarity or Topology Zeyu Yun Juexiao Zhang Bruno A. Olshausen Yann LeCun 174 1 0 06 Oct 2023
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers L. Yu Daniel Simig Colin Flaherty Armen Aghajanyan Luke Zettlemoyer M. Lewis 63 91 0 12 May 2023
AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders W. G. C. Bandara Naman Patel A. Gholami Mehdi Nikkhah M. Agrawal Vishal M. Patel 50 41 0 16 Nov 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 143 642 0 22 Aug 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 89 282 0 13 Jul 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 125 414 0 14 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 498 6,240 0 05 Apr 2022
Masked Autoencoders for Point Cloud Self-supervised Learning Yatian Pang Wenxiao Wang Francis E. H. Tay Wen Liu Yonghong Tian Liuliang Yuan 3DPC ViT 86 475 0 13 Mar 2022
On Embeddings for Numerical Features in Tabular Deep Learning Yura Gorishniy Ivan Rubachev Artem Babenko LMTD 97 172 0 10 Mar 2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language Alexei Baevski Wei-Ning Hsu Qiantong Xu Arun Babu Jiatao Gu Michael Auli SSL VLM ViT 97 858 0 07 Feb 2022
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling Xumin Yu Lulu Tang Yongming Rao Tiejun Huang Jie Zhou Jiwen Lu 3DPC 136 675 0 29 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 467 7,757 0 11 Nov 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 76 583 0 30 Jul 2021
Revisiting Deep Learning Models for Tabular Data Yu. V. Gorishniy Ivan Rubachev Valentin Khrulkov Artem Babenko LMTD 119 760 0 22 Jun 2021
ByT5: Towards a token-free future with pre-trained byte-to-byte models Linting Xue Aditya Barua Noah Constant Rami Al-Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel 98 504 0 28 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 694 6,079 0 29 Apr 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 197 1,015 0 04 Mar 2021
CLEAR: Contrastive Learning for Sentence Representation Zhuofeng Wu Sinong Wang Jiatao Gu Madian Khabsa Fei Sun Hao Ma SSL 71 323 0 31 Dec 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 285 5,801 0 20 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 371 6,806 0 13 Jun 2020
Gradient Boosting Neural Networks: GrowNet Sarkhan Badirli Xuanqing Liu Zhengming Xing Avradeep Bhowmik Khoa D. Doan S. Keerthi FedML 56 86 0 19 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 369 18,778 0 13 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 445 20,181 0 23 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 237 3,490 0 30 Sep 2019
Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data Sergei Popov S. Morozov Artem Babenko LMTD 139 314 0 13 Sep 2019
TabNet: Attentive Interpretable Tabular Learning Sercan O. Arik Tomas Pfister LMTD 188 1,353 0 20 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 665 24,464 0 26 Jul 2019
Evaluating Protein Transfer Learning with TAPE Roshan Rao Nicholas Bhattacharya Neil Thomas Yan Duan Xi Chen John F. Canny Pieter Abbeel Yun S. Song SSL 94 803 0 19 Jun 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 169 2,403 0 13 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 620 4,780 0 13 May 2019
GuacaMol: Benchmarking Models for De Novo Molecular Design Nathan Brown Marco Fiscato Marwin H. S. Segler Alain C. Vaucher ELM 112 708 0 22 Nov 2018
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks Weiping Song Chence Shi Zhiping Xiao Zhijian Duan Yewen Xu Ming Zhang Jian Tang CML 84 856 0 29 Oct 2018
AutoAugment: Learning Augmentation Policies from Data E. D. Cubuk Barret Zoph Dandelion Mané Vijay Vasudevan Quoc V. Le 131 1,772 0 24 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 280 9,764 0 25 Oct 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 460 2,516 0 08 Jun 2017
MoleculeNet: A Benchmark for Molecular Machine Learning Zhenqin Wu Bharath Ramsundar Evan N. Feinberg Joseph Gomes C. Geniesse Aneesh S. Pappu K. Leswing Vijay S. Pande OOD 337 1,827 0 02 Mar 2017
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 215 2,357 0 30 Mar 2016