Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning

22 February 2024

Papers citing "Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning"

34 / 34 papers shown

Title
URLOST: Unsupervised Representation Learning without Stationarity or Topology Zeyu Yun Juexiao Zhang Bruno A. Olshausen Yann LeCun 138 1 0 06 Oct 2023
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers L. Yu Daniel Simig Colin Flaherty Armen Aghajanyan Luke Zettlemoyer M. Lewis 56 89 0 12 May 2023
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 133 640 0 22 Aug 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 68 280 0 13 Jul 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 116 407 0 14 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 416 6,202 0 05 Apr 2022
Masked Autoencoders for Point Cloud Self-supervised Learning Yatian Pang Wenxiao Wang Francis E. H. Tay Wen Liu Yonghong Tian Liuliang Yuan 3DPC ViT 75 466 0 13 Mar 2022
On Embeddings for Numerical Features in Tabular Deep Learning Yura Gorishniy Ivan Rubachev Artem Babenko LMTD 79 171 0 10 Mar 2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language Alexei Baevski Wei-Ning Hsu Qiantong Xu Arun Babu Jiatao Gu Michael Auli SSL VLM ViT 89 852 0 07 Feb 2022
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling Xumin Yu Lulu Tang Yongming Rao Tiejun Huang Jie Zhou Jiwen Lu 3DPC 111 669 0 29 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 427 7,705 0 11 Nov 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 52 574 0 30 Jul 2021
Revisiting Deep Learning Models for Tabular Data Yu. V. Gorishniy Ivan Rubachev Valentin Khrulkov Artem Babenko LMTD 94 720 0 22 Jun 2021
ByT5: Towards a token-free future with pre-trained byte-to-byte models Linting Xue Aditya Barua Noah Constant Rami Al-Rfou Sharan Narang Mihir Kale Adam Roberts Colin Raffel 83 501 0 28 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 611 6,029 0 29 Apr 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 159 1,007 0 04 Mar 2021
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 221 5,767 0 20 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 327 6,773 0 13 Jun 2020
Gradient Boosting Neural Networks: GrowNet Sarkhan Badirli Xuanqing Liu Zhengming Xing Avradeep Bhowmik Khoa D. Doan S. Keerthi FedML 54 84 0 19 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 367 20,053 0 23 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 208 3,480 0 30 Sep 2019
Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data Sergei Popov S. Morozov Artem Babenko LMTD 132 310 0 13 Sep 2019
TabNet: Attentive Interpretable Tabular Learning Sercan O. Arik Tomas Pfister LMTD 145 1,343 0 20 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 514 24,351 0 26 Jul 2019
Evaluating Protein Transfer Learning with TAPE Roshan Rao Nicholas Bhattacharya Neil Thomas Yan Duan Xi Chen John F. Canny Pieter Abbeel Yun S. Song SSL 86 796 0 19 Jun 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 151 2,395 0 13 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 604 4,766 0 13 May 2019
GuacaMol: Benchmarking Models for De Novo Molecular Design Nathan Brown Marco Fiscato Marwin H. S. Segler Alain C. Vaucher ELM 103 706 0 22 Nov 2018
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks Weiping Song Chence Shi Zhiping Xiao Zhijian Duan Yewen Xu Ming Zhang Jian Tang CML 56 854 0 29 Oct 2018
AutoAugment: Learning Augmentation Policies from Data E. D. Cubuk Barret Zoph Dandelion Mané Vijay Vasudevan Quoc V. Le 118 1,771 0 24 May 2018
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 269 9,743 0 25 Oct 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 364 2,507 0 08 Jun 2017
MoleculeNet: A Benchmark for Molecular Machine Learning Zhenqin Wu Bharath Ramsundar Evan N. Feinberg Joseph Gomes C. Geniesse Aneesh S. Pappu K. Leswing Vijay S. Pande OOD 323 1,825 0 02 Mar 2017
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 187 2,352 0 30 Mar 2016