Revisiting Pretraining Objectives for Tabular Deep Learning

7 July 2022

Papers citing "Revisiting Pretraining Objectives for Tabular Deep Learning"

29 / 29 papers shown

Title
TabSTAR: A Foundation Tabular Model With Semantically Target-Aware Representations Alan Arazi Eilam Shapira Roi Reichart LMTD 156 0 0 23 May 2025
TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling Yury Gorishniy Akim Kotelnikov Artem Babenko LMTD MoE 224 11 0 31 Oct 2024
On Embeddings for Numerical Features in Tabular Deep Learning Yura Gorishniy Ivan Rubachev Artem Babenko LMTD 79 171 0 10 Mar 2022
Are Large-scale Datasets Necessary for Self-Supervised Pre-training? Alaaeldin El-Nouby Gautier Izacard Hugo Touvron Ivan Laptev Hervé Jégou Edouard Grave SSL 62 149 0 20 Dec 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 427 7,705 0 11 Nov 2021
SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning Talip Uçar Ehsan Hajiramezanali Lindsay Edwards LMTD SSL 58 131 0 08 Oct 2021
Contrastive Mixup: Self- and Semi-Supervised learning for Tabular Domain Sajad Darabi Shayan Fazeli Ali Pazoki S. Sankararaman Majid Sarrafzadeh SSL 38 29 0 27 Aug 2021
Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks A. Malinin Neil Band Ganshin Alexander German Chesnokov ... Roginskiy Denis Mariya Shmatova Panos Tigas Boris Yangel UQCV OOD 73 131 0 15 Jul 2021
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption Dara Bahri Heinrich Jiang Yi Tay Donald Metzler SSL 54 173 0 29 Jun 2021
Revisiting Deep Learning Models for Tabular Data Yu. V. Gorishniy Ivan Rubachev Valentin Khrulkov Artem Babenko LMTD 96 720 0 22 Jun 2021
Well-tuned Simple Nets Excel on Tabular Datasets Arlind Kadra Marius Lindauer Frank Hutter Josif Grabocka 38 193 0 21 Jun 2021
Tabular Data: Deep Learning is Not All You Need Ravid Shwartz-Ziv Amitai Armon LMTD 139 1,256 0 06 Jun 2021
Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning Jannik Kossen Neil Band Clare Lyle Aidan Gomez Tom Rainforth Y. Gal OOD 3DPC 80 139 0 04 Jun 2021
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training Gowthami Somepalli Micah Goldblum Avi Schwarzschild C. Bayan Bruss Tom Goldstein LMTD 82 324 0 02 Jun 2021
TabTransformer: Tabular Data Modeling Using Contextual Embeddings Xin Huang A. Khetan Milan Cvitkovic Zohar Karnin ViT LMTD 195 447 0 11 Dec 2020
Supervised Contrastive Learning Prannay Khosla Piotr Teterwak Chen Wang Aaron Sarna Yonglong Tian Phillip Isola Aaron Maschinot Ce Liu Dilip Krishnan SSL 139 4,532 0 23 Apr 2020
Gradient Boosting Neural Networks: GrowNet Sarkhan Badirli Xuanqing Liu Zhengming Xing Avradeep Bhowmik Khoa D. Doan S. Keerthi FedML 54 84 0 19 Feb 2020
The Tree Ensemble Layer: Differentiability meets Conditional Computation Hussein Hazimeh Natalia Ponomareva P. Mol Zhenyu Tan Rahul Mazumder UQCV AI4CE 302 79 0 18 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 325 18,721 0 13 Feb 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 165 12,065 0 13 Nov 2019
Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data Sergei Popov S. Morozov Artem Babenko LMTD 132 310 0 13 Sep 2019
TabNet: Attentive Interpretable Tabular Learning Sercan O. Arik Tomas Pfister LMTD 149 1,343 0 20 Aug 2019
Optuna: A Next-generation Hyperparameter Optimization Framework Takuya Akiba Shotaro Sano Toshihiko Yanase Takeru Ohta Masanori Koyama 527 5,769 0 25 Jul 2019
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks Weiping Song Chence Shi Zhiping Xiao Zhijian Duan Yewen Xu Ming Zhang Jian Tang CML 60 854 0 29 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.5K 94,511 0 11 Oct 2018
Deep & Cross Network for Ad Click Predictions Ruoxi Wang Bin Fu Gang Fu Mingliang Wang 77 1,230 0 17 Aug 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 380 2,507 0 08 Jun 2017
XGBoost: A Scalable Tree Boosting System Tianqi Chen Carlos Guestrin 613 38,735 0 09 Mar 2016
OpenML: networked science in machine learning Joaquin Vanschoren Jan N. van Rijn B. Bischl Luís Torgo FedML AI4CE 139 1,321 0 29 Jul 2014