Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery

8 March 2024

Mubashir Noman

Muzammal Naseer

Hisham Cholakkal

Rao Muhammad Anwar

Salman Khan

Fahad Shahbaz Khan

ViT

ArXiv PDF HTML

Papers citing "Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery"

25 / 25 papers shown

Title
CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework Wentao Wu Xinyu Wang Chenglong Li Bo Jiang Jin Tang Bin Luo Qi Liu 34 0 0 17 Apr 2025
Satellite Federated Fine-Tuning for Foundation Models in Space Computing Power Networks Yan zhu Jingyang Zhu Ting Wang Yuanming Shi Chunxiao Jiang Khaled B. Letaief 29 0 0 14 Apr 2025
REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval Shabnam Choudhury Yash Salunkhe Sarthak Mehrotra Biplab Banerjee 36 0 0 04 Apr 2025
A Survey on Remote Sensing Foundation Models: From Vision to Multimodality Ziyue Huang Hongxi Yan Qiqi Zhan Shuai Yang Mingming Zhang Chenkai Zhang Yiming Lei Zeming Liu Qingjie Liu Yixuan Wang 49 0 0 28 Mar 2025
Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data Haozhe Si Yuxuan Wan Minh Do Deepak Vasisht Han Zhao Hendrik Hamann 48 0 0 17 Mar 2025
Towards a Unified Copernicus Foundation Model for Earth Vision Yi Wang Zhitong Xiong Chenying Liu Adam J. Stewart Thomas Dujardin ... Angelos Zavras Franziska Gerken Ioannis Papoutsis Laura Leal-Taixé Xiao Xiang Zhu 50 1 0 14 Mar 2025
Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation Leonard Waldmann Ando Shah Yi Wang Nils Lehmann Adam J. Stewart Zhitong Xiong Xiao Xiang Zhu Stefan Bauer John Chuang 51 1 0 13 Mar 2025
RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing Fengxiang Wang H. Wang Yixuan Wang Di Wang Mingshuo Chen ... Yangang Sun Shuo Wang L. Lan Wenjing Yang Jing Zhang Mamba 80 3 0 13 Mar 2025
SAR-W-MixMAE: SAR Foundation Model Training Using Backscatter Power Weighting Ali Caglayan Nevrez Imamoglu T. Kouyama 70 0 0 03 Mar 2025
CARE: Confidence-Aware Regression Estimation of building density fine-tuning EO Foundation Models Nikolaos Dionelis Jente Bosmans Nicolas Longépé 71 0 0 19 Feb 2025
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision Diego A. Velázquez Pau Rodríguez López Sergio Alonso Josep M. Gonfaus Jordi Gonzalez Gerardo Richarte Javier Marin Yoshua Bengio Alexandre Lacoste 60 0 0 14 Jan 2025
Pattern Integration and Enhancement Vision Transformer for Self-Supervised Learning in Remote Sensing Kaixuan Lu Ruiqian Zhang Xiao Huang Yuxing Xie Xiaogang Ning Hanchao Zhang Mengke Yuan Pan Zhang Tao Wang Tongkui Liao 37 0 0 09 Nov 2024
TaxaBind: A Unified Embedding Space for Ecological Applications S. Sastry Subash Khanal A. Dhakal Adeel Ahmad Nathan Jacobs 65 7 0 01 Nov 2024
How Effective is Pre-training of Large Masked Autoencoders for Downstream Earth Observation Tasks? Jose Sosa Mohamed Aloulou Danila Rukhovich Rim Sleimi Boonyarit Changaival Anis Kacem Djamila Aouada 40 0 0 27 Sep 2024
Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model Zhuo Zheng Stefano Ermon Dongjun Kim Liangpei Zhang Yanfei Zhong DiffM 45 20 0 26 Jun 2024
HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model Di Wang Meiqi Hu Yao Jin Yuchun Miao Jiaqi Yang ... Lefei Zhang Chen Wu Bo Du Dacheng Tao Liangpei Zhang 66 25 0 17 Jun 2024
SeeFar: Satellite Agnostic Multi-Resolution Dataset for Geospatial Foundation Models James Lowman Kelly Liu Zheng Roydon Fraser J. The Mojtaba Valipour 34 0 0 10 Jun 2024
MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation Zhiping Yu Chenyang Liu Liqin Liu Z. Shi Zhengxia Zou VGen 29 11 0 22 May 2024
CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation Chenying Liu C. Albrecht Yi Wang Xiao Xiang Zhu 65 2 0 02 May 2024
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining Di Wang Jing Zhang Minqiang Xu Lin Liu Dongsheng Wang ... Chengxi Han Haonan Guo Bo Du Dacheng Tao L. Zhang 39 44 0 20 Mar 2024
CtxMIM: Context-Enhanced Masked Image Modeling for Remote Sensing Image Understanding Mingming Zhang Qingjie Liu Yunhong Wang 32 5 0 28 Sep 2023
Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning Colorado Reed Ritwik Gupta Shufan Li S. Brockman Christopher Funk Brian Clipp Kurt Keutzer Salvatore Candido M. Uyttendaele Trevor Darrell 121 169 0 30 Dec 2022
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training Renrui Zhang Ziyu Guo Rongyao Fang Bingyan Zhao Dong Wang Yu Qiao Hongsheng Li Peng Gao 3DPC 178 244 0 28 May 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 183 21,819 0 09 Dec 2016