Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery

8 March 2024

Mubashir Noman

Muzammal Naseer

Hisham Cholakkal

Rao Muhammad Anwar

Salman Khan

Fahad Shahbaz Khan

ViT

ArXiv (abs)PDF HTML Github (106★)

Papers citing "Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery"

29 / 29 papers shown

Title
Scaling-Up the Pretraining of the Earth Observation Foundation Model PhilEO to the MajorTOM Dataset Nikolaos Dionelis Jente Bosmans Riccardo Musto Giancarlo Paoletti Simone Sarti Giacomo Cascarano Casper Fibaek Luke Camilleri B. L. Saux Nicolas Longépé 22 0 0 17 Jun 2025
Position Prediction Self-Supervised Learning for Multimodal Satellite Imagery Semantic Segmentation John Waithaka Moise Busogi SSL 17 0 0 07 Jun 2025
RemoteSAM: Towards Segment Anything for Earth Observation Liang Yao Fan Liu Delong Chen Chuanyi Zhang Yijun Wang Ziyun Chen Wei Xu Shimin Di Yuhui Zheng 239 0 0 23 May 2025
An Effective Training Framework for Light-Weight Automatic Speech Recognition Models Abdul Hannan Alessio Brutti Shah Nawaz Mubashir Noman 71 0 0 22 May 2025
REOBench: Benchmarking Robustness of Earth Observation Foundation Models Xiang Li Yong Tao Siyuan Zhang Siwei Liu Zhitong Xiong Chunbo Luo L. J. Liu Mykola Pechenizkiy Xiao Xiang Zhu T. Huang 69 0 0 22 May 2025
Parameter-Efficient Fine-Tuning of Multispectral Foundation Models for Hyperspectral Image Classification Bernardin Ligan Khalide Jbilou Fahd Kalloubi Ahmed Ratnani 201 0 0 21 May 2025
Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts Xi Chen Shen Yan Juelin Zhu Chen Chen Yu Liu Maojun Zhang 80 0 0 20 May 2025
CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework Wentao Wu Xinyu Wang Chenglong Li Bo Jiang Jin Tang Bin Luo Qi Liu 103 0 0 17 Apr 2025
Satellite Federated Fine-Tuning for Foundation Models in Space Computing Power Networks Yan zhu Jingyang Zhu Ting Wang Yuanming Shi Chunxiao Jiang Khaled B. Letaief 103 0 0 14 Apr 2025
REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval Shabnam Choudhury Yash Salunkhe Sarthak Mehrotra Biplab Banerjee 84 0 0 04 Apr 2025
A Survey on Remote Sensing Foundation Models: From Vision to Multimodality Ziyue Huang Hongxi Yan Qiqi Zhan Shuai Yang Mingming Zhang Yiming Lei Chenkai Zhang Zeming Liu Qingjie Liu Yansen Wang 149 2 0 28 Mar 2025
Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data Haozhe Si Yuxuan Wan Minh Do Deepak Vasisht Han Zhao Hendrik Hamann 175 0 0 17 Mar 2025
Towards a Unified Copernicus Foundation Model for Earth Vision Yi Wang Zhitong Xiong Chenying Liu Adam J. Stewart Thomas Dujardin ... Angelos Zavras Franziska Gerken Ioannis Papoutsis Laura Leal-Taixé Xiao Xiang Zhu 124 4 0 14 Mar 2025
Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation Leonard Waldmann Ando Shah Yi Wang Nils Lehmann Adam J. Stewart Zhitong Xiong Xiao Xiang Zhu Stefan Bauer John Chuang 74 4 0 13 Mar 2025
RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing Fengxiang Wang Hongru Wang Yansen Wang Di Wang Mingshuo Chen ... Yangang Sun Shuo Wang L. Lan Wenjing Yang Jing Zhang Mamba 122 3 0 13 Mar 2025
SAR-W-MixMAE: SAR Foundation Model Training Using Backscatter Power Weighting Ali Caglayan Nevrez Imamoglu T. Kouyama 160 0 0 03 Mar 2025
CARE: Confidence-Aware Regression Estimation of building density fine-tuning EO Foundation Models Nikolaos Dionelis Jente Bosmans Nicolas Longépé 226 0 0 19 Feb 2025
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision Diego A. Velázquez Pau Rodríguez López Sergio Alonso Josep M. Gonfaus Jordi Gonzalez Gerardo Richarte Javier Marin Yoshua Bengio Alexandre Lacoste 106 1 0 14 Jan 2025
Pattern Integration and Enhancement Vision Transformer for Self-Supervised Learning in Remote Sensing Kaixuan Lu Ruiqian Zhang Xiao Huang Yuxing Xie Xiaogang Ning Hanchao Zhang Mengke Yuan Pan Zhang Tao Wang Tongkui Liao 84 2 0 09 Nov 2024
TaxaBind: A Unified Embedding Space for Ecological Applications Srikumar Sastry Subash Khanal Aayush Dhakal Adeel Ahmad Nathan Jacobs 132 11 0 01 Nov 2024
How Effective is Pre-training of Large Masked Autoencoders for Downstream Earth Observation Tasks? Jose Sosa Mohamed Aloulou Danila Rukhovich Rim Sleimi Boonyarit Changaival Anis Kacem Djamila Aouada 75 1 0 27 Sep 2024
Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model Zhuo Zheng Stefano Ermon Dongjun Kim Liangpei Zhang Yanfei Zhong DiffM 87 20 0 26 Jun 2024
Harnessing Massive Satellite Imagery with Efficient Masked Image Modeling Fengxiang Wang H. Wang Di Wang Zonghao Guo Zhenyu Zhong Long Lan Wenjing Yang Jing Zhang 92 3 0 17 Jun 2024
HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model Di Wang Meiqi Hu Yao Jin Yuchun Miao Jiaqi Yang ... Lefei Zhang Chen Wu Di Lin Dacheng Tao Liangpei Zhang 164 27 0 17 Jun 2024
SeeFar: Satellite Agnostic Multi-Resolution Dataset for Geospatial Foundation Models James Lowman Kelly Liu Zheng Roydon Fraser J. The Mojtaba Valipour 66 0 0 10 Jun 2024
MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation Zhiping Yu Chenyang Liu Liqin Liu Z. Shi Zhengxia Zou VGen 73 16 0 22 May 2024
CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation Chenying Liu C. Albrecht Yi Wang Xiao Xiang Zhu 247 3 0 02 May 2024
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining Di Wang Jing Zhang Minqiang Xu Lin Liu Dongsheng Wang ... Chengxi Han Haonan Guo Bo Du Dacheng Tao Lefei Zhang 83 53 0 20 Mar 2024
CtxMIM: Context-Enhanced Masked Image Modeling for Remote Sensing Image Understanding Mingming Zhang Qingjie Liu Yunhong Wang 98 6 0 28 Sep 2023