Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series

Accurate intraday solar irradiance forecasting is crucial for optimizing dispatch planning and electricity trading. For this purpose, we introduce a novel and effective approach that includes three distinguishing components from the literature: 1) the uncommon use of single-frame public camera imagery; 2) solar irradiance time series scaled with a proposed normalization step, which boosts performance; and 3) a lightweight multimodal model, called Solar Multimodal Transformer (SMT), that delivers accurate short-term solar irradiance forecasting by combining images and scaled time series. Benchmarking against Solcast, a leading solar forecasting service provider, our model improved prediction accuracy by 25.95%. Our approach allows for easy adaptation to various camera specifications, offering broad applicability for real-world solar forecasting challenges.

@article{niu2025_2503.00250,
  title={ Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series },
  author={ Yanan Niu and Roy Sarkis and Demetri Psaltis and Mario Paolone and Christophe Moser and Luisa Lambertini },
  journal={arXiv preprint arXiv:2503.00250},
  year={ 2025 }
}