Simplicity is Key: An Unsupervised Pretraining Approach for Sparse Radio Channels

v1v2 (latest)

Simplicity is Key: An Unsupervised Pretraining Approach for Sparse Radio Channels

19 May 2025

Maximilian Stahlke

Bjoern M. Eskofier

Christopher Mutschler

ArXiv (abs)PDF HTML

Papers citing "Simplicity is Key: An Unsupervised Pretraining Approach for Sparse Radio Channels"

15 / 15 papers shown

Title
Radio Foundation Models: Pre-training Transformers for 5G-based Indoor Localization Jonathan Ott Jonas Pirkl Maximilian Stahlke Tobias Feigl Christopher Mutschler 23 7 0 01 Oct 2024
Improving Dictionary Learning with Gated Sparse Autoencoders Senthooran Rajamanoharan Arthur Conmy Lewis Smith Tom Lieberum Vikrant Varma János Kramár Rohin Shah Neel Nanda RALM 58 93 0 24 Apr 2024
Sparse Autoencoders Find Highly Interpretable Features in Language Models Hoagy Cunningham Aidan Ewart Logan Riggs R. Huben Lee Sharkey MILM 115 447 0 15 Sep 2023
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions S. Bhattamishra Arkil Patel Varun Kanade Phil Blunsom 91 49 0 22 Nov 2022
Indoor Localization with Robust Global Channel Charting: A Time-Distance-Based Approach Maximilian Stahlke G. Yammine Tobias Feigl Bjoern M. Eskofier Christopher Mutschler 65 28 0 07 Oct 2022
CSI-fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network Bowen Zhang Houssem Sifaou Geoffrey Ye Li HAI 46 37 0 11 May 2022
Attention Aided CSI Wireless Localization Artan Salihu Stefan Schwarz M. Rupp 35 7 0 20 Mar 2022
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 724 6,135 0 29 Apr 2021
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 398 6,844 0 13 Jun 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 387 18,897 0 13 Feb 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 213 12,136 0 13 Nov 2019
Towards Practical Indoor Positioning Based on Massive MIMO Systems Mark S. Widmaier Maximilian Arnold Sebastian Dörner Sebastian Cammerer S. Brink 20 27 0 28 May 2019
Channel Charting: Locating Users within the Radio Environment using Channel State Information Christoph Studer Said Medjkouh Emre Gönültaş Tom Goldstein O. Tirkkonen 38 161 0 13 Jul 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 795 132,454 0 12 Jun 2017
Language Modeling with Gated Convolutional Networks Yann N. Dauphin Angela Fan Michael Auli David Grangier 245 2,408 0 23 Dec 2016