Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves

Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves

2 March 2023

Hirokatsu Kataoka

Papers citing "Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves"

16 / 16 papers shown

Title
VibrantLeaves: A principled parametric image generator for training deep restoration models Raphaël Achddou Y. Gousseau Saïd Ladjal Sabine Süsstrunk 28 0 0 14 Apr 2025
MoireDB: Formula-generated Interference-fringe Image Dataset Yuto Matsuo Ryo Hayamizu Hirokatsu Kataoka Akio Nakamura 37 0 0 03 Feb 2025
Pre-training with Synthetic Patterns for Audio Yuchi Ishikawa Tatsuya Komatsu Yoshimitsu Aoki 35 0 0 01 Oct 2024
Formula-Supervised Visual-Geometric Pre-training Ryosuke Yamada Kensho Hara Hirokatsu Kataoka Koshi Makihara Nakamasa Inoue Rio Yokota Y. Satoh 23 1 0 20 Sep 2024
Detect Fake with Fake: Leveraging Synthetic Data-driven Representation for Synthetic Image Detection Hina Otake Yoshihiro Fukuhara Yoshiki Kubotani Shigeo Morishima ViT 56 0 0 13 Sep 2024
Data Collection-free Masked Video Modeling Yuchi Ishikawa Masayoshi Kondo Yoshimitsu Aoki ViT 19 1 0 10 Sep 2024
Scaling Backwards: Minimal Synthetic Pre-training? Ryo Nakamura Ryu Tadokoro Ryosuke Yamada Tim Puhlfürß Iro Laina Christian Rupprecht Walid Maalej Rio Yokota Hirokatsu Kataoka DD 19 2 0 01 Aug 2024
FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis Linshan Wu Jiaxin Zhuang Xuefeng Ni Hao Chen MedIm 36 12 0 03 Jun 2024
A Vision Check-up for Language Models Pratyusha Sharma Tamar Rott Shaham Manel Baradad Stephanie Fu Adrian Rodriguez-Munoz Shivam Duggal Phillip Isola Antonio Torralba VLM LRM 78 24 0 03 Jan 2024
FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models Lihe Yang Xiaogang Xu Bingyi Kang Yinghuan Shi Hengshuang Zhao 30 45 0 23 Oct 2023
Toward a Deeper Understanding: RetNet Viewed through Convolution Chenghao Li Chaoning Zhang ViT 35 7 0 11 Sep 2023
Fake it till you make it: Learning transferable representations from synthetic ImageNet clones Mert Bulent Sariyildiz Alahari Karteek Diane Larlus Yannis Kalantidis DiffM VLM 30 153 0 16 Dec 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,434 0 11 Nov 2021
Improving Fractal Pre-training Connor Anderson Ryan Farrell 88 27 0 06 Oct 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,775 0 29 Apr 2021
Pre-training without Natural Images Hirokatsu Kataoka Kazushige Okayasu Asato Matsumoto Eisuke Yamagata Ryosuke Yamada Nakamasa Inoue Akio Nakamura Y. Satoh 79 116 0 21 Jan 2021