Multimodal Sensor Fusion with Differentiable Filters

25 October 2020

Michelle A. Lee

Brent Yi

Roberto Martín-Martín

Silvio Savarese

Jeannette Bohg

ArXiv PDF HTML

Papers citing "Multimodal Sensor Fusion with Differentiable Filters"

37 / 37 papers shown

Title
DnD Filter: Differentiable State Estimation for Dynamic Systems using Diffusion Models Ziyu Wan Lin Zhao DiffM 71 0 0 03 Mar 2025
Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions Juan Del Aguila Ferrandis João Moura S. Vijayakumar 73 4 0 17 Dec 2024
Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors Bang You Huaping Liu SSL 25 5 0 23 Oct 2024
Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction Justin Kerr C. Kim Mingxuan Wu Brent Yi Qianqian Wang Ken Goldberg Angjoo Kanazawa 23 13 0 26 Sep 2024
Towards Interpretable Visuo-Tactile Predictive Models for Soft Robot Interactions Enrico Donato T. G. Thuruthel Egidio Falotico 23 1 0 16 Jul 2024
The Artificial Neural Twin -- Process Optimization and Continual Learning in Distributed Process Chains Johannes Emmert Ronald Mendez Houman Mirzaalian Dastjerdi Christopher Syben Andreas K. Maier AI4CE 18 0 0 27 Mar 2024
iRoCo: Intuitive Robot Control From Anywhere Using a Smartwatch F. Weigend Xiao Liu Shubham D. Sonawani Neelesh Kumar Venu Vasudevan H. B. Amor 27 3 0 11 Mar 2024
In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking Peng Gao Shi-Min Li Feng Gao Fei-Yue Wang Ruyue Yuan Hamido Fujita 27 8 0 27 Feb 2024
Learning active tactile perception through belief-space control J. Tremblay D. Meger F. Hogan Gregory Dudek 37 1 0 30 Nov 2023
Multimodal Learning of Soft Robot Dynamics using Differentiable Filters Xiao Liu Yifan Zhou Shuhei Ikemoto H. B. Amor 8 0 0 12 Nov 2023
MultiIoT: Benchmarking Machine Learning for the Internet of Things Shentong Mo Louis-Philippe Morency Russ Salakhutdinov Paul Pu Liang 24 1 0 10 Nov 2023
Streaming Motion Forecasting for Autonomous Driving Ziqi Pang Deva Ramanan Mengtian Li Yu-xiong Wang AI4TS 26 6 0 02 Oct 2023
RegBN: Batch Normalization of Multimodal Data with Regularization Morteza Ghahremani Christian Wachinger 28 6 0 01 Oct 2023
Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filter Hsu-kuang Chiu Chien-Yi Wang Min-Hung Chen Stephen F. Smith 11 3 0 26 Sep 2023
Learning Covariances for Estimation with Constrained Bilevel Optimization Mohamad Qadri Zachary Manchester Michael Kaess 24 4 0 18 Sep 2023
Learning Vision-based Pursuit-Evasion Robot Policies Andrea V. Bajcsy Antonio Loquercio Ashish Kumar Jitendra Malik 27 10 0 30 Aug 2023
Enhancing State Estimation in Robots: A Data-Driven Approach with Differentiable Ensemble Kalman Filters Xinyu Liu Geoffrey Clark Joseph Campbell Yifan Zhou H. B. Amor 23 10 0 19 Aug 2023
Learning Soft Robot Dynamics using Differentiable Kalman Filters and Spatio-Temporal Embeddings Xinyu Liu Shuhei Ikemoto Yuhei Yoshimitsu H. B. Amor 38 5 0 19 Aug 2023
MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning Paul Pu Liang Yiwei Lyu Xiang Fan Arav Agarwal Yun Cheng Louis-Philippe Morency Ruslan Salakhutdinov VLM 31 6 0 28 Jun 2023
Online Estimation of Self-Body Deflection With Various Sensor Data Based on Directional Statistics Hiroya Sato Kento Kawaharazuka Tasuku Makabe K. Okada Masayuki Inaba 14 0 0 06 Jun 2023
DNBP: Differentiable Nonparametric Belief Propagation Anthony Opipari Jana Pavlasek Chao Chen Shoutian Wang Karthik Desingh Odest Chadwicke Jenkins 6 3 0 08 Mar 2023
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework Paul Pu Liang Yun Cheng Xiang Fan Chun Kai Ling Suzanne Nie ... Nicholas B. Allen Randy P. Auerbach Faisal Mahmood Ruslan Salakhutdinov Louis-Philippe Morency 30 29 0 23 Feb 2023
An overview of differentiable particle filters for data-adaptive sequential Bayesian inference Xiongjie Chen Yunpeng Li 28 14 0 19 Feb 2023
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation Hao Li Yizhi Zhang Junzhe Zhu Shaoxiong Wang Michelle A. Lee Huazhe Xu Edward H. Adelson Li Fei-Fei Ruohan Gao Jiajun Wu 24 58 0 07 Dec 2022
Multimodal Learning for Multi-Omics: A Survey Sina Tabakhi M. N. I. Suvon Pegah Ahadian Haiping Lu 15 9 0 29 Nov 2022
Category-Independent Articulated Object Tracking with Factor Graphs Nick Heppert Toki Migimatsu Brent Yi Claire Chen Jeannette Bohg 27 22 0 07 May 2022
Learning Sequential Latent Variable Models from Multimodal Time Series Data Oliver Limoyo Trevor Ablett Jonathan Kelly 17 5 0 21 Apr 2022
Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness Paul Pu Liang 22 4 0 14 Apr 2022
Symbolic State Estimation with Predicates for Contact-Rich Manipulation Tasks Toki Migimatsu Wenzhao Lian Jeannette Bohg S. Schaal 24 10 0 04 Mar 2022
High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Jeffrey Tsaw Yudong Liu Shentong Mo Dani Yogatama Louis-Philippe Morency Ruslan Salakhutdinov 17 29 0 02 Mar 2022
Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning Piaopiao Jin Yi-Bin Lin Yanchao Tan Tiefeng Li Wei Yang 16 1 0 17 Feb 2022
LEO: Learning Energy-based Models in Factor Graph Optimization Paloma Sodhi Eric Dexheimer Mustafa Mukadam Stuart Anderson Michael Kaess 37 16 0 04 Aug 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 29 158 0 15 Jul 2021
Differentiable Factor Graph Optimization for Learning Smoothers Brent Yi Michelle A. Lee Alina Kloss Roberto Martín-Martín Jeannette Bohg 11 28 0 18 May 2021
Neural Motion Prediction for In-flight Uneven Object Catching Hongxiang Yu Dashun Guo Huan Yin An-Jen Chen Kechun Xu Yue Wang R. Xiong 11 10 0 15 Mar 2021
Differentiable Nonparametric Belief Propagation Anthony Opipari Chao Chen Shoutian Wang Jana Pavlasek Karthik Desingh Odest Chadwicke Jenkins 11 5 0 15 Jan 2021
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors Michelle A. Lee Matthew Tan Yuke Zhu Jeannette Bohg 38 25 0 01 Dec 2020