ResearchTrend.AI
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers

Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers

Vidhi Jain
Maria Attarian
Nikhil J. Joshi
Ayzaan Wahid
Danny Driess
Quan Vuong
Pannag R Sanketi
P. Sermanet
Stefan Welker
Christine Chan
Igor Gilitschenski
Yonatan Bisk
Debidatta Dwibedi
ArXiv (abs)PDFHTML

Papers citing "Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers"

37 / 37 papers shown
Title
Learning Generalizable Robot Policy with Human Demonstration Video as a Prompt
Learning Generalizable Robot Policy with Human Demonstration Video as a Prompt
Xiang Zhu
Yichen Liu
Hezhong Li
Jianyu Chen
101
0
0
27 May 2025
X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
Prithwish Dan
Kushal Kedia
Angela Chao
Edward Weiyi Duan
Maximus Adrian Pace
Wei-Chiu Ma
Sanjiban Choudhury
87
0
0
11 May 2025
ViSA-Flow: Accelerating Robot Skill Learning via Large-Scale Video Semantic Action Flow
ViSA-Flow: Accelerating Robot Skill Learning via Large-Scale Video Semantic Action Flow
Changhe Chen
Quantao Yang
Xiaohao Xu
Nima Fazeli
Olov Andersson
91
0
0
02 May 2025
What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning
What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning
Chi-Hsi Kung
Frangil Ramirez
Juhyung Ha
Yi-Ting Chen
David J. Crandall
Yi-Hsuan Tsai
107
1
0
27 Mar 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos
Sicheng Xie
Haidong Cao
Zejia Weng
Zhen Xing
Shiwei Shen
Jiaqi Leng
Xipeng Qiu
Yanwei Fu
Zuxuan Wu
Yu Jiang
123
0
0
23 Feb 2025
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
Jiange Yang
Haoyi Zhu
Yanjie Wang
Gangshan Wu
Tong He
Limin Wang
174
3
0
21 Nov 2024
One-Shot Imitation under Mismatched Execution
One-Shot Imitation under Mismatched Execution
Kushal Kedia
Prithwish Dan
Sanjiban Choudhury
Maximus Adrian Pace
Sanjiban Choudhury
111
5
0
10 Sep 2024
R+X: Retrieval and Execution from Everyday Human Videos
R+X: Retrieval and Execution from Everyday Human Videos
Georgios Papagiannis
Norman Di Palo
Pietro Vitiello
Edward Johns
127
17
0
17 Jul 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation
Jiaming Zhou
Teli Ma
Kun-Yu Lin
Ronghe Qiu
Zifan Wang
Junwei Liang
120
7
0
20 Jun 2024
Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search
Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search
Max Liu
Chan-Hung Yu
Wei-Hsu Lee
Cheng-Wei Hung
Yen-Chun Chen
Shao-Hua Sun
97
5
0
26 May 2024
VIEW: Visual Imitation Learning with Waypoints
VIEW: Visual Imitation Learning with Waypoints
Ananth Jonnavittula
Sagar Parekh
Dylan P. Losey
SSL
145
11
0
27 Apr 2024
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot: Open-Vocabulary Mobile Manipulation
Sriram Yenamandra
A. Ramachandran
Karmesh Yadav
Austin S. Wang
Mukul Khanna
...
Devendra Singh Chaplot
Dhruv Batra
Roozbeh Mottaghi
Yonatan Bisk
Chris Paxton
LM&Ro
110
83
0
20 Jun 2023
Sigmoid Loss for Language Image Pre-Training
Sigmoid Loss for Language Image Pre-Training
Xiaohua Zhai
Basil Mustafa
Alexander Kolesnikov
Lucas Beyer
CLIPVLM
232
1,200
0
27 Mar 2023
Open-World Object Manipulation using Pre-trained Vision-Language Models
Open-World Object Manipulation using Pre-trained Vision-Language Models
Austin Stone
Ted Xiao
Yao Lu
K. Gopalakrishnan
Kuang-Huei Lee
...
Sean Kirmani
Brianna Zitkovich
F. Xia
Chelsea Finn
Karol Hausman
LM&Ro
249
155
0
02 Mar 2023
VIMA: General Robot Manipulation with Multimodal Prompts
VIMA: General Robot Manipulation with Multimodal Prompts
Yunfan Jiang
Agrim Gupta
Zichen Zhang
Guanzhi Wang
Yongqiang Dou
Yanjun Chen
Li Fei-Fei
Anima Anandkumar
Yuke Zhu
Linxi Fan
LM&Ro
101
355
0
06 Oct 2022
Human-to-Robot Imitation in the Wild
Human-to-Robot Imitation in the Wild
Shikhar Bahl
Abhi Gupta
Deepak Pathak
97
173
0
19 Jul 2022
Flamingo: a Visual Language Model for Few-Shot Learning
Flamingo: a Visual Language Model for Few-Shot Learning
Jean-Baptiste Alayrac
Jeff Donahue
Pauline Luc
Antoine Miech
Iain Barr
...
Mikolaj Binkowski
Ricardo Barreira
Oriol Vinyals
Andrew Zisserman
Karen Simonyan
MLLMVLM
418
3,602
0
29 Apr 2022
R3M: A Universal Visual Representation for Robot Manipulation
R3M: A Universal Visual Representation for Robot Manipulation
Suraj Nair
Aravind Rajeswaran
Vikash Kumar
Chelsea Finn
Abhi Gupta
LM&Ro
101
584
0
23 Mar 2022
Masked Visual Pre-training for Motor Control
Masked Visual Pre-training for Motor Control
Tete Xiao
Ilija Radosavovic
Trevor Darrell
Jitendra Malik
SSL
105
250
0
11 Mar 2022
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Eric Jang
A. Irpan
Mohi Khansari
Daniel Kappler
F. Ebert
Corey Lynch
Sergey Levine
Chelsea Finn
LM&Ro
263
549
0
04 Feb 2022
Towards More Generalizable One-shot Visual Imitation Learning
Towards More Generalizable One-shot Visual Imitation Learning
Zhao Mandi
Fangchen Liu
Kimin Lee
Pieter Abbeel
66
61
0
26 Oct 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs
Perceiver IO: A General Architecture for Structured Inputs & Outputs
Andrew Jaegle
Sebastian Borgeaud
Jean-Baptiste Alayrac
Carl Doersch
Catalin Ionescu
...
Olivier J. Hénaff
M. Botvinick
Andrew Zisserman
Oriol Vinyals
João Carreira
MLLMVLMGNN
82
584
0
30 Jul 2021
Goal-Conditioned Reinforcement Learning with Imagined Subgoals
Goal-Conditioned Reinforcement Learning with Imagined Subgoals
Elliot Chane-Sane
Cordelia Schmid
Ivan Laptev
77
144
0
01 Jul 2021
XIRL: Cross-embodiment Inverse Reinforcement Learning
XIRL: Cross-embodiment Inverse Reinforcement Learning
Kevin Zakka
Andy Zeng
Peter R. Florence
Jonathan Tompson
Jeannette Bohg
Debidatta Dwibedi
SSL
85
122
0
07 Jun 2021
Forecasting Action through Contact Representations from First Person Video
Forecasting Action through Contact Representations from First Person Video
Eadom Dessalene
Chinmaya Devaraj
Michael Maynord
Cornelia Fermuller
Yiannis Aloimonos
EgoV
88
60
0
01 Feb 2021
Learning Object Manipulation Skills via Approximate State Estimation from Real Videos
Learning Object Manipulation Skills via Approximate State Estimation from Real Videos
Vladimir Petrik
Makarand Tapaswi
Ivan Laptev
Josef Sivic
50
23
0
13 Nov 2020
Reinforcement Learning with Videos: Combining Offline Observations with Interaction
Reinforcement Learning with Videos: Combining Offline Observations with Interaction
Karl Schmeckpeper
Oleh Rybkin
Kostas Daniilidis
Sergey Levine
Chelsea Finn
OffRL
72
107
0
12 Nov 2020
Transformers for One-Shot Visual Imitation
Transformers for One-Shot Visual Imitation
Sudeep Dasari
Abhinav Gupta
LM&Ro
86
94
0
11 Nov 2020
Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
Oliver Groth
Chia-Man Hung
Andrea Vedaldi
Ingmar Posner
OCL
62
10
0
19 Mar 2020
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
Laura M. Smith
Nikita Dhawan
Marvin Zhang
Pieter Abbeel
Sergey Levine
138
160
0
10 Dec 2019
Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
Pratyusha Sharma
Deepak Pathak
Abhinav Gupta
SSL
77
120
0
21 Nov 2019
Goal-conditioned Imitation Learning
Goal-conditioned Imitation Learning
Yiming Ding
Carlos Florensa
Mariano Phielipp
Pieter Abbeel
64
227
0
13 Jun 2019
Online Object Representations with Contrastive Learning
Online Object Representations with Contrastive Learning
Soren Pirk
Mohi Khansari
Yunfei Bai
Corey Lynch
P. Sermanet
SSL
58
35
0
10 Jun 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Antoine Miech
Dimitri Zhukov
Jean-Baptiste Alayrac
Makarand Tapaswi
Ivan Laptev
Josef Sivic
VGen
118
1,207
0
07 Jun 2019
FiLM: Visual Reasoning with a General Conditioning Layer
FiLM: Visual Reasoning with a General Conditioning Layer
Ethan Perez
Florian Strub
H. D. Vries
Vincent Dumoulin
Aaron Courville
FAttAIMatOffRLAI4CE
356
2,230
0
22 Sep 2017
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
YuXuan Liu
Abhishek Gupta
Pieter Abbeel
Sergey Levine
111
380
0
11 Jul 2017
One-Shot Imitation Learning
One-Shot Imitation Learning
Yan Duan
Marcin Andrychowicz
Bradly C. Stadie
Jonathan Ho
Jonas Schneider
Ilya Sutskever
Pieter Abbeel
Wojciech Zaremba
OffRL
82
688
0
21 Mar 2017

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.