SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

18 April 2021

Babak Damavandi

Papers citing "SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations"

44 / 44 papers shown

Title
Disambiguating Reference in Visually Grounded Dialogues through Joint Modeling of Textual and Multimodal Semantic Structures Shun Inadumi Nobuhiro Ueda Koichiro Yoshino ObjD 12 0 0 16 May 2025
PicPersona-TOD : A Dataset for Personalizing Utterance Style in Task-Oriented Dialogue with Image Persona Jihyun Lee Yejin Jeon Seungyeon Seo G. G. Lee MLLM 55 0 0 24 Apr 2025
Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach Xingyu Li Chen Gong Guohong Fu VGen 34 0 0 19 Apr 2025
Vision-Language Models Struggle to Align Entities across Modalities Iñigo Alonso Ander Salaberria Gorka Azkune Jeremy Barnes Oier López de Lacalle VLM 66 0 0 05 Mar 2025
Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you! Jiwan Chung Seungwon Lim Jaehyun Jeon Seungbeen Lee Youngjae Yu 37 0 0 01 Oct 2024
Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models Javier Chiyah-Garcia Alessandro Suglia Arash Eshghi KELM 40 1 0 21 Sep 2024
Multi-Modal Video Dialog State Tracking in the Wild Adnen Abdessaied Lei Shi Andreas Bulling 19 2 0 02 Jul 2024
Multimodal Contextualized Semantic Parsing from Speech Jordan Voas Raymond Mooney David Harwath 54 0 0 10 Jun 2024
J-CRe3: A Japanese Conversation Dataset for Real-world Reference Resolution Nobuhiro Ueda Hideko Habe Yoko Matsui Akishige Yuguchi Seiya Kawano Yasutomo Kawanishi Sadao Kurohashi Koichiro Yoshino 36 3 0 28 Mar 2024
MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations Hanlei Zhang Xin Wang Hua Xu Qianrui Zhou Kai Gao Jianhua Su jinyue Zhao Wenrui Li Yanting Chen 45 2 0 16 Mar 2024
OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog Adnen Abdessaied Manuel von Hochmeister Andreas Bulling 40 2 0 20 Feb 2024
TOAD: Task-Oriented Automatic Dialogs with Diverse Response Styles Yinhong Liu Yimai Fang David Vandyke Nigel Collier 44 3 0 15 Feb 2024
Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests Brielen Madureira David Schlangen 50 2 0 30 Jan 2024
Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition David M. Chan Shalini Ghosh Hitesh Tulsiani Ariya Rastrow Björn Hoffmeister 28 1 0 04 Jan 2024
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions Libo Qin Wenbo Pan Qiguang Chen Lizi Liao Zhou Yu Yue Zhang Wanxiang Che Min Li 34 11 0 15 Nov 2023
UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt Yucheng Cai Wentao Ma Yuchuan Wu Shuzheng Si Yuan Shao Zhijian Ou Yongbin Li 43 3 0 20 Sep 2023
VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue Yunshui Li Binyuan Hui Zhaochao Yin Wanwei He Run Luo Yuxing Long Min Yang Fei Huang Yongbin Li 26 1 0 14 Sep 2023
Collecting Visually-Grounded Dialogue with A Game Of Sorts Bram Willemsen Dmytro Kalpakchi Gabriel Skantze 13 2 0 10 Sep 2023
'What are you referring to?' Evaluating the Ability of Multi-Modal Dialogue Models to Process Clarificational Exchanges Javier Chiyah-Garcia Alessandro Suglia Arash Eshghi Helen F. Hastie 29 6 0 28 Jul 2023
SimpleMTOD: A Simple Language Model for Multimodal Task-Oriented Dialogue with Symbolic Scene Representation Bhathiya Hemanthage Christian Dondrup P. Bartie Oliver Lemon MLLM 24 1 0 10 Jul 2023
Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark Yuxing Long Binyuan Hui Caixia Yuan1 Fei Huang Yongbin Li Xiaojie Wang 29 4 0 26 May 2023
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts Yunshui Li Binyuan Hui Zhichao Yin Min Yang Fei Huang Yongbin Li MoE 35 19 0 24 May 2023
Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue Holy Lovenia Samuel Cahyawijaya Pascale Fung 24 1 0 28 Feb 2023
SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph Yuxing Long Binyuan Hui Fulong Ye Yanyang Li Zhuoxin Han Caixia Yuan Yongbin Li Xiaojie Wang LLMAG 38 7 0 05 Jan 2023
CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation Yinpei Dai Wanwei He Bowen Li Yuchuan Wu Zhen Cao Zhongqi An Jian Sun Yongbin Li ELM ALM 41 12 0 21 Nov 2022
Navigating Connected Memories with a Task-oriented Dialog System Seungwhan Moon Satwik Kottur A. Geramifard Babak Damavandi 35 2 0 15 Nov 2022
Tell Your Story: Task-Oriented Dialogs for Interactive Content Creation Satwik Kottur Seungwhan Moon Aram H. Markosyan Hardik Shah Babak Damavandi A. Geramifard 31 2 0 08 Nov 2022
Dialog Acts for Task-Driven Embodied Agents Spandana Gella Aishwarya Padmakumar P. Lange Dilek Z. Hakkani-Tür LM&Ro 30 16 0 26 Sep 2022
SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation Wanwei He Yinpei Dai Min Yang Jian Sun Fei Huang Luo Si Yongbin Li 33 60 0 14 Sep 2022
SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding Wanwei He Yinpei Dai Binyuan Hui Min Yang Zhen Cao Jianbo Dong Fei Huang Luo Si Yongbin Li VLM 43 31 0 14 Sep 2022
GRILLBot: An Assistant for Real-World Tasks with Neural Semantic Parsing and Graph-Based Representations Carlos Gemmell Iain Mackie Paul Owoicho Federico Rossetto Sophie Fischer Jeffrey Stephen Dalton LM&Ro GNN 29 0 0 31 Aug 2022
"Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking Léo Jacqmin L. Rojas-Barahona Benoit Favre 43 27 0 29 Jul 2022
Multimodal Dialogue State Tracking Hung Le Nancy F. Chen Guosheng Lin 30 9 0 16 Jun 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 45 29 0 13 May 2022
Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene Duo Zheng Fandong Meng Q. Si Hairun Fan Zipeng Xu Jie Zhou Fangxiang Feng Xiaojie Wang 27 0 0 16 Mar 2022
Exploring Multi-Modal Representations for Ambiguity Detection & Coreference Resolution in the SIMMC 2.0 Challenge Javier Chiyah-Garcia Alessandro Suglia José Lopes Arash Eshghi Helen F. Hastie 27 8 0 25 Feb 2022
Database Search Results Disambiguation for Task-Oriented Dialog Systems Kun Qian Ahmad Beirami Satwik Kottur Shahin Shayandeh Paul A. Crook A. Geramifard Zhou Yu Chinnadhurai Sankar 35 15 0 15 Dec 2021
Multimodal Interactions Using Pretrained Unimodal Models for SIMMC 2.0 Joosung Lee Kijong Han 45 6 0 10 Dec 2021
UNITER-Based Situated Coreference Resolution with Rich Multimodal Input Yichen Huang Yuchen Wang Yik-Cheung Tam 38 8 0 07 Dec 2021
Building Goal-Oriented Dialogue Systems with Situated Visual Context Sanchit Agarwal Jan Jezabek Arijit Biswas Emre Barut Shuyang Gao Tagyoung Chung 23 1 0 22 Nov 2021
Few-Shot Bot: Prompt-Based Learning for Dialogue Systems Andrea Madotto Zhaojiang Lin Genta Indra Winata Pascale Fung 48 81 0 15 Oct 2021
The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service Nan Zhao Haoran Li Youzheng Wu Xiaodong He Bowen Zhou 27 8 0 27 Sep 2021
BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling Zhaojiang Lin Andrea Madotto Genta Indra Winata Peng Xu Feijun Jiang Yuxiang Hu Chen Shi Pascale Fung 29 61 0 05 Jun 2021
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue Hung Le Chinnadhurai Sankar Seungwhan Moon Ahmad Beirami A. Geramifard Satwik Kottur VGen 39 18 0 01 Jan 2021