Past Challenges

Task: Dialog state tracking on human-computer dialogs in the restaurant information domain

Organizers: Matthew Henderson, Blaise Thomson, Jason D. Williams

Venue: Special Session @ SIGDIAL 2014

Website & Datasets, Overview Paper

Task: Domain adaptation for human-computer dialog state tracking from the restaurant to the tourist information domain

Organizers: Matthew Henderson, Blaise Thomson, Jason D. Williams

Venue: Special Session @ IEEE SLT 2014

Website & Datasets, Overview Paper

Task: Dialog state tracking on human-human dialogs in the tourist information domain

Organizers: Seokhwan Kim, Luis Fernando D’Haro, Rafael E. Banchs, Matthew Henderson, Jason D. Williams

Venue: Special Session @ IWSDS 2016

Website, Datasets, Overview Paper

Task: Cross-language dialog state tracking on human-human dialogs in the tourist information domain

Organizers: Seokhwan Kim, Luis Fernando D’Haro, Rafael E. Banchs, Matthew Henderson, Jason D. Williams, Koichiro Yoshino

Venue: Special Session @ IEEE SLT 2016

Website, Datasets, Overview Paper

Three Parallel Tracks

• End-to-End Goal Oriented Dialog Learning: Task Description, Resources, Overview Paper

• End-to-End Conversation Modeling: Task Description, Resources, Overview Paper

• Dialogue Breakdown Detection: Task Description, Resources, Track Website, Overview Paper

Challenge Organizers: Chiori Hori, Julien Perez, Koichiro Yoshino, Seokhwan Kim

Venue: DSTC6 Workshop

Challenge Website

Three Parallel Tracks

• Noetic End-to-End Response Selection: Task Description, Resources, Overview Paper

• Grounded Response Generation: Task Description, Resources, Overview Paper

• Audio Visual Scene-aware Dialog: Task Description, Resources, Overview Paper

Challenge Organizers: Koichiro Yoshino, Chiori Hori, Julien Perez, Luis Fernando D'Haro

Venue: DSTC7 Workshop @ AAAI-19

Challenge Website

Four Parallel Tracks

• Multi-domain Task Completion: Task Description, Resources, Overview Paper

• NOESIS II: Predicting Responses, Identifying Success, and Managing Complexity in Task-Oriented Dialogue: Task Description, Resources, Overview Paper

• Audio Visual Scene-aware Dialog: Task Description, Resources, Overview Paper

• Schema-Guided Dialogue State Tracking: Task Description, Resources, Overview Paper

Challenge Organizers: Seokhwan Kim, Michel Galley, Chulaka Gunasekara, Sungjin Lee

Venue: DSTC8 Workshop @ AAAI-20

Challenge Website

Four Parallel Tracks

• Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access: Task Description, Resources, Overview Paper

• Multi-domain Task-oriented Dialog Challenge II: Task Description, Resources, Overview Paper

• Interactive Evaluation of Dialog: Task Description, Resources, Overview Paper

• SIMMC: Situated Interactive Multi-Modal Conversational AI: Task Description, Resources, Overview Paper

Challenge Organizers: Yun-Nung (Vivian) Chen, Chulaka Gunasekara, Luis Fernando D'Haro, Seokhwan Kim, Abhinav Rastogi 

Venue: DSTC9 Workshop @ AAAI-21

Challenge Website

Five Parallel Tracks

• MOD: Internet Meme Incorporated Open-domain Dialog: Task Description, Resources, Overview Paper

• Knowledge-grounded Task-oriented Dialogue Modeling on Spoken Conversations: Task Description, Resources, Overview Paper

• SIMMC 2.0: Situated Interactive Multimodal Conversational AI: Task Description, Resources, Overview Paper

• Reasoning for Audio Visual Scene-Aware Dialog: Task Description, Resources, Overview Paper

• Automatic Evaluation and Moderation of Open-domain Dialogue Systems: Task Description, Resources, Overview Paper

Challenge Organizers: Koichiro Yoshino, Yun-Nung (Vivian) Chen, Paul Crook, Satwik Kottur, Jinchao Li, Behnam Hedayatnia, Seungwhan (Shane) Moon 

Venue: DSTC10 Workshop @ AAAI-22

Challenge Website

Challenge Website