NIPS Conversational Intelligence Challenge Call for Human Evaluators

Join ConvAI Competition as a Human Evaluator to make next generation chatbots truly intelligent

Dialogue systems and conversational agents – including chatbots, personal assistants and voice control interfaces – are becoming increasingly widespread in our daily lives. Nevertheless, conversational intelligence remains one of the major unsolved tasks in artificial intelligence. This year,  NIPS is sponsoring an open competition the Conversational Intelligence Challenge (http://convai.io) to create a chatbot that can hold an intelligent conversation with a human partner.  Goals of the competition are evaluation of state-of-the-art dialogue systems, and collection of an open-source dataset for future training of end-to-end systems.

“Learning to communicate with humans is a crucial ability for intelligent agents. Among the primary forms of communication between humans is natural language dialogue. As such, building systems that can naturally and meaningfully converse with humans has been a central goal of artificial intelligence since the formulation of the Turing test,” say co-organizers in their recent paper on Automated Turing test.

Current breakthroughs of deep learning critically depend on quantity and quality of data used to train models. This is why every human evaluator volunteer is so valuable for the success of the competition. Please join us in our efforts to build an open dataset for the development of the next generation of conversational AI.

First round of evaluation will be done remotely on 24th-30th of July, 2017 via a messaging platform.  During the week competition teams members  and human-volunteers will chat with bots and evaluate the quality of the dialogue systems, submitted  to the competition. Winning teams will be qualified for the NIPS finals. To contribute to the conversational intelligence research, fill the registration form at https://goo.gl/PWrZ0e now, and we will send you email at the beginning of the evaluation.

The competition is organised by Université de Montréal, McGill University, Carnegie Mellon University and Moscow Institute of Physics and Technology in partnership with Facebook, Flint Capital, IVADO, Maluuba and Element AI.

IVADO is a proud Silver Sponsor of ConvAI!

Conversational Intelligence Challenge Organizers

Mikhail Burtsev, Valentin Malykh, MIPT, Moscow
Ryan Lowe, McGill University, Montreal
Iulian Serban, Yoshua Bengio, University of Montreal, Montreal
Alexander Rudnicky, Alan W. Black, Shrimai Prabhumoye, Carnegie Mellon University, Pittsburgh