{"id":8101,"date":"2017-11-24T22:16:45","date_gmt":"2017-11-25T03:16:45","guid":{"rendered":"https:\/\/vieux.ivado.ca\/?p=8101"},"modified":"2020-02-27T09:16:36","modified_gmt":"2020-02-27T14:16:36","slug":"the-conversational-intelligence-challenge","status":"publish","type":"post","link":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/","title":{"rendered":"The Conversational Intelligence Challenge"},"content":{"rendered":"

Call for human evaluators to enter pre-NIPS human evaluation round<\/b><\/p>\n

Donate your dialogue to chatbot research<\/b><\/p>\n

We\u00a0experience conversational artificial intelligence increasingly often, from bots that we can chat with on messaging apps to personal assistants and voice control interfaces. While dialogue systems are becoming more widespread, there is still a great deal of work involved in making conversational intelligence more sophisticated.<\/span><\/p>\n

To\u00a0create dialogue systems, we need conversations. So this year, the Conference on Neural Information Processing Systems\u00a0(NIPS) <\/span>is sponsoring<\/span><\/a> an open competition called <\/span>The Conversational Intelligence Challenge<\/span><\/a> aimed at creating a chatbot that can hold an intelligent conversation with a human partner. The\u00a0event will produce evaluations of state-of-the-art dialogue systems and an open-source data set for the future training of end-to-end systems.<\/span><\/p>\n

Traditionally, chatbot technology was developed using sets of conversational rules, which were fine-tuned manually. Now, given the availability of large collections of real human conversations on the web, it is becoming increasingly feasible to generate rules automatically. However, the results depend on the quantity and quality of the data used to train models. This is why every volunteer human evaluator is critical for the success of the competition.<\/span><\/p>\n

\u201cWithin the first round of human evaluation, we collected 2,500 dialogues. From this data we learned that people get involved less in a conversation with a bot, compared to chatting with a person. When two people chat, they often exchange short messages, which they can interpret correctly. With bots, the vocabulary is not rich enough, and they still lack understanding,\u201d says <\/span>Mikhail Burtsev<\/b>, one of the challenge organizers and the head of the Laboratory of Neural Networks and Deep Learning at the Moscow Institute of Physics and Technology\u00a0(<\/span>MIPT<\/span><\/a>). \u201cWe see that dialogue quality evaluations do not depend exclusively on whether bots understand messages correctly and respond adequately. There are other parameters at play. To\u00a0improve the ability of bots to maintain meaningful conversation, we need to know what other factors are involved and how to measure them. Our aim in the pre-NIPS round of evaluation is to get at least 3,000 more dialogues to draw statistically sound conclusions and improve the tools for creating bots.\u201d<\/span><\/p>\n

The\u00a0pre-NIPS human evaluation round will be held remotely from\u00a0Nov.\u00a020 to Dec.\u00a08, 2017, via the @ConvaiBot on messaging platforms. Starting with these two weeks before the NIPS conference, teams and volunteers will chat with bots and perform quality evaluation of the teams\u2019 solutions, which will then be adjusted over the tuning round. The\u00a0final ratings of the submissions will be presented Dec.\u00a08, during the competition session at NIPS. Please join our effort to create an open data set for the development of the next generation of conversational AI solutions and donate your dialogue via the @ConvaiBot on <\/span>Telegram<\/span><\/a> or <\/span>Facebook Messenger<\/span><\/a>.<\/span><\/p>\n

Six teams from universities and companies from around the world are taking part in the competition finals: the University of Wroclaw, MIPT, McGill University, KAIST, AIBrain, Crosscert, UMass Lowell\u2019s Text Machine Lab, Trinity College, the Hong Kong Polytechnic University, and Fudan University.<\/span><\/p>\n

The competition is organized by MIPT, Universit\u00e9 de Montr\u00e9al, McGill University, and Carnegie Mellon University in partnership with Facebook, Flint Capital, IVADO, Maluuba, and Element\u00a0AI.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Call for human evaluators to enter pre-NIPS human evaluation round Donate your dialogue to chatbot research We\u00a0experience conversational artificial intelligence increasingly often, from bots that we can chat with on… <\/p>\n

<\/div>\n

Read More<\/a><\/p>\n","protected":false},"author":14,"featured_media":8102,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[76],"tags":[],"acf":[],"yoast_head":"\nThe Conversational Intelligence Challenge - IVADO<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Conversational Intelligence Challenge - IVADO\" \/>\n<meta property=\"og:description\" content=\"Call for human evaluators to enter pre-NIPS human evaluation round Donate your dialogue to chatbot research We\u00a0experience conversational artificial intelligence increasingly often, from bots that we can chat with on... Read More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/\" \/>\n<meta property=\"og:site_name\" content=\"IVADO\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-25T03:16:45+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-02-27T14:16:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png\" \/>\n\t<meta property=\"og:image:width\" content=\"842\" \/>\n\t<meta property=\"og:image:height\" content=\"1191\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Sandra Estrela\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sandra Estrela\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/\",\"url\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/\",\"name\":\"The Conversational Intelligence Challenge - IVADO\",\"isPartOf\":{\"@id\":\"https:\/\/vieux.ivado.ca\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png\",\"datePublished\":\"2017-11-25T03:16:45+00:00\",\"dateModified\":\"2020-02-27T14:16:36+00:00\",\"author\":{\"@id\":\"https:\/\/vieux.ivado.ca\/#\/schema\/person\/6aa152aa0e33e45ec25475d10dc22fd0\"},\"breadcrumb\":{\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage\",\"url\":\"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png\",\"contentUrl\":\"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png\",\"width\":842,\"height\":1191},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/vieux.ivado.ca\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Conversational Intelligence Challenge\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/vieux.ivado.ca\/#website\",\"url\":\"https:\/\/vieux.ivado.ca\/\",\"name\":\"IVADO\",\"description\":\"Institut de valorisation des donn\u00e9es\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/vieux.ivado.ca\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/vieux.ivado.ca\/#\/schema\/person\/6aa152aa0e33e45ec25475d10dc22fd0\",\"name\":\"Sandra Estrela\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vieux.ivado.ca\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1cc42a053bcbd01458777f6d1263133f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1cc42a053bcbd01458777f6d1263133f?s=96&d=mm&r=g\",\"caption\":\"Sandra Estrela\"},\"url\":\"https:\/\/vieux.ivado.ca\/en\/author\/sestrela\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Conversational Intelligence Challenge - IVADO","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"The Conversational Intelligence Challenge - IVADO","og_description":"Call for human evaluators to enter pre-NIPS human evaluation round Donate your dialogue to chatbot research We\u00a0experience conversational artificial intelligence increasingly often, from bots that we can chat with on... Read More","og_url":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/","og_site_name":"IVADO","article_published_time":"2017-11-25T03:16:45+00:00","article_modified_time":"2020-02-27T14:16:36+00:00","og_image":[{"width":842,"height":1191,"url":"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png","type":"image\/png"}],"author":"Sandra Estrela","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Sandra Estrela","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/","url":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/","name":"The Conversational Intelligence Challenge - IVADO","isPartOf":{"@id":"https:\/\/vieux.ivado.ca\/#website"},"primaryImageOfPage":{"@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage"},"image":{"@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage"},"thumbnailUrl":"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png","datePublished":"2017-11-25T03:16:45+00:00","dateModified":"2020-02-27T14:16:36+00:00","author":{"@id":"https:\/\/vieux.ivado.ca\/#\/schema\/person\/6aa152aa0e33e45ec25475d10dc22fd0"},"breadcrumb":{"@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#primaryimage","url":"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png","contentUrl":"https:\/\/vieux.ivado.ca\/wp-content\/uploads\/2017\/12\/ConvAI_image.png","width":842,"height":1191},{"@type":"BreadcrumbList","@id":"https:\/\/vieux.ivado.ca\/en\/news\/the-conversational-intelligence-challenge\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/vieux.ivado.ca\/en\/"},{"@type":"ListItem","position":2,"name":"The Conversational Intelligence Challenge"}]},{"@type":"WebSite","@id":"https:\/\/vieux.ivado.ca\/#website","url":"https:\/\/vieux.ivado.ca\/","name":"IVADO","description":"Institut de valorisation des donn\u00e9es","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/vieux.ivado.ca\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/vieux.ivado.ca\/#\/schema\/person\/6aa152aa0e33e45ec25475d10dc22fd0","name":"Sandra Estrela","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vieux.ivado.ca\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1cc42a053bcbd01458777f6d1263133f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1cc42a053bcbd01458777f6d1263133f?s=96&d=mm&r=g","caption":"Sandra Estrela"},"url":"https:\/\/vieux.ivado.ca\/en\/author\/sestrela\/"}]}},"_links":{"self":[{"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/posts\/8101\/"}],"collection":[{"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/posts\/"}],"about":[{"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/types\/post\/"}],"author":[{"embeddable":true,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/users\/14\/"}],"replies":[{"embeddable":true,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/comments\/?post=8101"}],"version-history":[{"count":0,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/posts\/8101\/revisions\/"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/media\/8102\/"}],"wp:attachment":[{"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/media\/?parent=8101"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/categories\/?post=8101"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vieux.ivado.ca\/en\/wp-json\/wp\/v2\/tags\/?post=8101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}