It's Good to Chat?: Evaluation and Design Guidelines for Combining Open-Domain Social Conversation with Task-Based Dialogue in Intelligent Buildings.

A deployed conversational AI system for a public university building demonstrates no significant difference in task completion between open-domain social chat and task-only versions, but users prefer task-only interactions for future use.

Published on Oct 19, 2020

Arxiv Link: https://dl.acm.org/doi/10.1145/3383652.3423889

Github

Technology

Docker

Reading time: 1 minutes.

Abstract :

We present and evaluate a deployed conversational AI system that acts as a host of a working public building on a university campus. The system combines open-domain social chat with task-based conversation regarding navigation in the building, live resource updates (e.g. available computers), and events in the building. We investigated the impact of open-domain social chat on task completion and user preferences by comparing the combined system with a task-only version. We find that there is no significant difference in task completion or several aspects of user preference between the two systems, but that users would be significantly happier to talk to the task-only system in the future. This suggests that the “walk-up” public setting and workplace nature of the environment creates a markedly different use case to the in-home, and more individual and private “companion/assistant” setting which is commonly assumed for systems like Alexa. We discuss the implications for the design of conversational systems in other public settings.