ONYX : Towards Extending Natural Language Interfaces for Data Visualization Tools through Interactive Task Learning

Abstract

While natural language interfaces (NLIs) are increasingly utilized to simplify the interaction with data visualization tools, adapting NLIs to the individual needs and requirements of end users still requires the support of developers. Our ONYX system introduces an interactive task learning (ITL)-based approach which enables NLIs to effectively learn from end users through natural interactions. End users can enhance the NLI with new commands or adapt existing commands using direct manipulation, natural language instructions, or a combination of both. ONYX guides end users through the demonstration process and provides them with recommendations for possible actions based on background knowledge of the system to enable an efficient interaction. In order to trigger reflections and gain feedback on the design of ONYX, we are currently preparing a formative study to understand how to best integrate guidance and recommendation capabilities provided by the ONYX system into the interaction.

Marcel Ruoff

PhD of Information Systems

My research interests include human-computer interaction, natural language interfaces, data visualizations, and end-user development.