In the Industry 4.0 vision, collaborative robots or “cobots” will assist workers on the shop floors and factories of the future. Cobot assistants will need to be able to respond to requests and work on plans with human coworkers interactively, in that they should be able to ask for clarification and explanations when they do not understand a request or action. In other words, these assistants will need to be conversational.

Today’s cobots rely on extensive and time-consuming manual programming for complex tasks. And while conversational assistants have become prevalent in our daily lives, their utility is limited to the commands for which they are manually programmed or to (often unsatisfying) linguistic responses inferable from training on vast amounts of data.

Bringing together experts on visual grounding, conversation modeling, and robotics, the aim of COCOBOTS is to develop a natural language programming toolkit that will pave the way to a new generation of conversational assistants and cobots, ones able to use a more sophisticated understanding of a conversation and a visual scene to learn new concepts and update action plans in real time.

ANR-DLR bilateral French-German project (ANR-21-FAI2-0005)

1 November , 2021 – 31 October, 2024

Lead (French partner): LINAGORA