Multimodal Human-Machine Interfaces for Morphing Computerized Mats with Embodied Sensing and Artificial Intelligence

grant

Human-machine interaction, multimodal processing, novel manipulation, usability test

The MOZART project aims to advance the state of the art in robotic handling and manipulation of soft and heterogeneous objects. Its
approach is a radical departure from existing technologies as it develops a new concept based on deformable manipulation surfaces
supported by AI-powered control and learning tools. While the running project primarily addresses the development of hardware and
control algorithms, human-machine interaction is less explored. The role of the new partner - Brno University of Technology, Czechia
(BUT), in the proposed project extension is mainly to research the field of Human-Machine Interaction (HMI), explicitly identifying user
roles, needs, and necessary interface types, iteratively developing prototypes, and performing evaluation. Usability testing will be
carried out in multiple stages of development, including aspects of the social science perspective (with the help of SSH research
experts from partner TCD), to assess the system's natural fit with diverse operators. Various input and output modalities and their
combinations will be evaluated, including the manipulation surface itself and augmented and virtual reality. Machine learning may be
used to achieve the adaptivity of the interface. Various types of interfaces that will be developed will be backed up by a cloudenabled
server, acting as a data platform, enabling data sharing between users and offloading computationally intensive tasks. BUT
aims to contribute to MOZART's goal of automating the food industry by designing effective user interfaces that enhance system
acceptance, leveraging the team's extensive experience in this field.

Smrž Pavel, doc. RNDr., Ph.D. (UPGM FIT VUT) , research leader
Bambušek Daniel, Ing. (UPGM FIT VUT)
Materna Zdeněk, Ing., Ph.D. (UPGM FIT VUT)
Zemčík Pavel, prof. Dr. Ing. (UPGM FIT VUT)