WELCOME - Multiple Intelligent Conversation Agent Sevices for Reception, Management and Integration of Third Country Nationals

Czech title

Vícenásobné služby inteligentního konverzačního agenta pro přijetí, řízení a integraci občanů třetích zemí v EU

Type

grant

Keywords

migration, social issues, public sector innovation, personalized embodied conversation agents, language teaching, social and societal inclusion, decision support, multilingual language technologies, virtual/augmented reality, knowledge processing.

Abstract

The objective of WELCOME is to research and develop intelligent technologies for support of the reception and integration of migrants in Europe. Unlike the majority of the projects that address the problem of migrant integration by frontends with "one-for-all" interfaces for information acquisition and services, WELCOME will offer a personalized and psychologically and socially competent solution for both migrants and public administrations. It will develop immersive and intelligent services, in which embodied intelligent multilingual agents will act as dedicated personalized assistants of migrants in contexts of registration, orientation, language teaching, civic education, and social and societal inclusion. The agents will be personalized in the sense that they will dynamically adapt their interaction behavior to the topic, given context and the profile of the interlocutor. To provide "real-life" experience and thus deeper and swifter integration, immersive virtual and augmented reality technologies will be used. For public administrations, decision support technologies that will draw upon visual analytics and semantic reasoning techniques will be developed. To achieve its objective, WELCOME will innovate in the areas of human - multiple service-oriented agent interaction, multilingual spoken language technologies, knowledge processing, immersive personalized education and social inclusion technologies, and decision support strategies. WELCOMEs solution will be validated and its portability to different European contexts demonstrated in three pilot use cases. Both WELCOME as a whole and its individual technologies are expected to have a very significant societal and economic impact. WELCOME counts with a highly competent Consortium of 15 partners: 6 research institutions, 3 ICT companies, and 6 entities related to migrant reception and integration. To increase its impact and dissemination, WELCOME involves the International Migrant Organization as subcontractor.

Team members

Černocký Jan, prof. Dr. Ing. (UPGM FIT VUT) , research leader
Burget Lukáš, doc. Ing., Ph.D. (UPGM FIT VUT) , team leader
Karafiát Martin, Ing., Ph.D. (UPGM FIT VUT) , team leader
Kocour Martin, Ing. (UPGM FIT VUT)
Plchot Oldřich, Ing., Ph.D. (UPGM FIT VUT)
Žižka Josef, Ing. (UPGM FIT VUT)

Support

[img]
This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No 870930.

Publications

2023

YUSUF Bolaji, ČERNOCKÝ Jan and SARAÇLAR Murat. End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, vol. 31, no. 08, 2023, pp. 3070-3080. ISSN 2329-9290. Detail

2022

ŠVEC Ján, ŽMOLÍKOVÁ Kateřina, KOCOUR Martin, DELCROIX Marc, OCHIAI Tsubasa, MOŠNER Ladislav and ČERNOCKÝ Jan. Analysis of impact of emotions on target speech extraction and speech separation. In: Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022). Bamberg: IEEE Signal Processing Society, 2022, pp. 1-5. ISBN 978-1-6654-6867-1. Detail
ONDEL Yang Lucas Antoine Francois, YUSUF Bolaji, BURGET Lukáš and SARAÇLAR Murat. Non-Parametric Bayesian Subspace Models for Acoustic Unit Discovery. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, vol. 30, no. 5, 2022, pp. 1902-1917. ISSN 2329-9290. Detail
EGOROVA Ekaterina, VYDANA Hari K., BURGET Lukáš and ČERNOCKÝ Jan. Spelling-Aware Word-Based End-to-End ASR. IEEE Signal Processing Letters, vol. 29, no. 29, 2022, pp. 1729-1733. ISSN 1558-2361. Detail

2021

YUSUF Bolaji, ONDEL Yang Lucas Antoine Francois, BURGET Lukáš, ČERNOCKÝ Jan and SARAÇLAR Murat. A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery. In: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Toronto, Ontario: IEEE Signal Processing Society, 2021, pp. 3710-3714. ISBN 978-1-7281-7605-5. Detail
YUSUF Bolaji, GOK Alican, GUNDOGDU Batuhan and SARAÇLAR Murat. End-to-End Open Vocabulary Keyword Search. In: Proceedings Interspeech 2021. Brno: International Speech Communication Association, 2021, pp. 4388-4392. ISSN 1990-9772. Detail
EGOROVA Ekaterina, VYDANA Hari K., BURGET Lukáš and ČERNOCKÝ Jan. Out-of-Vocabulary Words Detection with Attention and CTC Alignments in an End-to-End ASR System. In: Proceedings Interspeech 2021. Brno: International Speech Communication Association, 2021, pp. 2901-2905. ISSN 1990-9772. Detail
WANNER Leo, KLUSCH Matthias, MAVROPOULOS Athanasios, JAMIN Emmanuel, MARIN Puchades Victor, CASAMAYOR Gerard, ČERNOCKÝ Jan and EGOROVA Ekaterina et al. Towards a Versatile Intelligent Conversational Agent as Personal Assistant for Migrants. In: The PAAMS Collection. PAAMS 2021: Advances in Practical Applications of Agents, Multi-Agent Systems, and Social Good. . Lecture Notes in Computer Science book series , vol. 12946. Salamanca: Springer International Publishing, 2021, pp. 316-327. ISBN 978-3-030-85739-4. ISSN 0302-9743. Detail