Project Details
Multilingual and Cross-cultural interactions for context-aware, and bias-controlled dialogue systems for safety-critical applications
Project Period: 1. 1. 2024 – 31. 12. 2026
Project Type: grant
Agency: European Union
Program: HORIZON EUROPE
Human computer interaction and interface, visualization and natural language,
artificial intelligence, intelligence systems, multi agents systems, natural
language processing, data protection and privacy, machine learning, statistical
data processing and applications using data processing, formal, cognitive,
functional and computational linguistics, distributed and federated adaptation of
Large Language Models, Multilinguality, Multimodality, Human-in-the-loop, Bias
mitigation, Grounding.
ELOQUENCE aims to research and develop new technologies supporting collaborative
voice/chat bots for both low secure (low risk) and highly secure (high risk)
applications. Dialogue engines powered by voice assistants have already been
present in various commercial/governmental applications with lower or higher
level complexities. In both cases, this complexity can be translated to a problem
of analysing unstructured dialogues. Key objective of ELOQUENCE is to understand
unstructured dialogues and conduct them in an explainable, safe,
knowledge-grounded, trustworthy and unbiased way, while considering and building
on top of prior achievements in this domain (e.g. recently launched chatGPT Large
Language Models (LLMs). While including key industrial enterprises from Europe in
this project (i.e. Omilia, Telefonica. ...) will approach safety with
human-in-the-loop for safety-critical applications (i.e., emergency services) and
via information retrieval and fact-checking against an online knowledge base for
less critical autonomous systems (i.e., home-assistants). ELOQUENCE will target
the R&D of these novel conversational AI technologies in multilingual and
multimodal environments. Both basic research and its direct deployment through
two pilots will be targeted: 1) emergency call contact centres and 2) smart
assistants through decentralised training in smart homes.
Beneš Karel, Ing., Ph.D. (FIT)
Burget Lukáš, doc. Ing., Ph.D. (DCGM)
Fajčík Martin, Ing., Ph.D. (DCGM)
Heřmanský Hynek, prof. Ing., Dr. Eng. (DCGM)
Kesiraju Santosh, Ph.D. (DCGM)
Kohlová Renata, Ing. (DCGM)
Peng Junyi (DCGM)
Pešán Jan, Ing. (DCGM)
Rohdin Johan Andréas, M.Sc., Ph.D. (DCGM)
Sarvaš Marek, Ing.
Sedláček Šimon, Ing. (DCGM)
Schwarz Petr, Ing., Ph.D. (DCGM)
Udupa Sathvik (DCGM)
Valentini Francisco Tomas, BSc
Yusuf Bolaji (DCGM)
2026
- THORBECKE, I.; VILLATORO-TELLO, E.; ZULUAGA, J.; KUMAR, S.; BURDISSO, S.; RANGAPPA, P.; CAROFILIS, A.; MADIKERI, S.; MOTLÍČEK, P.; PANDIA, K.; HACIOGLU, K.; STOLCKE, A. Unifying Global and Near-Context Biasing in a Single Trie Pass. In Lecture Notes in Artificial Intelligence. Lecture Notes in Computer Science. CHAM: Springer Nature, 2026.
p. 170-181. ISBN: 978-3-032-02547-0. Detail
2025
- Alexander Polok, Jiangyu Han, Dominik Klement, Samuele Cornell, Jan Černocký, Lukáš Burget. BUT System for the MLC-SLM Challenge. ISCA: ISCA, 2025.
p. 23-27. Detail - FAJČÍK, M.; DOČEKAL, M.; DOLEŽAL, J.; ONDŘEJ, K.; BENEŠ, K.; SMRŽ, P.; POLOK, A.; HRADIŠ, M. BenCzechMark : A Czech-Centric Multitask and Multimetric Benchmark for Large Language Models with Duel Scoring Mechanism. Transactions of the Association for Computational Linguistics, 2025, vol. 13, no. 9,
p. 1068-1095. Detail - HEGDE, P.; KESIRAJU, S.; ŠVEC, J.; SEDLÁČEK, Š.; YUSUF, B.; PLCHOT, O.; DEEPAK, K.; ČERNOCKÝ, J. Factors affecting the in-context learning abilities of LLMs for dialogue state tracking. In Proceedings of the Annual Conference of the International Speech Communication Association Interspeech. Interspeech. Rotterdam, The Netherlands: International Speech Communication Association, 2025.
p. 4818-4822. Detail - KUMAR, S.; THORBECKE, I.; BURDISSO, S.; VILLATORO-TELLO, E.; MANJUNATH, K.; HACIOGLU, K.; RANGAPPA, P.; MOTLÍČEK, P.; GANAPATHIRAJU, A.; STOLCKE, A. Performance Evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward. In 2025 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING WORKSHOPS, ICASSPW. Hyderabad, Indická republika: IEEE, 2025.
p. 1-5. ISBN: 979-8-3315-1932-2. Detail - RANGAPPA, P.; CAROFILIS, A.; PRAKASH, J.; KUMAR, S.; BURDISSO, S.; MADIKERI, S.; VILLATORO-TELLO, E.; SHARMA, B.; MOTLÍČEK, P.; HACIOGLU, K.; VENKATESAN, S.; VYAS, S.; STOLCKE, A. Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-Labels and Multi-Stage Filtering. In Interspeech. Interspeech. Rotterdam, The Netherlands: Isca-Int Speech Communication Assoc, 2025.
p. 4928-4932. Detail - SEDLÁČEK, Š.; YUSUF, B.; ŠVEC, J.; HEGDE, P.; KESIRAJU, S.; PLCHOT, O.; ČERNOCKÝ, J. Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs. In Proceedings of the Annual Conference of the International Speech Communication Association Interspeech. Interspeech. Rotterdam, The Netherlands: International Speech Communication Association, 2025.
p. 1748-1752. Detail
2024
- BURDISSO, S.; MADIKERI, S.; MOTLÍČEK, P. Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: Association for Computational Linguistics, 2024.
p. 5421-5440. ISBN: 9798891761643. Detail - PEŠÁN, J.; JUŘÍK, V.; RŮŽIČKOVÁ, A.; SVOBODA, V.; JANOUŠEK, O.; NĚMCOVÁ, A.; BOJANOVSKÁ, H.; ALDABAGHOVÁ, J.; KYSLÍK, F.; VODIČKOVÁ, K.; SODOMOVÁ, A.; BARTYS, P.; CHUDÝ, P.; ČERNOCKÝ, J. Speech production under stress for machine learning: multimodal dataset of 79 cases and 8 signals. Scientific Data, 2024, vol. 11, no. 1,
p. 1-9. ISSN: 2052-4463. Detail - POLOK, A.; KLEMENT, D.; HAN, J.; SEDLÁČEK, Š.; YUSUF, B.; MACIEJEWSKI, M.; WIESNER, M.; BURGET, L. BUT/JHU System Description for CHiME-8 NOTSOFAR-1 Challenge. Proceedings of CHiME 2024 Workshop. Kos Island: International Speech Communication Association, 2024.
p. 18-22. Detail - ROHDIN, J.; ZHANG, L.; PLCHOT, O.; STANĚK, V.; MIHOLA, D.; PENG, J.; STAFYLAKIS, T.; BEVERAKI, D.; SILNOVA, A.; BRUKNER, J.; BURGET, L. BUT systems and analyses for the ASVspoof 5 Challenge. Proceedings of ASV spoof 2024 Workshop. Kos Island: International Speech Communication Association, 2024.
p. 24-31. Detail - ŠTĚTINA, J.; FAJČÍK, M.; HRADIŠ, M.; ŠTEFÁNIK, M. A Comparative Study of Text Retrieval Models on DaReCzech. Recent Advances in Slavonic Natural Language Processing, 2024, no. 7,
p. 85-100. Detail - YUSUF, B.; ČERNOCKÝ, J.; SARAÇLAR, M. Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. Proceedings of Interspeech. Kos: International Speech Communication Association, 2024. no. 9,
p. 5068-5072. ISSN: 1990-9772. Detail