Result Details
Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
Kesiraju Santosh, Ph.D., DCGM (FIT)
Švec Ján, Ing., DCGM (FIT)
Sedláček Šimon, Ing., DCGM (FIT)
Yusuf Bolaji, DCGM (FIT)
Plchot Oldřich, Ing., Ph.D., DCGM (FIT)
Deepak K. T.
Černocký Jan, prof. Dr. Ing., DCGM (FIT)
This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.
dialog state tracking | in-context learning
@inproceedings{BUT199388,
author="Pradyoth {Hegde} and Santosh {Kesiraju} and Ján {Švec} and Šimon {Sedláček} and Bolaji {Yusuf} and Oldřich {Plchot} and {} and Jan {Černocký}",
title="Factors affecting the in-context learning abilities of LLMs for dialogue state tracking",
booktitle="Proceedings of the Annual Conference of the International Speech Communication Association Interspeech",
year="2025",
journal="Interspeech",
pages="4818--4822",
publisher="International Speech Communication Association",
address="Rotterdam, The Netherlands",
doi="10.21437/Interspeech.2025-2071",
url="https://www.isca-archive.org/interspeech_2025/hegde25_interspeech.pdf"
}Linguistics, Artificial Intelligence and Language and Speech Technologies: from Research to Applications, EU, MEZISEKTOROVÁ SPOLUPRÁCE, EH23_020/0008518, start: 2025-01-01, end: 2028-12-31, running
Multilingual and Cross-cultural interactions for context-aware, and bias-controlled dialogue systems for safety-critical applications, EU, HORIZON EUROPE, start: 2024-01-01, end: 2026-12-31, running
Practical verification of the possibility of integrating artificial intelligence for receiving emergency calls using a voice chatbot, developed within the research project BV No. VI20192022169, with technology for receiving emergency communications, MV, 1 VS OPSEC, VK01020132, start: 2023-01-06, end: 2025-10-31, completed