Publication Details

Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models

YUSUF, B.; BASKAR, M.; ROSENBERG, A.; RAMABHADRAN, B. Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models. In Proceedings of Interspeech 2024. Proceedings of Interspeech. Kos: International Speech Communication Association, 2024. p. 792-796. ISSN: 1990-9772.
Czech title
Spekulativní rozpoznávání řeči pomocí low-rank adaptace jazykových modelů prefixovanch audiem
Type
conference paper
Language
English
Authors
Yusuf Bolaji (DCGM)
BASKAR, M.
Rosenberg Andrew
Ramabhadran Bhuvana
URL
Keywords

low-latency speech recognition, speculative speech recognition, prefix language model, low-rank adaptation

Abstract

This paper explores speculative speech recognition (SSR),
where we empower conventional automatic speech recognition
(ASR) with speculation capabilities, allowing the recognizer to
run ahead of audio. We introduce a metric for measuring SSR
performance and we propose a model which does SSR by com
bining a RNN-Transducer-based ASR system with an audioprefixed language model (LM). The ASR system transcribes
ongoing audio and feeds the resulting transcripts, along with
an audiodependent prefix, to the LM, which speculates likely
completions for the transcriptions. We experiment with a variety of ASR datasets on which show the efficacy our method and
the feasibility of SSR as a method of reducing ASR latency.

Published
2024
Pages
792–796
Journal
Proceedings of Interspeech, vol. 2024, no. 9, ISSN 1990-9772
Proceedings
Proceedings of Interspeech 2024
Conference
Interspeech Conference, Kos, GR
Publisher
International Speech Communication Association
Place
Kos
DOI
EID Scopus
BibTeX
@inproceedings{BUT193739,
  author="YUSUF, B. and BASKAR, M. and ROSENBERG, A. and RAMABHADRAN, B.",
  title="Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models",
  booktitle="Proceedings of Interspeech 2024",
  year="2024",
  journal="Proceedings of Interspeech",
  volume="2024",
  number="9",
  pages="792--796",
  publisher="International Speech Communication Association",
  address="Kos",
  doi="10.21437/Interspeech.2024-298",
  issn="1990-9772",
  url="https://www.isca-archive.org/interspeech_2024/yusuf24_interspeech.pdf"
}
Files
Back to top