Result Details
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Fajčík Martin, Ing., Ph.D., DCGM (FIT)
Dočekal Martin, Ing., DCGM (FIT)
Ondřej Karel, Ing., DCGM (FIT)
Smrž Pavel, doc. RNDr., Ph.D., DCGM (FIT)
and others
We review the EfficientQA competition from NeurIPS 2020. The competition
focused on open-domain question answering (QA), where systems take natural
language questions as input and return natural language answers. The aim of the
competition was to build systems that can predict correct answers while also
satisfying strict on-disk memory budgets. These memory budgets were designed to
encourage contestants to explore the trade-off between storing retrieval
corpora or the parameters of learned models. In this report, we describe the
motivation and organization of the competition, review the best submissions,
and analyze system predictions to inform a discussion of evaluation for
open-domain QA.
question answering, QA, ODQA, efficientQA, memory, disk memory, budget, efficient parameter, retrieval corpora
@inproceedings{BUT175821,
author="MIN, S. and FAJČÍK, M. and DOČEKAL, M. and ONDŘEJ, K. and SMRŽ, P.",
title="NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned",
booktitle="Proceedings of the NeurIPS 2020 Competition and Demonstration Track",
year="2021",
series="Proceedings of Machine Learning Research",
journal="Proceedings of Machine Learning Research",
volume="133",
number="133",
pages="86--111",
publisher="Proceedings of Machine Learning Research",
address="online",
url="http://proceedings.mlr.press/v133/min21a/min21a.pdf"
}