MOBIO D7.1: Planning of evaluation campaigns

This interim report addressed the evaluations that will be conducted in the MOBIO project. It pointed out the need for standard evaluation metrics so that the results are comparable across project partners and to the rest of the world. Another important issue is the coherence of evaluation data that will be ensured by participation in standard benchmark tests, (such as NIST SRE campaigns, or ICB2009 face verification competition) but also by careful definition of data-sets on the MOBIO database.
The report also points out the importance of unsupervised adaptation of both uni- and bi-modal authentication systems that will be important for adaptation of the system in
varying environmental conditions.

The evaluations are important part of the MOBIO project, as they allow to monitor the progress within MOBIO, compare the participating labs to the state-of-the-art, ensure that the project progresses as planned and to know the accuracy of different technologies prior to their integration. This report deals with the metrics used and describes uni-modal (speech, face) as well as bimodal evaluations. It covers important evaluation campaigns running outside MOBIO, such as NIST speaker recognition benchmark tests, but also deals with evaluations that will be performed on the MOBIO database.