Animace tváře avatara na základě záběrů lidské tváře

English title

Animation of Avatar Face Based on Human Face Video

Language

Czech

Abstract

This thesis presents an application for animating 3D avatar based on a single camera or video input of human face in real time. The resulting application consists of three modules -- face tracking, avatar animator, and a server for transferring face data. The face tracking module computes new transforms for the animation from human face and benefits from Mediapipe's Facemesh to estimate the current face geometry. Avatar animator module is a web-based application for rendering and animating 3D avatars through skeletal animations, based on the Three.js library. Both modules make use of the continuous bidirectional communication of websockets through the single server. Performance of the face tracking module depends on the camera and device on which it is running, but regular web camera is usually enough for speed of 30+ FPS and animation runs at the speed of 60+ FPS with multiple avatars.Main contributions of this project are (a) Calculating transforms from human face is suitable for the skeletal animations, which are usually easier to create and more accessible than blend shapes. (b) Using the face tracking and the avatar animator as independent modules causes that from the single camera/video input it is possible to animate multiple avatars on different devices.

Keywords

face tracking, 3D character, face animation, skeletal animation, websockets

Department

Department of Computer Graphics and Multimedia FIT BUT

Degree Programme

Information Technology

Files

Status

defended, grade B

Date

15 June 2022

Reviewer

Milet Tomáš, Ing., Ph.D.

Committee

Herout Adam, prof. Ing., Ph.D. (DCGM FIT BUT), předseda
Kočí Radek, Ing., Ph.D. (DITS FIT BUT), člen
Křivka Zbyněk, Ing., Ph.D. (DIFS FIT BUT), člen
Španěl Michal, Ing., Ph.D. (DCGM FIT BUT), člen
Vašíček Zdeněk, doc. Ing., Ph.D. (DCSY FIT BUT), člen

Citation

TAKÁCS, Martin. Animace tváře avatara na základě záběrů lidské tváře. Brno, 2022. Bachelor's Thesis. Brno University of Technology, Faculty of Information Technology. 2022-06-15. Supervised by Herout Adam. Available from: https://www.fit.vut.cz/study/thesis/25040/

BibTeX

@bachelorsthesis{FITBT25040,
    author = "Martin Tak\'{a}cs",
    type = "Bachelor's thesis",
    title = "Animace tv\'{a}\v{r}e avatara na z\'{a}klad\v{e} z\'{a}b\v{e}r\r{u} lidsk\'{e} tv\'{a}\v{r}e",
    school = "Brno University of Technology, Faculty of Information Technology",
    year = 2022,
    location = "Brno, CZ",
    language = "czech",
    url = "https://www.fit.vut.cz/study/thesis/25040/"
}