Skip to main content

Posts

Showing posts from February, 2020

End-to-End Speech-Driven Facial Animation with Temporal GANs

In this paper , the authors present a system for generating videos of a talking heard, using a still image of a person and an audio clip containing speech. As per the authors this is the first paper that achieves this without any handcrafted features or post-processing of the output. This is achieved using a temporal GAN with 2 discriminators. Novelties in this paper Talking head video generated from still image and speech audio without any subject dependency. Also no handcrafted audio or visual features are used for training and no post-processing of the output (generate facial features from learned metrics). The model captures the dynamics of the entire face producing natural facial expressions such as eyebrow raises, frowns and blinks. This is due to the Recurrent Neural Network ( RNN ) based generator and sequence discriminator. Ablation study to quantify the effect of each component in the system. Image quality is measured using Model Architecture The model consists