Face-tracking Software Lets You Make Anyone Say Anything in Real Time

Project Page: http://www.graphics.stanford.edu/~nie…

You know how they say, “Show me pictures or video, or it didn’t happen”?

Well, the days when you could trust what you see on video in real time are officially coming to an end thanks to a new kind of face tracking

A team from Stanford, the Max Planck Institute for Informatics and the University of Erlangen-Nuremberg has produced a video demonstrating how its software, called Face2Face, in combination with a common webcam, can make any person on video appear to say anything a source actor wants them to say.

In addition to perfectly capturing the real-time talking motions of the actor and placing them seamlessly on the video subject, the software also accounts for real-time facial expressions, including distinct movements such as eyebrow raises.

To show off the system, the team used YouTube videos of U.S. President George W. Bush, Russian President Vladimir Putin and Republican presidential candidate Donald Trump. In each case, the facial masking is flawless, effectively turning the video subject into the actor’s puppet.

It might be fun to mix this up with something like “Say it with Trump,” but for now the software is still in the research phase. “Unfortunately, the software is currently not publicly available — it’s just a research project,” team member Matthias Niessner told Mashable. “However, we are thinking about commercializing it given that we are getting so many requests.”

We knew this kind of stuff was possible in the special effects editing room, but the ability to do it in real time — without those nagging “uncanny valley” artifacts — could change how we interpret video documentation forever.