Generate real-time avatars from your 3D selfie without using Hollywood tech

The researchers at École Polytechnique Fédérale de Lausanne (EPFL), Switzerland, have developed a technology that can generate avatars with 3D selfies taken from a smartphone camera without using any hi-fi Hollywood technology. A user will just have to capture a few selfies along with his/her various facial expressions. After that, the algorithm will generate a digital double of the user and animate the avatar in real time by following user’s movement through the video camera.

Researcher Alexandru Ichim from the Computer Graphics and Geometry Laboratory at EPFL says, “The goal was to make the process accessible to anyone with a smartphone, even an old model, as long as it can take video.”

However, the underlying algorithms are not as simple as creating an avatar for the user. The real challenge was to replace studio conditions, which include proper lighting and numerous cameras with a simple smartphone technology. The researchers had to assume that a user will be taking selfies in variable conditions such as changing light conditions, blurry shots, and limited picture quality.

Alexandru added, “A small detail will turn people off immediately. The avatar has to have the right facial geometry and reproduce the texture, color and details like face wrinkles.”

Animating the selfies in real-time was another challenge to be considered. Everyone has their own way of smiling, yawning and frowning. If these expressions were to be recreated then new algorithms for real-time face animation were required to bring 3D avatars alive.

The research promises to be very useful in future for gaming, virtual reality, online discussions with other avatars, embedding in films, video conferences, and even avatar therapy for people suffering from schizophrenia.

For now, EPFL researchers are working on refining the process further and duplicating face features such as hair, ears, teeth and tongue of each individual person. Until now, all avatars had these features same irrespective of the 3D selfie.

You can watch researcher Alexandru explaining about the project here.

Via EPFL