Meet Maxine, Nvidia’s AI system for fixing some of the most common problems in video calls

Nvidia has been a very relevant name in the world of commercial AI applications, with its GPUs powering some of the most innovative research projects and some of the most lucrative AI-based ventures. Its new project, however, is one that promises to fix some of the most pressing problems we face in this age of remote communications.

Nvidia Maxine is an AI system that can process video calls in the cloud using the company’s powerful GPUs and enhance call quality in various ways. For one, it can use AI to realign callers’ faces and gazes so that they’re always looking directly at the camera and thus maintaining a uniform mode of interaction.

For another, it can reduce the bandwidth requirement for video “down to one-tenth of the requirements of the H:264 streaming video compression standard”. It achieves this incredible feat by only transmitting key facial features and by generally upscaling the resolution of videos.

And if it’s superior ability to realign faces and fix poor video resolution wasn’t enough, Maxine can also re-light faces, translate and transcribe sentences in real-time, and produce animated avatars.

Nvidia says its compression feature uses an AI method known as generative adversarial networks or GANs to partially reconstruct callers’ faces in the cloud. This is the same technique used in many deepfakes.

“Instead of streaming the entire screen of pixels, the AI software analyzes the key facial points of each person on a call and then intelligently re-animates the face in the video on the other side,” said the company in a blog post. “This makes it possible to stream video with far less data flowing back and forth across the internet.”

Nvidia can actually rise above its competition in the domain of video call processing by relying on its cloud computing resources and the bulk of AI R&D it has done over the years. It’s also worth noting that Maxine is not a consumer platform per se. Instead, Nvidia will be providing it as a toolkit for third-party firms so that they can improve their own services. For now, it has only announced a partnership with a communications firm Avaya.