In a significant breakthrough, Meta has successfully achieved real-time rendering of full-body Codec Avatars on its Quest 3 standalone headset, marking a major milestone in the company's decade-long research and development of this technology. Codec Avatars are photorealistic digital representations of humans that are driven in real-time by face and eye tracking, with the goal of delivering social presence - the subconscious feeling that you're truly with another person, despite them not physically being there. This achievement is a testament to Meta's dedication to pushing the boundaries of what is possible in virtual reality and social interaction.
The Codec Avatars technology has been steadily advancing, with recent progress including the development of highly realistic head-only avatars that can be generated from a selfie video and around an hour of processing on a server GPU. However, this system was still designed to run on a powerful PC graphics card, limiting its potential for widespread adoption. The latest breakthrough, outlined in a paper called "SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars", describes how Meta researchers have distilled their full-body photorealistic avatars to run on a mobile chipset, leveraging both the NPU and GPU. This distillation process involves using the output of a large, computationally expensive model to train a much smaller model, allowing for efficient replication with minimal quality loss.
The results are impressive, with SqueezeMe capable of rendering 3 full-body avatars at 72 FPS on a Quest 3, with almost no quality loss compared to the versions rendered on a PC. This achievement brings Meta one step closer to shipping Codec Avatars as a feature of its Horizon OS headsets, a prospect that has generated significant public interest and pressure. However, there are some notable tradeoffs to this breakthrough, including the use of traditional massive custom capture arrays and flat lighting, which limits the avatars' ability to dynamically adapt to different environments. Additionally, the lack of eye tracking and face tracking capabilities on the Quest 3 and Quest 3S headsets means that Meta may need to explore alternative approaches, such as AI-simulated face tracking, to bring Codec Avatars to a wider audience.
The implications of this breakthrough are significant, with potential applications in social media, video conferencing, and virtual reality environments. As Meta continues to refine and develop its Codec Avatars technology, we can expect to see new and innovative uses for this technology emerge. The fact that Apple is already shipping its new Personas in visionOS 26, effectively delivering on Meta's promise, adds an extra layer of urgency to Meta's efforts. With Meta Connect 2025 just around the corner, we may soon learn more about the company's plans for Codec Avatars and how it intends to bring this technology to the masses.
As we look to the future, it's clear that Meta's Codec Avatars technology has the potential to revolutionize the way we interact with each other in virtual environments. With its ability to deliver social presence and create a sense of connection with others, this technology could have a profound impact on our daily lives. Whether it's through video calls, social media, or virtual reality experiences, Codec Avatars is poised to change the way we interact with each other and the world around us. As Meta continues to push the boundaries of what is possible with this technology, we can expect to see new and exciting developments emerge in the months and years to come.
In conclusion, Meta's achievement of real-time rendering of full-body Codec Avatars on its Quest 3 headset is a significant milestone in the development of this technology. While there are still challenges to overcome, the potential implications of this breakthrough are vast and far-reaching. As we look to the future, it's clear that Codec Avatars is a technology to watch, with the potential to revolutionize the way we interact with each other and the world around us. With Meta Connect 2025 on the horizon, we may soon learn more about the company's plans for this technology and how it intends to bring it to the masses.
