I’m almost embarrassed to admit that the first time I saw one of Meta’s new holographic avatars today, during a WhatsApp call with a Meta representative I’d just met a few minutes before, I didn’t immediately notice that I was talking to an avatar. That’s how far the company’s avatar technology has come over the past two years.
Meta announced at Connect that it will bring its holographic avatars to its new VR glasses and Meta Ray-Ban Display frames. I had the chance to experience both before the keynote and was surprised at how far the technology has come since the first time I experienced it on Meta’s Orion prototype.
Setting up an avatar is a little different depending on the device you’re using. With display-enabled glasses, you create an avatar in the Meta AI app through a process similar to creating a Vision Pro “character.” Using your phone’s camera, you’re asked to turn your head to view different angles of your face. It then asks you a series of questions designed to capture the sound of your voice during different interactions.
In my demo, he asked me things like “what does your perfect day look like from start to finish” and “what is one of your hobbies and why do you love it.” This part of the demo actually had a problem. The Meta AI app kept skipping questions as if it couldn’t detect my voice, but the hiccups didn’t seem to affect the process. I was then asked to “smile and show my teeth” and repeat a sentence as if I was surprised.
In total, creating a holographic avatar for the display glasses should only take about 5 minutes from start to finish. Once set up, you can use it on a video call while wearing your display-enabled glasses, whether you’re chatting via WhatsApp or Zoom.
Avatars are powered by Meta’s new Muse Realtime Avatar model, and it uses this brief training period to approximate your real-life facial expressions while you speak. Watching my avatar speak was a strange experience. While I was wearing the glasses and speaking as the avatar, I could see a small glimpse of myself in the bottom corner of the screen. It looked exactly like me, but my brain was also able to immediately recognize that it was not Me.
But when I tried it from the other side, holding the phone and chatting with an avatar, I was fooled at first. Maybe it was because I had just met the person I was talking to. So I wasn’t familiar with his mannerisms or expressions, but it looked a lot like how I expected him to appear on a typical video call. Only by looking closer at its background was I able to detect that the video wasn’t quite normal. Meta also adds a transparent watermark to all video calls with an avatar. On my first viewing, I mistook this for a logo on his shirt, but if you know what to look for, it’s a clear identifier.
Meta’s plan for holographic avatars in its new VR glasses is even more ambitious. Since the cameras and sensors on this device are much more advanced, you can create a complete avatar and you don’t need to “teach” it your expressions. Instead, it relies on inward-facing cameras and sensors to understand your facial expressions.
I was able to chat with a life-size hologram of a Meta product manager while wearing the VR glasses. And it was less weird than I expected. It definitely felt a lot more natural than Apple’s “persona” avatars, which always seemed strangely smooth and alien-like to me. (These also don’t show the lower body or arms, which doesn’t help.)
But the VR avatar I spoke with looked pretty similar to the person I had just met and didn’t seem scary to me either. There was some distortion towards his feet, especially if I moved around the room, and I could see some blurring towards the edges of his head. Avatars are not fully 3D. They only show a frontal view of a person, so there are some limitations. But I was surprised at how much more realistic it seemed than any other interaction I’ve ever had with a VR avatar (a low bar, to be honest).
Holographic avatars are still in the “first look” stage, and there are clearly some bugs to work out. Availability is also limited to the device you’re using, so someone wearing Meta Ray-Ban Display Glasses can call someone else as their avatar, but only while they’re wearing the glasses. And someone wearing display glasses can’t call someone wearing VR glasses. VR avatars are limited to glasses-to-glass interactions at this time.
Meta said these avatars could help people feel more of a sense of presence during a video call, and I definitely felt that in my demo. There’s always something about calling a close friend or family member my avatar that would seem weird and inauthentic, but I could see how some might feel otherwise. The Meta reps I spoke with told me they often call business meetings as an avatar, which could be a good alternative when you’re not quite ready to shoot.
The holograms will be available on Meta Ray-Ban Display in early access this fall.
