Nvidia revealed an AI avatar prototype at CES 2025 that lives on your PC’s desktop. The AI assistant, R2Xlooks like a video game character and can help you navigate applications on your computer.
The R2X avatar is rendered and animated using Nvidia’s AI models, and users can run the avatar on popular LLMs of their choice, such as OpenAI’s GPT-4o or xAI’s Grok. Users can talk with R2X via text and voice, upload files to it for processing, or even allow the AI assistant to see what’s happening live on your screen or camera.
Tech companies are creating many AI avatars recently, not only in video games, but also for businesses and individuals. The first demos are strange, but some think these avatars make a promising user interface for AI assistants. With R2X, Nvidia is attempting to combine generative video game capabilities with cutting-edge AI assistants to create an AI assistant that looks like a human.
Much like Microsoft’s Recall feature (which has been delayed due to privacy concerns), R2X can take constant screenshots of your screen and run them through an AI model for processing, although this feature is disabled by default. When enabled, it can offer information about applications running on your computer and, for example, help you complete a complex coding task.
R2X is still a prototype, and even Nvidia admits there are still some bugs to work out. In demos with TechCrunch, Nvidia’s avatar had a strange valley feel to it – its face would sometimes get stuck in odd positions and its tone seemed a bit aggressive at times. And generally speaking, I find it strange to have a little humanoid avatar watching me while I do my work.
It generally offered helpful instructions and accurately visualized what was on the screen. But at one point the avatar gave us incorrect instructions, and later the avatar couldn’t see the screen at all. This may be a problem with the underlying AI model (in this case, GPT-4o), but the example shows the limitations of this first technology.
In a demo, an Nvidia product manager demonstrated how R2X can display and help users with apps on your screen. Specifically, R2X helped us use Adobe Photoshop’s Generative Fill feature. The photo we selected was of Nvidia CEO Jensen Huang standing in an Asian restaurant with two restaurant employees. Nvidia’s avatar hallucinated and gave wrong instructions on where to find the generative fill feature. But after replacing the AI model we were using with Grok from xAI, the avatar regained its screen viewing capabilities.
In another demo, R2X was able to ingest a CEO from the office and then answer questions about it. This process is powered by a local retrieval augmented generation feature, which gives these AI avatars the ability to extract information from a document and process it using its underlying LLM.
Nvidia uses some AI models from its video game division to optimize the appearance of these avatars. To generate avatars, Nvidia uses its RTX neural faces algorithm. To automate facial, lip and tongue movements, Nvidia uses a new model called Audio2Face™-3D. This model seemed to stall at times, holding the avatars’ faces in awkward positions.
The company also claims that these R2X avatars will be able to join Microsoft Teams meetings, acting as a personal assistant.
An Nvidia product manager says the company is working to give these AI avatars agent capabilities as well, so R2X could one day act on your desktop. These capabilities still seem far from achieved and would likely require partnerships with software publishers like Microsoft and Adobe, who themselves are trying to develop similar agent systems.
It’s not immediately clear how Nvidia generates the voices in these products. R2X’s voice when using GPT-4o sounds unique compared to all of ChatGPT’s preset voices, while xAI’s Grok chatbot does not yet have a voice mode.
The company plans to open source these avatars in the first half of 2025. Nvidia sees this as a new user interface that developers can build with, allowing users to plug in their favorite AI software products or even run these avatars locally.
