The video game Valorant, a fast-paced team-based shooter, has recently become a testing ground for a promising new direction in artificial intelligence research. The game’s developers at Riot Games (a Tencent subsidiary) are using 3D-native AI models to prototype new characters, scenes, and storylines, according to a researcher familiar with the company’s efforts who spoke on the condition of anonymity.
While many AI models can generate text, images, and video, Tencent’s Hunyuan (混元 or “first mix”) family of models can dream up 3D objects and interactive scenes. The source says that Tencent’s models are also being used by the developers of another Tencent game, GKART, and by some independent developers, too. Tencent declined to comment.
“The games industry requires a lot of investment,” the source says. “Previously you would need a month to design a character. Now you can just type in some text, and Hunyuan can give you four choices in 60 seconds.”
The news is an early signal that models capable of understanding and re-creating the physical world could become a standard ingredient in game design. In addition to generating game content, these models could also enable more advanced virtual and augmented reality and help robots learn to do new things.
“There’s a real explosion of 3D vision research nowadays,” says Alexander Raistrick, a graduate student at Princeton University working on novel approaches to generating 3D content. “There are many killer applications: There’s content creation, there’s self driving, and there’s a whole stack of problems involved in augmented reality.”
Raistrick adds that video games are an obvious application for 3D AI models. “Outputting 3D meshes [a standard way of representing 3D objects] is your typical kind of bread and butter of game development,” he says.
But, as in other creative fields, using AI to create video games is controversial. Concerns about AI-fueled job loss loom large. Some developers say games should be labeled when they contain AI-made content. Others say it’s too late: The technology is already ubiquitous in the industry.
Tencent released HunyuanWorld 1.0, a model that generates interactive scenes, in July. I tested it a few months back, exploring a scene that looked like it was part of a Lego movie—a valley of brightly colored blocks disappearing into the distance. More recently, I’ve been playing with a more basic model, Hunyuan 3D, which can conjure up 3D objects. I used it to generate some very nice custom Dungeons & Dragons characters to 3D print. In October, Tencent released a new version of HunyuanWorld that lets users upload video to generate 3D scenes.
Tencent’s Hunyuan models point to a broader shift happening in AI research. Many experts believe that AI models will need a deeper understanding of the physical world to advance. Because of this, Tencent is far from alone in building 3D-native AI models. Microsoft, Meta, Stability AI, and Bytedance all offer 3D models, but Hunyuan sits at the top of one leaderboard designed to rank such tools.
A number of startups are doing interesting work in this space, too. World Labs, founded by Fei-Fei Li, a Stanford computer scientist who played a key role in building modern AI, has developed a tool called Marble that produces fully consistent and persistent 3D scenes. This could be useful for generating games on the fly or producing reliable training data for robots.
3D AI is also an exciting area for academic research. A Stanford University project called 3D Generalist used an LLM to decide how to modify scenes with new objects. Raistrick, the graduate student at Princeton, is developing a way of generating 3D scenes using code, an approach that makes it possible for LLMs to generate and interact with scenes in a more powerful way. And projects like Google DeepMind’s SIMA 2 show how AI agents could interact with virtual worlds to create new forms of gameplay.
As 3D-capable AI becomes more important, Tencent may emerge as an increasingly important player among a host of Chinese AI firms clamoring to win in this space. Besides producing some of the world’s most popular video games and movies, it operates WeChat, a chat app with a wide range of other functions, that is ubiquitous in China. Tencent also has its own chatbot, called YuanBao, which is integrated into WeChat. But Tencent’s video game skills may give it a distinct edge in an increasingly 3D AI world.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
