New AI model “learns” how to simulate Super Mario Bros. from video footage

At first glance, these AI-generated Super Mario Bros. videos are pretty impressive. The more you watch, though, the more glitches you'll see. (credit: MarioVGG)

Last month, Google's GameNGen AI model showed that generalized image diffusion techniques can be used to generate a passable, playable version of Doom. Now, researchers are using some similar techniques with a model called MarioVGG to see if an AI model can generate plausible video of Super Mario Bros. in response to user inputs.

The results of the MarioVGG model—available as a pre-print paper published by the crypto-adjacent AI company Virtuals Protocol—still display a lot of apparent glitches, and it's too slow for anything approaching real-time gameplay at the moment. But the results show how even a limited model can infer some impressive physics and gameplay dynamics just from studying a bit of video and input data.

The researchers hope this represents a first step toward "producing and demonstrating a reliable and controllable video game generator," or possibly even "replacing game development and game engines completely using video generation models" in the future.