World simulators, or world models, are taking the artificial intelligence (AI) world by storm, with many experts suggesting they represent the next significant advancement in the field. AI trailblazer, Fei-Fei Li’s World Labs, has garnered $230 million in funding to develop large world models. In addition, AI giant DeepMind has recently recruited one of the brains behind OpenAI’s video generator, Sora, to work on world simulators.
World models draw inspiration from human cognition, whereby our brains convert abstract sensory inputs into a concrete understanding of our surroundings. This mechanism, long referred to as ‘models’ even before AI adopted the term, allows us to predict and interpret the world around us.
A research paper by AI experts David Ha and Jürgen Schmidhuber presents the example of a baseball batter to explain this concept. According to the researchers, a batter is able to hit a fast-moving baseball due to the brain’s ability to predict the ball’s trajectory in milliseconds. They argue that understanding the subliminal reasoning behind world models is essential for achieving human-level intelligence in AI.
World models have gained traction recently due to their potential applications in generative video creation. Most AI-generated videos suffer from uncanny anomalies, such as limbs merging into each other. However, a world model with a rudimentary understanding of why a basketball bounces can better depict this action.
World models are trained using diverse data sources, including photos, audio, videos, and text. The goal is to create an internal representation of the world that can infer the implications of various actions. Alex Mashrabov, the former AI chief at Snap and current CEO of Higgsfield, believes that strong world models can eliminate the need for creators to manually define object movement, saving time and effort.
According to Yann LeCun, chief AI scientist at Meta, the potential of world models extends beyond video generation to sophisticated forecasting and planning in both digital and physical environments. However, LeCun estimates that achieving the desired level of world models is at least a decade away.
Today’s world models exhibit promising capabilities as basic physics simulators. OpenAI claims that its world model, Sora, can simulate actions like a painter leaving brush strokes on a canvas and can effectively simulate video games.
Despite the exciting potential of world models, several technical challenges impede their development. Training and operating world models require significant computational power, and like all AI models, they are prone to hallucinations and biases in training data. Moreover, the lack of sufficient training data could exacerbate these issues.
If these hurdles can be overcome, world models could facilitate significant advancements in virtual world creation, robotics, and AI decision-making. They could also lead to the development of more sophisticated robots with a better understanding of their surroundings.
Stay updated with the latest advancements in AI by subscribing to TechCrunch’s AI-focused newsletter.
