Thursday, 3 April 2025
26.1 C
Singapore
29.2 C
Thailand
27 C
Indonesia
27.3 C
Philippines

Elon Musk acknowledges we’ve run out of real-world AI training data

Elon Musk warns that AI training data is running out and suggests synthetic data as the solution while experts weigh its benefits and risks.

Elon Musk has agreed with leading AI experts that the world is running out of real-world data to train artificial intelligence models. During a live-streamed conversation with Stagwell chairman Mark Penn on X (formerly Twitter) on Wednesday night, Musk remarked, “We’ve now exhausted the cumulative sum of human knowledge … in AI training. That happened last year.”

The data drought and its implications

Musk, who owns the AI company xAI, is not the first to raise concerns about the shortage of AI training data. Ilya Sutskever, a former chief scientist at OpenAI, highlighted this issue during a December speech at the NeurIPS machine learning conference. He termed the situation “peak data” and predicted that this shortage would fundamentally change how AI models are developed.

As the pool of real-world data dwindles, researchers are turning to synthetic data—data created by AI models themselves—to fill the gap. Musk echoed this sentiment, explaining, “The only way to supplement [real-world data] is with synthetic data, where the AI creates [training data]. With synthetic data … [AI] will sort of grade itself and go through this process of self-learning.”

Tech giants embrace synthetic data

Several major tech companies, including Microsoft, Meta, OpenAI, and Anthropic, are already using synthetic data to train their flagship AI models. A report from Gartner estimates that by 2024, 60% of the data used in AI and analytics projects will be synthetically generated.

For instance, Microsoft’s recently open-sourced Phi-4 model combines synthetic data with real-world inputs. Similarly, Google’s Gemma models and Meta’s latest Llama series have also benefited from synthetic data in their development. Anthropic used synthetic data in creating its Claude 3.5 Sonnet system. At the same time, AI startup Writer developed its Palmyra X 004 model almost entirely with synthetic data at a fraction of the usual cost.

Synthetic data has clear cost advantages. The writer revealed that its Palmyra X 004 model cost just US$700,000 to develop, compared to the estimated US$4.6 million spent on an OpenAI model of similar size.

The challenges of synthetic data

Despite the advantages, synthetic data is not without its drawbacks. Research has shown that reliance on synthetic data can lead to “model collapse,” where AI systems become less creative and more biased over time. This happens because models generating synthetic data often amplify the biases and limitations present in the original training datasets, which can severely compromise the functionality of AI systems in the long run.

Musk’s comments reflect a growing consensus in the AI industry: synthetic data is the way forward, but it must be used cautiously. The challenge now lies in finding ways to create artificial data that mitigate biases and maintain the creativity and reliability of AI systems.

Hot this week

Facebook introduces friends-only feed to cut out algorithmic content

Facebook’s new Friends tab removes algorithmic recommendations, letting you see only posts from friends. It is now rolling out in the US and Canada.

NTT DATA partners with UPS on 10-year digital transformation deal

NTT DATA signs 10-year deal with UPS to modernise IT infrastructure, support AI innovation, and manage data centre operations.

Apple prepares for M5 iPad Pro and MacBook Pro release

Apple is set to launch the M5 iPad Pro and MacBook Pro in late 2024, with the M6 models expected to introduce an in-house modem in 2027.

Samsung’s latest vacuum alerts you to calls and texts while you clean

Samsung’s new Bespoke AI Jet Ultra vacuum can alert you to calls and texts while cleaning as the brand expands smart home screens across appliances.

Intel’s future in the GPU market looks uncertain

Intel may not release a high-end Battlemage GPU, and Arc Celestial’s future is unclear, leaving gamers with limited options in a challenging market.

Qualcomm expands AI research with MovianAI acquisition

Qualcomm has acquired Vietnamese AI research firm MovianAI to boost its AI development in smartphones, PCs, and software-defined vehicles.

Roblox introduces new parental controls to enhance child safety

Roblox introduces new parental controls, allowing parents to block games, restrict friends, and monitor their child’s activity for better safety.

Anthropic introduces Claude for Education, a new AI chatbot plan for universities

Anthropic launches Claude for Education, an AI chatbot plan for universities that offers advanced learning tools and administration support.

Exabeam introduces Nova, an agentic AI that boosts cybersecurity operations

Exabeam unveils Nova, a proactive AI agent that boosts security team productivity and reduces incident investigation time by over 50%.

Related Articles