Thursday, 24 April 2025
26.8 C
Singapore
29.9 C
Thailand
19.5 C
Indonesia
28.8 C
Philippines

Elon Musk acknowledges we’ve run out of real-world AI training data

Elon Musk warns that AI training data is running out and suggests synthetic data as the solution while experts weigh its benefits and risks.

Elon Musk has agreed with leading AI experts that the world is running out of real-world data to train artificial intelligence models. During a live-streamed conversation with Stagwell chairman Mark Penn on X (formerly Twitter) on Wednesday night, Musk remarked, “We’ve now exhausted the cumulative sum of human knowledge … in AI training. That happened last year.”

The data drought and its implications

Musk, who owns the AI company xAI, is not the first to raise concerns about the shortage of AI training data. Ilya Sutskever, a former chief scientist at OpenAI, highlighted this issue during a December speech at the NeurIPS machine learning conference. He termed the situation “peak data” and predicted that this shortage would fundamentally change how AI models are developed.

As the pool of real-world data dwindles, researchers are turning to synthetic data—data created by AI models themselves—to fill the gap. Musk echoed this sentiment, explaining, “The only way to supplement [real-world data] is with synthetic data, where the AI creates [training data]. With synthetic data … [AI] will sort of grade itself and go through this process of self-learning.”

Tech giants embrace synthetic data

Several major tech companies, including Microsoft, Meta, OpenAI, and Anthropic, are already using synthetic data to train their flagship AI models. A report from Gartner estimates that by 2024, 60% of the data used in AI and analytics projects will be synthetically generated.

For instance, Microsoft’s recently open-sourced Phi-4 model combines synthetic data with real-world inputs. Similarly, Google’s Gemma models and Meta’s latest Llama series have also benefited from synthetic data in their development. Anthropic used synthetic data in creating its Claude 3.5 Sonnet system. At the same time, AI startup Writer developed its Palmyra X 004 model almost entirely with synthetic data at a fraction of the usual cost.

Synthetic data has clear cost advantages. The writer revealed that its Palmyra X 004 model cost just US$700,000 to develop, compared to the estimated US$4.6 million spent on an OpenAI model of similar size.

The challenges of synthetic data

Despite the advantages, synthetic data is not without its drawbacks. Research has shown that reliance on synthetic data can lead to “model collapse,” where AI systems become less creative and more biased over time. This happens because models generating synthetic data often amplify the biases and limitations present in the original training datasets, which can severely compromise the functionality of AI systems in the long run.

Musk’s comments reflect a growing consensus in the AI industry: synthetic data is the way forward, but it must be used cautiously. The challenge now lies in finding ways to create artificial data that mitigate biases and maintain the creativity and reliability of AI systems.

Hot this week

Zoho partners with Cradle to boost Malaysia’s startup ecosystem

Zoho and Cradle are partnering to provide RM44 million in software credits to 4,400 startups, supporting Malaysia's tech innovation drive.

GenAI investment could unlock ‘co-working’ future in financial services, says Moody’s

Moody’s sees GenAI transforming finance with AI co-workers, helping firms grow, automate decisions, and lower access barriers.

Chinese tech companies race to expand AI services using new open-standard protocol

Chinese tech firms race to adopt MCP, boosting AI agent use and shaping the future of smart services in payment, maps, and cloud tools.

Meta steps up AI age checks on Instagram to protect teens

Meta adds AI tools to find underage Instagram users and apply teen safety settings, even if their accounts list an adult age.

Proofpoint launches unified cybersecurity platform to cut costs and tackle multichannel threats

Proofpoint launches Prime Threat Protection, a unified cybersecurity platform tackling human risk and multichannel threats while reducing costs.

POCO launches entry-level C71 smartphone in Singapore with premium features

POCO launches the budget-friendly C71 smartphone in Singapore, offering premium design, enhanced cameras, and smooth performance at S$109.

NVIDIA uses AI to address climate, wildlife and disaster risks

NVIDIA’s AI tools support climate action, wildlife monitoring, and disaster risk mitigation, with uses spanning sea, land, sky and space.

Netflix raises subscription prices in Singapore again

Netflix again raises subscription prices in Singapore, with new rates for all plans and extra member slots.

GameMax unveils Blade Concept ATX case with bold design and powerful features

GameMax launches the Blade Concept ATX case, which features a striking blade design, RGB lighting, and support for high-end liquid-cooled PC builds.

Related Articles

Popular Categories