← Back to archive

AI Breakthroughs Keep Coming From New Data Sources

17 September 2025 · 61 views · 3 likes · 1 reposts · 106 blog views

AI automation neuroscience productivity research

We always talk about AI getting smarter, faster, cheaper every year - and it does feel inevitable ... well, kind of. I read an article where the author mentioned a sort of "Moore's Law for AI" - where the length of tasks AIs can handle doubles every seven months.

The "big breakthroughs" in AI weren’t actually new ideas at all - they were old ideas finally unlocked by new data:

➡️ 2012: Alex Krizhevsky’s Deep Neural Networks took off after AlexNet crushed an image competition, thanks to the giant ImageNet dataset. ➡️ 2017: Google gave us Transformers, which unlocked the entire text of the internet. ➡️ 2022: OpenAI used Reinforcement Learning from Human Feedback (RLHF) to let models learn from human preferences. ➡️ 2024: They pushed further with OpenAI O1 and DeepSeek-R1, which use reasoning and "verifiers" like compilers and calculators to guide models.

Each leap forward came when we found a new data source to learn from - images, the web, humans, verifiers.

Maybe our next big breakthrough won’t come from some new algorithm… but from a new kind of data we haven’t used yet.

Like:

  • YouTube - which is this bottomless ocean of rich video data (about 500 hours uploaded every minute!).
  • Robots - sensors, cameras, embodied systems that could gather real-world data at scale, if only our hardware and training systems could keep up.

Remember the phrase 'data is the new oil'? It basically means that companies controlling user-generated content - like Reddit, YouTube, X, and Vimeo - hold all the power... until, of course, the government steps in and forces them to share it (which might not be that unlikely).

AI Breakthroughs Keep Coming From New Data Sources

Originally posted on X

PS: I'm pretty active on 𝕏 if you'd like to follow along

Related posts

Archive

2026

2025

2024

2018