The Challenge
Can AI scrape enough music chart data from enough sources across the web, reliably and accurately, to build a comprehensive multi-country chart browser? The data doesn't exist in one place — it's scattered across dozens of websites in different formats, languages, and levels of completeness. (Note: the 11 hours shown above is human time spent guiding and building. The scrapers themselves ran considerably longer to collect all the data.)
The Process
Supporting multiple countries made this project more involved than expected. The Netherlands and UK had solid, well-structured sources that the AI found entirely on its own — no guidance needed. The data volume was large so collection took time, but the process itself was effortless. For the Netherlands, the AI was even smart enough to find separate sources for older chart data and automatically combine them. The US was similar — a mix of different chart sources that came together cleanly.
France and Germany were another story. The AI struggled to find good sources, and it took quite a bit of manual research to locate decent sites and guide the scraping. Still, once pointed at a URL, the AI's ability to figure out how to scrape any website was remarkable — it basically takes zero time to produce a working scraper script.
With the raw chart data collected, the next step was smart aggregation and enrichment with metadata needed for the future Hitstory project. The AI was fast at suggesting possible APIs, evaluating the pros and cons, and integrating the best choice. When some data turned out to be unreliable — especially song language detection — we had it build a small web app to manually review and correct any errors.
That correction tool sparked a realization: this could easily become a proper app for anyone to search and browse chart history. Building the TopCharts interface with AI was fast, though getting it to look and work well on phones took some extra tweaking and guidance.
Surprises & Disappointments
Web scraping with AI is astonishingly easy. Just provide a URL and it figures out almost everything by itself — page structure, pagination, data extraction. The speed at which it produces working scrapers is genuinely impressive.
The biggest disappointment was audio playback on iOS. Song previews kept failing on iPhones and iPads, and it turned into a long, frustrating cycle of failed attempts and wrong suggestions. Only after explicitly asking the AI to first research whether it was even possible — before trying to fix it — did it quickly discover the iOS audio playback limitations and the correct workaround. It was a valuable lesson in how to work with AI: sometimes you need to tell it to stop and think before it starts throwing code at a problem.