Query Public Datasets with Claude Code and SQL

What you’ll get

A way to dig through huge public datasets (like all of Hacker News or arXiv) by asking questions in plain English and letting Claude write the SQL for you. SQL is the standard language for querying databases; this lets you skip learning it. The reproducible, free part is “describe what you want, get a correct query”; actually running it needs access to a database holding the data.

Tools you need

• Claude (freemium): writes the SQL from your plain-English question. The free tier handles this.
• SQL (free): the query language Claude generates. You do not need to know it, but you do need somewhere to run the query if you want results.
• Voyage-3.5-lite (paid, optional): an embedding model the original author used for advanced semantic search. Not needed just to generate SQL.

Steps

1. Open Claude: go to claude.ai and start a chat (a free account is enough).

2. Decide what you want to find: be concrete, a topic, a time range, what columns you want back.

3. Ask Claude for the SQL: tell it you want a query, and give it the table and column names you expect (its “schema”):

   Write an SQL query to find Hacker News posts about "AI ethics" from the last year.
   Assume a table named "posts" with columns: title, url, created_at, and score.
   Return the title, url, and score, sorted by score.

   You should get a ready-to-use SQL query.

4. Check the query: read it over. Make sure the table and column names match your actual data, and that it asks for what you meant.

5. Run it (if you have the data): if you have a database with these public datasets loaded, run the query there to get results. If not, you have still learned the exact query to use.

6. Refine the prompt: if the query is off, make your request more specific (exact dates, extra filters, the columns you want) and ask again.

Original source

Based on a Hacker News post by Xyra showing how Claude can query large datasets like Hacker News and arXiv by generating SQL from natural-language requests.

Notes & variations

• Free-tier alternative: no database of your own? Use Claude’s free tier just to practice turning questions into SQL against example schemas, a genuinely useful skill on its own.
• Common mistake: a vague request gives generic or wrong SQL. State the topic, the time frame, and the exact columns you want.
• Tip for better results: give Claude the real table and column names if you know them. The more it knows about your data’s structure, the more accurate the query.