feat(wiki-index): add wave 2 pipeline for wikidata-only places

Browse source

Processes places with wikidata but no wikipedia tag:
- Batch resolve Q-IDs via Wikidata API (50/request)
- Validate resolved titles against local ZIM
- Generate summaries with Gemini API (3-4 sentences)
- Circuit breaker: 50 consecutive 429s triggers 5min pause
- Revalidate any remaining unvalidated entries

Filters for US+CA places, skips existing wave 1 entries.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

...

This commit is contained in:

Matt 2026-04-29 19:50:43 +00:00

parent b250d0c257

commit 6be1e4cfa6

1 changed files with 1128 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1128

scripts/wiki_index_wave2.py Executable file

View file

File diff suppressed because it is too large Load diff