wiki_rewrite: paginate fully + extend rewriter to wikivoyage

The Twin Falls Wikivoyage investigation surfaced two coordinated gaps that kept
OSM wikivoyage tags from rewriting to local Kiwix URLs even though the
wikivoyage ZIM is loaded and serving.

Fix 1 (pagination) -- wiki_rewrite.py: append ?count=-1 to the Kiwix OPDS catalog
fetch. kiwix-serve's /catalog/v2/entries defaults to the first 10 entries; the
library has 17, so wikivoyage (and other page-2 ZIMs) were never seen by
_discover_zims and never entered _zim_map, so their tags always fell back to public.

Fix 2 (tag passthrough) -- place_detail.py: add wikivoyage to both nominatim
extratags whitelists, mirroring the existing wikipedia/wikidata lines. The
rewriter (classify_wiki_link / build_kiwix_url / rewrite_wiki_link) and the
_enrich_wiki_links loop were already source_type-generic and covered wikivoyage;
the only missing link was that the nominatim parser dropped the wikivoyage tag
before enrichment ever saw it. No rewriter refactor was needed.

Tests (test_place.py):
- test_catalog_url_requests_full_library: the OPDS fetch URL contains count=-1.
- test_wikivoyage_tag_rewrites_to_local: a wikivoyage OSM tag for a mirrored
  article rewrites to https://wiki.echo6.co/content/wikivoyage_en_all_maxi_<date>/...
  with sources.wiki_rewrites.wikivoyage == "local" (Kiwix mocked).
Full navi-places suite: 15 passed.

Follow-up (separate ops step, not in this PR): prune 3 dangling library.xml
entries on the Kiwix host (wikiloc.com_eng_2026-04_18, meshtastic.org_eng_2026-04_14,
meshtastic.org_eng_2026-04_15) whose ZIM files are absent; kiwix-serve silently
skips them.

Note: Twin Falls (R/121355) itself has no OSM wikivoyage tag, so it still will not
get a local Wikivoyage link from tag rewrite -- that needs the separate name-based
discovery feature (discover_wikivoyage_article stub).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt 2026-05-26 06:23:24 +00:00
commit f537a55266
3 changed files with 40 additions and 1 deletions

View file

@ -291,6 +291,7 @@ def _parse_nominatim(data):
'email': raw_extra.get('email') or raw_extra.get('contact:email'),
'wikipedia': raw_extra.get('wikipedia'),
'wikidata': raw_extra.get('wikidata'),
'wikivoyage': raw_extra.get('wikivoyage'),
'cuisine': raw_extra.get('cuisine'),
'operator': raw_extra.get('operator'),
'wheelchair': raw_extra.get('wheelchair'),
@ -376,6 +377,7 @@ def _parse_overpass(data, osm_type, osm_id):
'email': tags.get('email') or tags.get('contact:email'),
'wikipedia': tags.get('wikipedia'),
'wikidata': tags.get('wikidata'),
'wikivoyage': tags.get('wikivoyage'),
'cuisine': tags.get('cuisine'),
'operator': tags.get('operator'),
'wheelchair': tags.get('wheelchair'),