de:wasserversorger-discover

Journal entry appended. Run complete — final report below.

Ran a clean phase-2b discovery batch of 60 Gemeindeverband units (2 synchronous shards, no background-and-wait failures). Population coverage rose 83.29% → 84.32% (+856k people, the largest single-run gain since phase 2b started), municipalities claimed 7,771 → 7,943, residual dropped under 3,000 municipalities (13.11M people) for the first time. Roster grew 1,685 → 1,740 companies (55 net new after both shards self-caught and merged 23 duplicate mints), zero new domain collisions or duplicate company_ids introduced, and the queue/journal were rebuilt for the next run — the main open item is provisioning pdftotext/poppler for scanned Satzung PDFs, which blocked a couple of lookups this run and in prior ones.

Scheduled run, clean throughout — no process failures, no background-and-wait incidents. Read SKILL.md and the country CLAUDE.md first, then confirmed the on-disk state matched the prior journal entry exactly (1685 companies, 83.29% coverage) before doing anything, so no recovery work was needed. Re-ran build_gv_layer.mjs fresh (512→515 multi-member units, 1885 singletons, close to the last run's count) and took the top 60 units. Wrote a small one-off helper, build_batch_input.mjs, to generate the two shard input JSONs directly from gemeindeverband-queue.csv + gemeindeverband-members.csv (re-deriving the same claimed-AGS set build_gv_layer.mjs uses) rather than hand-assembling them — worth reusing/keeping for future runs instead of re-deriving this by hand each time.

Batch: phase2b-vg4-shardA-input.json (30 units, Auenland Südholstein 7.4k down to Kappeln-Land 1.4k residual pop — the top of the queue, mostly SH Ämter and BY VGem) and phase2b-vg4-shardB-input.json (30 units, Mittelholstein down to GVV Bönnigheim — a mix of a few more SH Ämter plus a long tail of Baden-Württemberg VVG/GVV units that were already mostly-claimed, only 3-4 residual members each). Dispatched 2 synchronous subagents, each with the mandatory "no Agent/Task tool" line, full skill context, and shard-specific notes (shard A: don't re-crawl Auenland Südholstein's VG site a 4th time; shard B: the BW VVG/GVV units are mostly top-ups, check each residual town's own Stadtwerke first).

Shard A (Auenland Südholstein → Kappeln-Land): 96/120 municipalities (80%) resolved, 24 unresolved. 24 of 30 units fully resolved, 4 partial, 2 fully unresolved (Gerolzhofen VGem, Krumbach VGem). Notable self-catch: minted 34 companies on the first pass, then a collision check against roster.jsonl found 23 already existed under earlier IDs (e.g. its "Trinkwasserzweckverband Thüringer Becken" = existing wv-de-912, its "Wasserverband Krempermarsch" = existing wv-de-907) — it rewrote all 120 links to the pre-existing IDs and dropped the 23 dupes before reporting, leaving 11 genuinely new companies (wv-de-6003, 6004, 6010, 6013–6017, 6026, 6031, 6034). This is a bigger duplicate-catch than prior runs have reported and worth noting as a pattern: a shard given units scattered across many Länder is more likely to rediscover an existing supplier under a new name than a shard given a geographically clustered batch — Thüringer Becken and Krempermarsch were both already in the roster from entirely different discovery angles. Also caught a second-order issue: independently reproduced a previously-flagged-and-corrected hallucination (wv-de-1415 Wiesenbachgruppe claiming Aletshausen/ Deisenhausen/Ebershausen/Waltenhausen as members, already noted in the roster as contradicted by BayernPortal) from three fresh queries before finding the existing correction note — downgraded those 4 links back to unresolved rather than re-propagate it. New host trap: schmalfeld.net has a TLS cert mismatch (issued for kasserver.com); schmalfeld.de redirects to an unrelated personal site.

Shard B (Mittelholstein → GVV Bönnigheim): 76/92 municipalities (83%) resolved, 16 unresolved — 8 with a real negative finding (Amt Kellinghusen's own site states 4 of its villages have no central supply, i.e. private wells, a genuine self_only-adjacent negative rather than a search failure; Amt Mittelholstein's other 4 got real effort across 4 candidate Verbände plus two unreadable scanned PDFs with no match), 8 genuine per-municipality gaps in BW after checking neighboring Verbände's own member lists. 44 new companies (wv-de-6501–6544), 9 links correctly reused pre-existing roster companies (FIWA, Stadtwerke Radolfzell/Kiel/Gaggenau, Mühlbach, ASG, Kleiner Heuberg, Überlandwerk Leinetal, Adelburggruppe) after checking first — zero duplicate mints this shard, and it cross-checked its mints against shard A's fresh IDs (6001–6024) for zero overlap despite running concurrently in a separate context. Handled real fragmentation cases cleanly (Frankenhardt, Fluorn-Winzeln, Oberndorf am Neckar, Wangen im Allgäu all split across multiple non-deduped suppliers per the overlap rule). New host trap: hardtgruppe.de (Sandhausen/Walldorf/Leimen near Heidelberg) is a same-named but unrelated Zweckverband, easily confused with the actual target "Hardt-Wasserversorgungsgruppe" (Aspach/Kirchberg/Marbach) — caught and noted to prevent future conflation. Tooling gap flagged: no pdftotext/poppler in this sandbox and no permission to apt-get install it — blocked 2 scanned-PDF lookups outright (Hohenwestedt's Wasserversorgungssatzung, an SH LWBV Wasserverbände list). SKILL.md explicitly expects scanned Satzung PDFs to be common; worth provisioning poppler-utils for future discovery runs, or documenting a WebFetch-based workaround, since this will keep recurring.

Merge/finish (done by me, orchestrator, after both shards reported):

• Verified both shard output files existed and were non-trivial (11+120 rows, 44+97 rows) and jq-validated cleanly before trusting either report.
• Domain-collision check across both new shard roster files + existing roster.jsonl before merging: 0 duplicate company_ids, same 5 known pre-existing domain collisions (sw-augsburg.de, wvv.de, zvo.com, vgrd.de, emkendorf.de), nothing new introduced.
• map_roster.mjs: 9084 total link rows (+225 vs. prior 8859), 8300 preresolved + 586 exact_norm
  • 61 land_scoped + 3 expanded_qualifier + 3 ambiguous + 127 unmatched (+1 vs. prior 126 — one genuinely new unmatched name, not a regression).
• consolidate_roster.mjs: 1771 shard rows → 1740 distinct companies (+55 net vs. 1685 going in — 11 shard A + 44 shard B, 0 lost to accidental collapse).
• coverage.mjs: population 83.29% → 84.32% (+1.03pp, +856k people — the largest single-run jump since phase 2b started, likely because shard A's 24 SH/BY units resolved cleanly and shard B's BW top-up units were cheap wins on units already mostly claimed). Municipalities claimed 7,771 → 7,943 (+172). Residual: 3,172 → 3,000 municipalities, 13.96M → 13.11M people.
• build_gv_layer.mjs (fresh queue for next run): multi-member units 515 → 462 (covering 1,109 municipalities, was 1,287), singletons 1,885 → 1,891. New top of queue: Gerolzhofen VGem, Krumbach (Schwaben) VGem (both shard A's fully-unresolved units, immediately at the top again — a future run should try a genuinely different method, not re-crawl the same VG sites), Heeseberg, Kropp-Stapelholm, Mittelholstein (shard A/B's partial leftovers).
• Final domain-collision check across roster.jsonl + companies.jsonl (skill Finish step 2): same 5 known collisions, nothing new.

For next run: gemeindeverband-queue.csv/gemeindeverband-members.csv freshly rebuilt as of this run. Residual is now under 3,000 municipalities and 13.11M people (12.7% of the population) for the first time. company_id max on disk after this run: 6544 (shard B's range) — check roster.jsonl before minting next time, several ranges (6001-6499, 6501-6999) are now partially used. Genuinely hard leftovers to try a different angle on: Gerolzhofen VGem, Krumbach (Schwaben) VGem (Wiesenbachgruppe's 4 villages — the true supplier is still unknown after 2 runs' worth of attempts and one hallucination already corrected once), Heeseberg, Kropp-Stapelholm (Klein Rheide specifically), Untersteinach VGem's Guttenberg. Amt Mittelholstein's 4 remaining villages have two unreadable scanned Satzung PDFs blocking them — a pdftotext/poppler-utils install (or a documented OCR/rendering workaround) would likely unblock this and future scanned-PDF cases; this has now been hit by multiple runs (see also the 2026-08-18 Herrstein-Rhaunen note in this journal). New one-off script build_batch_input.mjs (generates shard input JSON from the queue+members CSVs) left on disk in campaigns/wasserversorger/ — reusable for future batches instead of hand-rolling.