Netzbetreiber contact enrichment

You are in the country workspace. Campaign files: campaigns/netzbetreiber/.

Batch selection

1. Read campaigns/netzbetreiber/seed.csv and campaigns/netzbetreiber/enrichment.jsonl (if present). A row is pending when its MaStR-Nr. has no JSONL record.
2. Exclude rows where: Marktrollen contains Übertragungsnetzbetreiber, or Geschlossenes Verteilernetz = Ja, or Tätigkeitsstatus ≠ Aktiv, or Tätigkeitsende is non-empty. (Filter pending confirmation by Stefanie — skip, don't delete.)
3. Take the next 20 pending rows in file order, unless requester instructions say otherwise (a different count, or specific MaStR-Nrs to research/re-research).

Research each operator

Spawn at most 2 parallel subagents (each is a separate multi-GB claude process; the container has a 6 GB memory cap — 4+ concurrent agents OOM the run).

campaigns/netzbetreiber/mastr_prefill.csv holds register data per MaStR-Nr (actor_name, website, email, phone, role_emails) — start from it instead of searching blind. Work operator by operator:

1. Official website. Start from the prefill website; verify it loads and is official: the Impressum must name the operator (or its parent Stadtwerke — many Netz-GmbHs live on the parent's domain; note that in notes). Prefer the Netz entity's own site over the parent's when both exist. Only if the prefill URL is missing, dead, or wrong entity: web-search the operator name (+ Ort).
2. Contact path. Find the contact page/form (Kontakt, Kontaktformular — prefer a general or grid/Netz contact over sales). Record form_url only if an actual form is there, not just address info.
3. Anti-bot. Fetch the form page and look for markers in the HTML. MEMORY SAFETY: when searching saved HTML, use fixed-string searches (grep -F -c 'g-recaptcha') or read the file directly — NEVER wide-context regexes like .{0,150}pattern.{0,150} or unbounded .*pattern.* with -o: on minified single-line HTML these allocate gigabytes and have OOM-killed the host (2026-07-22). Markers to check: reCAPTCHA (g-recaptcha, recaptcha), hCaptcha, Cloudflare Turnstile (cf-turnstile), Friendly Captcha. A blocked/challenged fetch (403, "checking your browser") ⇒ cloudflare_challenge. Nothing found ⇒ none.
4. Form fields. If a form is readable, map its fields: {"field name/id": "what to fill (semantic)"}, and classify form_type: plain_post (regular HTML form), js (rendered/submitted by JavaScript), iframe, unknown.
5. Email. Look on Kontakt page and Impressum. Record the most general contact address (info@/kontakt@/netz@). Never guess an address (hard rule 1). The prefill email/role_emails stay in the prefill CSV (they are register contacts, often department-specific — netzbetreiber@, marktpartner@, einspeiser@ — or personal); if the site publishes nothing, email may stay null even when the prefill has one. Note contradictions (e.g. bounced domain) in notes.
6. mastr_public_contact: set true when the prefill CSV has email or website for this MaStR-Nr — do not scrape marktstammdatenregister.de (JS SPA, unreliable).

Record

Append one JSON object per operator (one line) to enrichment.jsonl:

{"mastr_nr": "SNB...", "name": "<from seed>", "website": "https://... or null",
 "form_url": "https://... or null", "email": "... or null",
 "anti_bot": "none|recaptcha|hcaptcha|turnstile|friendly_captcha|cloudflare_challenge|other|unknown",
 "form_type": "plain_post|js|iframe|unknown|null", "form_fields": {...} or null,
 "contact_status": "enriched|no_contact_found|needs_human",
 "source": "website|mastr|search", "mastr_public_contact": true|false|null,
 "checked_at": "<today YYYY-MM-DD>", "notes": "... or null"}

contact_status: enriched = at least one usable channel (form_url or email); no_contact_found = nothing verifiable; needs_human = something is there but a human must look (ambiguous entity, login-walled portal, contradictory info).

Finish

1. Run node campaigns/netzbetreiber/export_csv.mjs to regenerate enrichment.csv (Node port of export_csv.py; the container has no python3).
2. Journal entry (see country CLAUDE.md).
3. Final message: batch size, how many enriched / no_contact_found / needs_human, form vs email availability counts, and anything odd worth a human's attention.