MySQL replica connection pool exhausted, hitting live traffic
Entry OS-002 in Global's register.
What is failing. SQLSTATE[HY000] [1226] User 'telecom-comparator' has exceeded the 'max_user_connections' resource (current value: 25) against
replica.mysql.db.infra.production.aws.selectra.io. 10 Sentry shortIds, ~6,000
events, from unrelated call sites — the public API /offers/results, the
Filament dashboard, and background jobs alike.
Why it is here rather than in a patch. It is not one endpoint leaking connections; the limit of 25 is being reached by normal concurrent load from many places at once. There is no small code change that fixes it, and the tempting "fix" — retry or swallow — would hide a capacity problem while making it worse.
Why it is not filed as infra noise. Unlike the one-off replica timeouts this
run classified IGNORE: infra, this reaches live production traffic, not just
background jobs: the two largest contributors (2,970 and 2,718 events) are the
public offers API and the dashboard. A visitor sees this.
What a human needs to decide: whether 25 is simply too low for current traffic,
or whether something is holding connections longer than it should. Sentry shortIds:
TELECOM-COMPARATOR-NK, -CQ, -MY, -P8, -NY, -NX, -P9, -P0, -P4, -P1.
Answer (2026-08-28, users/108934946863892655718) : "OS-002 it's infra, we don't touch. You can check to improve cache to avoid SQL query instead" — read as: raising/tuning the connection pool itself is off the table (infra, not ours to touch); instead, look at whether caching can cut the query volume hitting the replica.
Bookkeeping
- Opened
- 2026-08-27
- Settled
- 2026-08-28
- Announced
- 2026-08-28
- Register
countries/_global/decisions.md- Overdue after
- 30 days
- Reminded every
- 7 days