PM-SAMPLE-001checkout-api — p99 latency regressionAwaiting human approval
Conclusion
The connection-pool ceiling was cut from 80 to 20 in deploy a83f9c, starving checkout-api under normal traffic.
p99 410 ms → 2.9 s · onset 14:23 UTC · seconds after deploy
Evidence
E-01
p99 request latency stepped from 410 ms to 2.9 s and held.
metrics · http_request_duration_seconds · 14:23:11 UTC
E-02
DB_POOL_MAX reduced 80 → 20 in the deployed values file.deploy diff · checkout-api@a83f9c · 14:23:14 UTC
E-03
Pool exhaustion in application logs: 0 idle connections at the new limit,
timeout_ms=2000.logs · checkout-api · 2,140 matching lines · 14:23:19 UTC
E-04
Cache hit rate within baseline and upstream latency unchanged across the window.
metrics · cache and upstream telemetry · 14:27:42 UTC
Considered · rejectedCache degradation
Cache degradation does not match the evidence — the hit rate held within baseline through the entire regression window [E-04], and no cache-layer change was deployed. Discarded before the conclusion was formed.
Confidence
87%
4 of 4 supporting claims independently sourced. 3 alternatives tested and rejected.
Proposed action
Restore the previously reviewed DB_POOL_MAX value, then observe p99 and pool errors for ten minutes.
Human approval required
Production Master will not apply this change itself. It waits here.
Sources consulted
- deploy history1 commit
- metrics3 series
- application logs2,140 lines
- cluster events18 events