A cluster rotates webhook serving certs in place and later admission failures appear even though the pod and service remain healthy.
A Kubernetes admission webhook serves valid TLS and requests still fail because the API server cached an older CA bundle after the webhook secret rotated with the same service name
The webhook pod is healthy, yet admission calls break because the client trust bundle lagged behind the newly issued serving certificate.
Scenario
What to check first
- Identify the primary failure signal in the The Webhook Rotated Certs Faster Than the API Server Rotated Trust scenario.
- Separate visible symptoms from the underlying technical dependency.
- Describe the safest recovery path and the follow-up prevention work.
Checking checklist
- Summarize the current impact and the last known change.
- Collect direct evidence from logs, runtime state, and configuration before changing anything.
- Separate immediate recovery from permanent prevention work.
Recovery and prevention
Compare the webhook serving chain with the CA bundle currently observed by the API server before deleting the webhook.
Questions worth viewing together
Community-field security problem inspired by Discuss Kubernetes patterns where admission webhooks failed after cert rotation due to stale CA trust. Healthy webhook pods do not prove the API server trusts the current certificate chain.
Teams often blame DNS or service routing when the trust bundle is the lagging component.
Webhook certificate rotation should verify client trust refresh, not only secret and pod rollout success.
Similar cases seen in the field