A monitoring credential is rotated and only one long-lived shipper path continues to see 401 responses.
An Elastic API key rotation succeeds and one Beats pipeline still gets 401
The key is updated, yet one data shipper keeps failing because it only read the secret store at startup.
Scenario
What to check first
- Identify the primary failure signal in the IAM scenario.
- Separate visible symptoms from the underlying technical dependency.
- Describe the safest recovery path and the follow-up prevention work.
Checking checklist
- Summarize the current impact and the last known change.
- Collect direct evidence from logs, runtime state, and configuration before changing anything.
- Separate immediate recovery from permanent prevention work.
Recovery and prevention
Check secret reload semantics of the running process before minting another API key.
Questions worth viewing together
Community-field security problem inspired by Elastic community patterns where a Beats pipeline kept using an old API key after keystore rotation. Key rotation success in storage does not prove the runtime consumer has reopened the new secret source.
Teams often blame invalid API key scopes when the process is still using the previous cached key.
Credential rotation should validate process reload behavior, not only secret distribution.
Similar cases seen in the field