Topic91 problems· 3 reviewed

Service Discovery

91 incident problems about Service Discovery. Start with the reviewed ones.

All problems (91)

K8S-1458An ExternalDNS rollout updates public records and one cluster path still flapsAn ExternalDNS rollout updates public records and one cluster path still flaps focuses on Service Discovery and asks the reader to isolate the key signal in AWS. Record flapping often comes from ownership-marker collisions rather than from...KubernetesIntermediate10 minProNETWORK-1475An MLAG pair keeps forwarding and one orphan host loses return trafficAn orphan host works one way after a migration while the pair otherwise looks healthy.NetworkIntermediate10 minProK8S-1487A cert-manager DNS01 challenge updates Route53 and still times outACME issuance fails intermittently only in hosted zones managed by both cert-manager and ExternalDNS.KubernetesIntermediate11 minProK8S-1467A Grafana Agent Flow pipeline duplicates one namespaceOne namespace begins double-counting metrics after a Grafana Agent Flow cleanup merged pod and service discovery blocks.KubernetesIntermediate11 minProNETWORK-1410A service discovery cutover finishes and clients keep dialing the old backendA service discovery cutover finishes and clients keep dialing the old backend focuses on Service Discovery and asks the reader to isolate the key signal in ubuntu. Negative caches can outlive the moment a record appears and make a correct...NetworkIntermediate11 minProNETWORK-1420A service discovery SRV cutover looks correct in dig and fails in appsA service discovery SRV cutover looks correct in dig and fails in apps focuses on Service Discovery and asks the reader to isolate the key signal in ubuntu. SRV migrations can fail on target-label caches even when the top-level SRV record a...NetworkIntermediate11 minProNETWORK-1430A service discovery SRV cutover looks correct in dig and still fails in appsA service discovery SRV cutover looks correct in dig and still fails in apps focuses on Service Discovery and asks the reader to isolate the key signal in ubuntu. SRV migrations can fail at the target-label layer even when the top-level SRV reco...NetworkIntermediate11 minProNETWORK-1450An SRV cutover looks correct in dig output and still fails in appsApplications keep failing after a service-discovery cutover even though direct queries show the intended answers.NetworkIntermediate11 minProNETWORK-1440An SRV cutover looks correct in direct dig and still fails in appsAn SRV cutover looks correct in direct dig and still fails in apps focuses on Service Discovery and asks the reader to isolate the key signal in ubuntu. SRV migrations can fail at the target-label level even when the top-level record already loo...NetworkIntermediate11 minProK8S-1465An IPVS service keeps sending traffic to a drained nodeTraffic continues to hit a drained backend for several minutes after a rollout with externalTrafficPolicy Local.KubernetesAdvanced12 minProK8S-1452A Cilium node-local redirect keeps readiness green and still breaks source-IP...A Cilium node-local redirect keeps readiness green and still breaks source... focuses on Service Discovery and asks the reader to isolate the key signal in cilium. Service redirects can change enforcement identity even when requests st...KubernetesAdvanced13 minProK8S-1447A Cilium FQDN policy allows startup traffic and blocks steady-state pullsImage pulls succeed at startup and fail later after node-local DNS takes over.KubernetesAdvanced14 minProK8S-1606A CoreDNS rewrite fix lands and one namespace still resolves old namesOne namespace still resolves old names after a CoreDNS rewrite fix.KubernetesIntermediate8 minProK8S-1586A CoreDNS rewrite fix lands and one namespace still resolves stale namesOne namespace still resolves stale names after a CoreDNS rewrite fix.KubernetesIntermediate8 minProK8S-1596A CoreDNS rewrite is fixed and one namespace still resolves stale namesOne namespace still resolves stale names after a CoreDNS rewrite fix.KubernetesIntermediate8 minProK8S-1576A CoreDNS rewrite rule changes and one service still resolves the old suffixOne service keeps resolving the old suffix after a CoreDNS rewrite update.KubernetesIntermediate8 minProCICD-1579A Spinnaker webhook stage points to the new endpoint and one region still calls the old hostOne webhook stage keeps calling the old host after endpoint migration.CI/CDIntermediate8 minProK8S-1483A CoreDNS autopath optimization speeds most lookups and one namespace loopsA migrated namespace sees intermittent DNS loops after autopath optimization is enabled cluster-wide.KubernetesIntermediate9 minProK8S-1516A CoreDNS change is applied and one namespace still resolves stale IPsOnly one namespace behind one node-local cache keeps seeing stale service IPs after a CoreDNS update.KubernetesIntermediate9 minProK8S-1474A CoreDNS negative cache hides a fixed ServiceA service recovers, yet callers keep failing for several minutes because the negative DNS result is still cached.KubernetesIntermediate9 minPro