Possible race condition? #597
Description
Kubernetes Version
Server Version: v1.30.0
Helm Chart Version
ngrok-operator-0.17.0
Operator Configuration
Not found?
System Info
Running Kind k8s on Podman.
Kubernetes Logs
W0301 17:43:31.805641 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:43:31.805847 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:15.232620 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:44:15.232781 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:31.612877 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:44:31.613076 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:45:10.256770 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:45:10.256996 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:45:28.390500 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:45:28.390596 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:43:25.200889 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:43:25.200995 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:08.761275 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:44:08.761392 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:17.623866 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:44:17.623911 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:47.660890 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:44:47.661044 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:44:52.912170 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:44:52.912222 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
Helm Chart configuration
Nothing extra supplied, only the credentials.
What happened
Looks like I got into a race condition after reinstalling and installing the chart multiple times. I could not reproduce the issue, but the operator get stuck and can't find resources like domain and etc.. even if they were created in the past
Please refer to the output below
kubectl logs ngrok-ingress-controller-ngrok-operator-agent-6bdd9fd9cd-krszv -n ngrok-ingress-controller
W0301 17:40:06.933194 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:40:06.933306 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:40:27.151005 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)
E0301 17:40:27.151212 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.Tunnel: failed to list *v1alpha1.Tunnel: the server could not find the requested resource (get tunnels.ingress.k8s.ngrok.com)" logger="UnhandledError"
W0301 17:40:42.377929 1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)
E0301 17:40:42.378072 1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1alpha1.AgentEndpoint: failed to list *v1alpha1.AgentEndpoint: the server could not find the requested resource (get agentendpoints.ngrok.k8s.ngrok.com)" logger="UnhandledError"
AWS: admin-sandbox felipe@workmbp ~/et/personal/ngrok-operator/helm/ngrok-operator main kubectl get ing -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
ngrok-ingress-controller game-2048-ingress ngrok bass-immortal-notably.ngrok-free.app bass-immortal-notably.ngrok-free.app 80 24h
I'm really not doing much science here, just trying to get the default app running. It worked and the beginning but after reinstalling the chart, it broke beyond repair.
What you think should happen instead
I believe the operator should be able to reconcile properly in this situations.
How to reproduce
Unfortunately I could not consistently reproduce this error.