Unable to Resolve DNS in Kubenretes
When you try to access service endpoints inside kubernetes, you might not be able to resolve the DNS saying
This error indicates a DNS resolution issue within the Kubernetes cluster. Here are some steps to diagnose and resolve the problem:
Check DNS Configuration in the Pod
Verify that the pod has the correct DNS settings.
Get a exec session to the pod using the following command.
kubectl run test-pod --rm -it --image=busybox --restart=Never -- /bin/sh
Run the following command inside your pod to check the /etc/resolv.conf
file
cat /etc/resolv.conf
Ensure that the nameserver
is set to the correct Kubernetes DNS service IP (usually 10.96.0.10
in a default setup).
Check CoreDNS
Ensure that the CoreDNS service is running and healthy.
kubectl get pods -n kube-system -l k8s-app=kube-dns
All DNS pods should be running without errors.
Look at DNS Logs
If CoreDNS is running but DNS resolution is still failing, inspect the logs for any errors.
kubectl logs -n kube-system -l k8s-app=kube-dns
Look for any warnings or errors that might indicate why DNS queries are failing.
Restart CoreDNS
If there are no issues witht he above steps, restarting the CoreDNS pods can resolve transient issues.
kubectl rollout restart deployment coredns -n kube-system
After the restart, try running wget to kubernetes service and see if you are able to reach the service endpoint and get a forbidden error error given below.
$ k run test-pod --rm -it --image=busybox --restart=Never -- wget https://kubernetes.default.svc.cluster.local
If you don't see a command prompt, try pressing enter.
Connecting to kubernetes.default.svc.cluster.local (10.96.0.1:443)
wget: note: TLS certificate validation not implemented
wget: server returned error: HTTP/1.1 403 Forbidden