Kubernetes namespaces Stuck in the terminating State

When you delete a namespace in Kubernetes, the cluster doesn’t remove it immediately. Instead, it enters a Terminating state, during which Kubernetes attempts to clean up all resources within that namespace. This process involves:

1. Removing all resources: Pods, services, deployments, etc.
2. Executing Finalizers: Special functions that ensure resources are properly cleaned before deletion.

If everything goes smoothly, the namespace is fully deleted after the cleanup. However, if Kubernetes encounters issues during this process, the namespace can remain stuck in the Terminating state as shown below.

Common Causes for Namespaces Stuck in Terminating

Several factors can cause a namespace to get stuck during termination:

1. Finalizers Not Completing: Finalizers are intended to perform cleanup tasks. If a finalizer hangs or fails, the namespace deletion halts.
2. Orphaned Resources: Some resources may not be deleted properly due to misconfigurations or errors.
3. API Server Issues: Problems with the Kubernetes API server can interfere with namespace deletion.
4. Third-Party Controllers: External controllers or operators might block namespace deletion by not handling finalizers correctly.

Diagnosing the Issue

Before attempting to fix the problem, it's essential to understand why the namespace is stuck.

Here's how you can diagnose the issue:

Check For Finalizers

A finalizer in Kubernetes is like a safety check that ensures certain tasks are completed before a resource (like a namespace) is fully deleted. Think of it as a "to-do list" that Kubernetes must finish before removing something permanently.

The namespace has a finalizer called kubernetes that ensures all resources within the namespace (like pods, services, deployments, etc.) are properly deleted before the namespace itself is removed.

This prevents orphaned resources from lingering in the cluster, which could consume resources unnecessarily or cause conflicts.

Check for finalizers field using the following command. If it contains entries, they might be preventing deletion.

kubectl get namespace <namespace-name> -o json | jq '.spec.finalizers'

If there are finalizers present, they might be the culprits.

For example,

$ k get  ns payment -o json | jq '.spec.finalizers'

[
  "kubernetes"
]

If there are any finalizers, remove it by editing the namespace.

kubectl edit namespace <namespace-name>

Find the finalizers field and remove it entirely. For example:

List & Force Delete Remaining Resources

The following command lists all resources within the namespace. Any lingering resources can block deletion.

kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind --ignore-not-found -n <namespace-name>

If there are any resources left, you can force delete them using the following command.

kubectl delete <resource-type> --all -n <namespace-name> --force --grace-period=0

# Example

kubectl delete po --all -n frontend --force --grace-period=0

Now, force delete the namespace.

kubectl delete ns payment --force