Skip to main content

Kubernetes Core Basics

Kubernetes Bin Packing Strategies

First lest understand what is bin packing.

In simple terms, bin packing is a way of arranging items so that you use the least number of containers (or bins) possible.

Imagine you have a bunch of items of different sizes and you need to pack them into boxes. Bin packing is about figuring out the best way to put those items into the boxes so that you use as few boxes as possible.

Here’s a basic example:

  • You have 10 items of different sizes and you want to pack them into boxes.
  • If you put the largest items in the boxes first and then fill the remaining space with smaller items, you might find you can fit everything into fewer boxes.

In computing and Kubernetes, bin packing works similarly but with resources.

It means trying to use the available resources on each server or node as efficiently as possible, packing as many workloads as you can onto each node to minimize the number of nodes needed.

NodeResourcesFit Scheduling Plugin

As we learned in the scheduler section, the scheduler performs a filtering operation to determine if a node has enough resources to accommodate a pod.

The NodeResourcesFit plugin is part of the Kubernetes scheduler framework. Its primary function is to check if a node has sufficient resources to meet a pod's resource requests.

For scoring, the NodeResourcesFit plugin can be configured to use any of the following scoring strategies:

  1. LeastAllocated (default strategy)
  2. MostAllocated
  3. RequestedToCapacityRatio

Lets understand each one in detail.

LeastAllocated Strategy

It is the default scoring policy used by the NodeResourcesFit plugin.

This strategy aims to spread out the workload evenly across nodes to prevent any single node from being overloaded. It prefers nodes with the least amount of already used resources (sparse distribution of pods).

Unlike bin packing, which tries to put as many workloads as possible onto fewer nodes, LeastAllocated focuses on spreading out the workloads.

It means the LeastAllocated does notspecifically aim to maximize the utilization of cluster nodes

To achieve maximum resource utilization in the default strategy, you need to follow a combination of deployment strategies (although it doest guarantee any bin packing).

For example,

  1. Set Resource Requests and Limits: Define how much CPU and memory each pod needs.
  2. Use Node Affinity and Anti-Affinity: Control where pods can be placed based on node labels.
  3. Apply Taints and Tolerations: Manage which pods can run on which nodes.
  4. Pod Topology Spread Constraints: Ensure that pods are distributed across nodes in a balanced way.

MostAllocated Strategy (Bin Packing)

The MostAllocated strategy is used to choose nodes that already have the most resources used.

When a new pod is created, the scheduler will pick a node that is already using more resources but still has enough available resources for the new pod. This approach helps improve the overall resource use in the cluster over time.

As this is not the default strategy, you need to modify the scheduler configuration to enable this policy.

For example.

apiVersion: kubescheduler.config.k8s.io/v1beta1
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: default-scheduler
    plugins:
      score:
        enabled:
          - name: MostAllocated
Note: When it comes to managed Kubernetes clusters like EKS, you cannot modify the policy. To enable this in managed clusters, you will have to create custom schedulers with the MostAllocated scoring policy.

RequestedToCapacityRatio Strategy (Bin Packing)

RequestedToCapacityRatio is a way for Kubernetes to decide which node ( is best for a new pod based on how well the node's resources match the pod's needs.

Imagine you have a node with 10 CPUs and a pod that needs 2 CPUs. The RequestedToCapacityRatio strategy looks at how well the requested 2 CPUs fit with the node’s 10 CPUs.

This strategy scores each node based on

  • How much of the node’s resources are already used (most allocated).
  • Nodes that are more heavily used but still have enough resources available for the new pod may get a higher score.

This policy favors nodes with higher resource utilization, meaning it prefers nodes that are already using a lot of their available resources but still have enough left for the new pod. This approach helps in bin packing

Also, you can give different importance (weights) to different resources like CPUs and memory. For example, you might decide that CPUs are more important than memory for a particular pod.

Here is an example scheduler configuration for RequestedToCapacityRatio

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
  - pluginConfig:
      - args:
          scoringStrategy:
            resources:
              - name: gpu  # Resource type for GPUs
                weight: 3  # Importance of GPUs
              - name: ssd  # Resource type for SSDs
                weight: 3  # Importance of SSDs
            requestedToCapacityRatio:
              shape:
                - utilization: 0
                  score: 0  # Score for no usage
                - utilization: 100
                  score: 10  # Score for full usage
            type: RequestedToCapacityRatio
        name: NodeResourcesFit

Lets look at a real world example.

Imagine you have a machine learning cluster where each node has GPUs and SSDs, and you want to deploy a new model.

Here’s how the scheduler configuration works in simple terms

  • GPU: Critical for training your model, so it's given a high weight (3).
  • SSD: Important for fast data access, also given a high weight (3).

The scheduler checks each node based on how well its current GPU and SSD usage fits the new model’s needs.

  • If a node is not using GPUs or SSDs (0% utilization), it gets a low score (0).
  • If a node is fully using its GPUs and SSDs (100% utilization), it gets a high score (10).

The scheduler prefers nodes that have a good balance between their resource usage and what the model needs.

For Example:

  • Node A: Has 4 GPUs and 2 SSDs, currently using 2 GPUs and 1 SSD. It might get a moderate score because it’s somewhat used.
  • Node B: Has 8 GPUs and 4 SSDs, currently using 7 GPUs and 3 SSDs. It might get a higher score if its resource usage is well-aligned with the model’s needs.

As this policy favours nodes with maximum reource utlization, it supports bin packing of pods.