Container Insights Part 1: An Introduction to cgroups

March 17, 2024

Cgroups, a powerful Linux kernel feature, provide a fine-grained control over system resources. By organizing processes into hierarchical groups, you can allocate, limit, and account for resources like CPU time, memory, disk I/O, and network bandwidth. This guide will delve into the intricacies of cgroups, exploring their configuration, usage, and practical applications in containerization and resource management.

My journey into understanding cgroups began with a simple curiosity: how do containers manage resources so efficiently? This led me down a rabbit hole of exploring cgroups (control groups) and their critical role in resource management for processes in Linux. In this blog, I’ll introduce you to the basics of cgroups using a code example that demonstrates setting resource limits for CPU, memory, I/O, and more.

What are cgroups?

Control groups, or cgroups, are a Linux kernel feature that allows you to allocate, monitor, and control resource usage (such as CPU, memory, and I/O) for groups of processes. In containerized environments like Docker or Kubernetes, cgroups are crucial for ensuring that individual containers don’t consume more than their allocated resources, thereby maintaining system stability and predictability.

Getting Hands-On with cgroups using Go

The following Go code leverages the cgroups/v3 library to demonstrate how to set up and manage cgroups programmatically. Through this example, we’ll explore how to configure resource limits, create a cgroup, and manage processes within it.

Here’s a look at the core components of this cgroup management program:

Setting Resource Limits Each resource (CPU, memory, I/O, and process IDs) has its own configuration function. For example, the setupCPUResources function sets a CPU quota of 200ms per 1000ms (i.e., 20% of CPU usage). Similarly, memory and I/O resources are limited, and the maximum number of processes is capped.
```
 func setupCPUResources() *cgroup2.CPU {
     quota := int64(200000)
     period := uint64(1000000)
     return &cgroup2.CPU{
         Max: cgroup2.NewCPUMax(&quota, &period),
     }
 }
```
Creating the group Once the resources are configured, we create a new cgroup using the createCgroup function, which initializes a systemd cgroup. This allows us to control the resources for a specific group of processes, like in a container environment.
```
 func createCgroup(res *cgroup2.Resources) (*cgroup2.Manager, error) {
 return cgroup2.NewSystemd("/", "my-cgroup-abc.slice", -1, res)
 }
```

Adding Processes to the cgroup A stress command is run to simulate CPU load, and the resulting process is added to the cgroup. This allows us to monitor and control the resource usage of the stress process according to the limits we set.

 func runStressCommand() (*exec.Cmd, error) {
     cmd := exec.Command("stress", "-c", "1", "--timeout", "30")
     err := cmd.Start()
     if err != nil {
         return nil, err
     }
     return cmd, nil
 }

Managing the cgroup With the process in place, we can freeze and thaw the cgroup to temporarily stop and restart process execution. This is particularly useful for testing and debugging resource control mechanisms.

 log.Println("Freezing Process")
 if err := m.Freeze(); err != nil {
     log.Fatalf("Error freezing process: %v\n", err)
 }

 time.Sleep(time.Second * 15)

 log.Println("Thawing Process")
 if err := m.Thaw(); err != nil {
     log.Fatalf("Error thawing process: %v\n", err)
 }

Cleaning Up After the stress command finishes, the cgroup is deleted to free up resources. Proper cleanup is essential to prevent orphaned cgroups from lingering on the system.
```
 if err := m.DeleteSystemd(); err != nil {
 log.Fatalf("Error deleting cgroup: %v\n", err)
 }
```

Why cgroups matter for Containers

Understanding how to use cgroups can help you gain better control over your containerized environments. For instance, by setting appropriate limits, you ensure that no single container can monopolize system resources, protecting overall system stability. The freeze/thaw mechanism also illustrates how container management can benefit from temporarily suspending and resuming processes based on resource availability.

Conclusion

This exploration of cgroups has shown how powerful they are for managing resources in Linux. In containerized environments, cgroups are foundational, ensuring that each container or process adheres to defined resource boundaries. As I dive deeper, I’ll cover more aspects of resource management and how these concepts integrate into broader container orchestration systems like Kubernetes.

Stay tuned for Part 2, where we’ll explore namespaces, another essential building block for container isolation.