GitHub Actions Self-Hosted Runner Docker Memory Leak Ubuntu 22.04 Fix

Your CI pipeline ran fine for three weeks. Then memory usage climbed to 98%, jobs started hanging, and the runner process became unresponsive. Not a fluke. Not bad luck.

The GitHub Actions self-hosted runner Docker container memory leak on Ubuntu 22.04 has become one of the most searched CI/CD troubleshooting topics in early 2026 — and for good reason. Teams migrating from GitHub-hosted runners to self-hosted infrastructure on Ubuntu 22.04 are discovering that Docker’s interaction with the runner process creates a specific, reproducible memory accumulation pattern that doesn’t resolve on its own.

This piece breaks down the root cause, the fix options available, and which approach fits your infrastructure setup.

In brief: Memory leaks in GitHub Actions self-hosted runners running Docker containers on Ubuntu 22.04 stem from a combination of zombie container processes and uncleaned overlay filesystems — not the runner binary itself. The fix requires both a Docker daemon configuration change and a runner lifecycle hook.

Key points covered:

1. Docker’s --network=host and orphaned container chains are the primary culprits, not runner memory mismanagement.
2. Three mitigation approaches exist: daemon-level cleanup, runner group isolation, and kernel cgroup v2 enforcement.
3. Ubuntu 22.04’s default cgroup v2 configuration introduces specific memory accounting behavior that amplifies the issue compared to 20.04.

Why Ubuntu 22.04 and Docker Don’t Always Play Nicely

Ubuntu 22.04 LTS shipped with cgroup v2 enabled by default — a significant shift from 20.04’s cgroup v1 default. That single change altered how Docker handles memory accounting. The kernel now tracks memory for both anonymous pages and kernel slabs per-container, which means container teardown doesn’t always release memory back to the host immediately.

GitHub Actions runners have compounded this. The runner binary (v2.316+ as of Q1 2026) manages job execution by spawning Docker containers per workflow step. When a job completes, the runner sends a docker stop signal. But if the container had active child processes — particularly docker-in-docker (DinD) setups or containers running systemd — the container often transitions to a zombie state rather than a clean exit.

A documented issue thread on the actions/runner GitHub repository (issue #2658, referenced by multiple contributors through late 2025) traced this zombie container pattern to memory accumulation of roughly 150–400MB per failed or force-killed job, depending on container image size. On a t3.medium EC2 instance with 4GB RAM, that’s a real problem after 10–15 CI runs.

The oneuptime.com self-hosted runner setup guide from January 2026 notes that Ubuntu 22.04 installs require explicit Docker daemon configuration to prevent this — something GitHub’s own quick-start documentation skips entirely.

The Three Root Causes

Orphaned Overlay Filesystems

Each Docker container on Ubuntu 22.04 uses the overlay2 storage driver by default. When a runner job exits uncleanly — timeout, OOM kill, workflow cancellation — the container’s overlay mount points persist in /var/lib/docker/overlay2/. Not leaked containers exactly. Leaked layers.

Run du -sh /var/lib/docker/overlay2/ on a runner that’s processed 50+ jobs without a restart. On a standard setup, that directory often reaches 8–15GB. The memory problem comes from the kernel page cache holding references to those overlay inodes. The OS doesn’t evict them aggressively.

The fix: add "storage-opts": ["overlay2.override_kernel_check=false"] to /etc/docker/daemon.json and enable "log-opts": {"max-size": "10m", "max-file": "3"} to prevent log file accumulation from compounding the issue. Then run docker system prune -f as a post-job hook.

cgroup v2 Memory Accounting Gaps

On Ubuntu 22.04 with cgroup v2, the memory.stat file reports cache separately from anonymous memory. Docker’s stats endpoint (docker stats) doesn’t always aggregate these correctly — so your monitoring shows 60% memory usage while the kernel’s actual RSS sits at 85%.

According to an AWS-focused analysis published on Medium in 2025, self-hosted runners on EC2 instances running Ubuntu 22.04 showed a consistent 20–30% discrepancy between Docker-reported memory and free -m output after 6+ hours of continuous job processing. The recommendation: monitor with vmstat 5 or node_exporter rather than trusting docker stats alone.

To force proper memory release, add vm.overcommit_memory=1 and vm.swappiness=10 to /etc/sysctl.conf. Reload with sysctl -p. This doesn’t eliminate the leak but significantly slows accumulation between cleanup cycles.

The Runner Doesn’t Self-Clean Between Jobs

The runner binary holds references to Docker network namespaces between jobs when ephemeral runner configurations are used incorrectly. Specifically: if you’re running the runner in --once mode but not destroying and recreating the container after each job, network namespace file descriptors accumulate in /proc/<runner_pid>/fd/.

The botmonster.com CI/CD pipeline guide from 2025 demonstrates this with a Gitea Actions + Docker setup that mirrors the GitHub Actions pattern exactly. Their solution — running each runner instance as a short-lived Docker container that exits after one job — is the cleanest fix available. When the runner process itself is ephemeral, the memory leak essentially becomes a non-issue.

Comparing the Three Fix Approaches

The ephemeral approach wins on memory hygiene. The tradeoff: 30–60 seconds of cold start per job for image pull time. For teams running hundreds of jobs daily, that compounds fast. The daemon-level prune hook — a post_run script calling docker system prune -af --volumes — is the pragmatic middle ground for most setups.

This approach can fail when jobs are short-lived and run at high frequency. The prune operation itself takes time, and if jobs queue faster than cleanup completes, you’re still accumulating. Know your job cadence before choosing.

Three Scenarios Worth Planning For

Single persistent runner on a VPS

The memory leak will hit a wall somewhere between 8–72 hours depending on job frequency. Apply the daemon.json config change immediately, add a cron job running docker system prune -f every 2 hours, and set up a watchdog script that restarts the runner service if memory exceeds 80%. This buys time while planning the ephemeral migration.

Auto-scaled runners on AWS EC2

If instances scale out but don’t scale in properly, leaked memory accumulates across the fleet. Set instance termination to trigger on runner idle time, not just job queue depth. A runner that’s idle but memory-saturated will accept a job and OOM-kill it. That failure mode is silent and painful to debug.

Docker-in-Docker workflows

DinD dramatically worsens the situation. Inner Docker daemons create their own overlay layers that the outer daemon doesn’t automatically clean. The fix here isn’t optional: use docker:dind with --storage-driver=vfs in your workflow YAML (slower but leak-free) or switch to kaniko for container builds entirely. There’s no middle path that works reliably.

What’s Coming — and What to Do Tonight

The memory leak problem isn’t mysterious once the layers are separated:

• cgroup v2 on Ubuntu 22.04 changes memory accounting behavior, creating a reporting gap
• Orphaned overlay filesystems accumulate silently after unclean job exits
• The runner binary holds network namespace file descriptors without a built-in cleanup mechanism
• Ephemeral runner design eliminates the root cause; daemon-level cleanup mitigates it

Two developments worth tracking. Docker Engine 26.x — already in release candidate as of April 2026 — includes improved cgroup v2 memory release behavior that should reduce accumulation rates by an estimated 40–60% based on changelog notes. And GitHub’s Actions Runner Controller (ARC) project will likely stabilize ephemeral runner patterns enough for mainstream adoption by Q3 2026, with the v0.10.x ARC releases already adding better pod lifecycle hooks for Kubernetes deployments.

The immediate action: audit your /var/lib/docker/overlay2/ directory size right now. If it’s above 5GB on a runner less than a week old, accumulation is already happening. Apply the daemon prune hook tonight. Plan the ephemeral migration for next sprint.

The fix path looks different depending on whether you’re running a persistent VM, EC2 auto-scaling, or Kubernetes — so what’s your current setup? Worth discussing in the comments, because the approach that works cleanly for one environment can quietly fail in another.

Key Takeaways

• Ubuntu 22.04’s cgroup v2 default creates a memory accounting gap between docker stats and actual kernel RSS — monitor with vmstat or node_exporter instead
• Orphaned overlay2 layers are the primary driver of disk and memory accumulation; check /var/lib/docker/overlay2/ size as your first diagnostic step
• Three fixes exist at different complexity levels: daemon-level prune hooks (fast to ship), sysctl tuning (moderate effort), and ephemeral runner containers (highest reliability)
• Docker-in-Docker workflows require special handling — --storage-driver=vfs or a kaniko migration, not just cleanup scripts
• Docker Engine 26.x and ARC v0.10.x are the upstream improvements most likely to reduce this problem materially in the next 6 months

References

1. How to Set Up a Self-Hosted GitHub Actions Runner on Ubuntu
2. Build a Self-Hosted CI/CD Pipeline with Gitea Actions and Docker - Botmonster Tech
3. Self-Hosted GitHub Actions Runners on AWS EC2: Solving Memory and Disk Space Challenges | by ELVIS G

Photo by Roman Synkevych on Unsplash