Why Is My Load Average High When CPU Looks Idle? (and How to Stop It)

A load average of 12 on a four-core box with an idle CPU is not a compute problem. It almost always means processes are queued behind slow disk, a saturated device, or a mount that has stopped answering.

The panic reflex is to reach for more cores. That is the wrong instinct, and it wastes money on the wrong bottleneck. If top shows the load average climbing while the CPU sits mostly idle, the machine is telling you something precise, and the fix is to read what it is actually saying rather than throw hardware at a number you have misread. This walks through the diagnosis on Ubuntu 24.04 with the standard sysstat toolkit, and gets to the culprit fast.

What load average actually counts

The single most common misconception is that load average is a CPU metric. On Linux it is not. It is a count of processes that are either runnable (state R, wanting CPU) or in uninterruptible sleep (state D, blocked in the kernel and unable to be interrupted, almost always waiting on I/O). Other operating systems count only runnable tasks; Linux folds D-state tasks in, and that is precisely why a Linux load figure can be high while the processor is idle.

So a load of 12 on a four-core machine does not mean “you need twelve cores”. It means twelve tasks were, on average, either running, waiting for a core, or stuck waiting on the kernel. If the CPU is idle, the third category is where your problem lives. The number is a symptom, not a diagnosis, and the whole job is to split it into its parts.

First triage: is it compute or is it wait?

Start with vmstat. Two columns decide everything: r (processes runnable and waiting for CPU) and b (processes in uninterruptible sleep). The wa column under cpu is the percentage of time the CPU spent idle with outstanding I/O.

# cores available, and load average
nproc                 # e.g. 4
uptime                # e.g. load average: 12.4, 11.8, 9.2

# five one-second samples of run queue vs blocked queue
vmstat 1 5
# procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
#  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
#  1 11      0 210344  18220 640112    0    0  4096     0  820 1540  3  2 15 80  0

# count processes stuck in uninterruptible sleep (D state)
ps -eo stat | grep -c '^D'

Read it like this. High r with high us/sy and near-zero id is a genuine CPU bottleneck — that is the case where more cores or less work is the answer. High b, high wa, and a large idle percentage is the opposite: the CPU is bored, and your tasks are queued behind I/O. The ps count is the blunt confirmation. If eleven processes are in D state and your load average is around twelve, you have found where the number comes from.

Find the device and the process

Once wa and the D-state count point at I/O, stop guessing and measure the device. iostat -x gives you the per-device extended statistics. Two fields matter most: %util (the percentage of time the device had at least one request in flight) and aqu-sz (the average queue length). On sysstat 12, which is what Ubuntu 24.04 ships, the wait columns are r_await and w_await — average milliseconds a read or write request spent queued plus serviced.

# extended per-device stats, skipping idle devices, refreshed each second
iostat -xz 1
# Device  r/s   w/s  rkB/s  wkB/s  r_await w_await  aqu-sz  %util
# nvme0n1 12.0  4.0  1536.0  512.0    0.42    0.55    0.02    3.10
# sdb    980.0 60.0 62720.0 3840.0   14.80   22.40    9.60   99.40

# which processes are actually issuing the I/O (needs root)
sudo iotop -oPa

# per-process read/write rate and, crucially, iodelay
sudo pidstat -d 1

A device sitting at %util near 100 with a queue length in the high single digits or more is saturated: it cannot absorb the request rate, and everything behind it waits. Now name the offender. iotop -oPa shows only processes actually doing I/O (-o), as processes rather than threads (-P), with accumulated totals (-a). pidstat -d 1 gives per-process kB_rd/s and kB_wr/s, and the column that earns its keep here is iodelay — clock ticks the task spent blocked waiting for I/O. The process with the highest iodelay is the one contributing most to your load, whether it is a database flushing, a backup job, or a log shipper gone rogue.

The uninterruptible-sleep trap: a hung mount

There is a nastier variant where the disk is fine and the load still climbs. This is the hung network mount. When an NFS server stops answering, every process that so much as stat()s a path on that mount goes into D state and stays there — uninterruptible means exactly that, so you cannot even kill -9 them. iostat shows the local disks idle, because no local I/O is happening; the tasks are wedged in the kernel waiting on a network round trip that never returns.

# show the wait channel of every D-state process
ps -eo pid,stat,wchan:24,cmd | awk '$2 ~ /D/'
# a wchan of nfs_* or rpc_wait_bit_killable points straight at NFS

# is the mount responding? this hangs if the server is gone
sudo timeout 5 stat -f /mnt/shared || echo 'mount not responding'

If the wchan column reads nfs-anything, the fix is not on this box — it is the server or the network path to it. Mounting with soft and a sane timeo/retrans, or with intr-equivalent behaviour, bounds the damage next time, at the cost of surfacing I/O errors to applications that must then handle them.

Fix the bottleneck you actually have

The pattern behind almost every one of these is amplification: one slow dependency backs up, callers retry, and the retries deepen the queue. An application hammering a saturated database with un-bounded retries turns a slow disk into a retry storm, which is one of several reasons retries need to be idempotent and rate-limited rather than fired blindly. Treat the load average as an input to root-cause analysis, not the finding itself.

None of this is exotic. It is vmstat to split run queue from blocked queue, iostat -x to find the saturated device, pidstat -d and iotop to name the process, and ps with wchan to catch the hung mount. The discipline worth building is doing it in that order every time, so that mean-time-to-resolution stops depending on who happens to be on call — the same reason it is worth instrumenting recovery metrics in the first place. A high load average with an idle CPU is not a mystery. It is the machine telling you where it is waiting, in a language the sysstat tools translate cleanly if you let them.