numactl

numactl
Name	numactl
Developer	Linux community
Released	2003
Operating system	Linux
License	GNU General Public License

numactl

numactl is a Linux utility that controls NUMA (Non-Uniform Memory Access) policy for processes and shared memory. It provides command-line interfaces and library hooks to bind processes, threads, and memory allocations to specified CPUs and NUMA nodes, enabling administrators and developers to optimize locality on systems from vendors such as Intel Corporation, Advanced Micro Devices, IBM, ARM Holdings and Fujitsu. numactl is commonly used alongside kernel features developed by contributors associated with projects like Linux kernel, GNU Project, and distributions including Debian, Red Hat Enterprise Linux, Ubuntu (operating system), and SUSE Linux Enterprise Server.

Overview

numactl exposes mechanisms to influence placement of CPU affinity and memory allocation to improve performance on systems with topologies produced by manufacturers like Dell Technologies, Hewlett-Packard Enterprise, or Lenovo. It interacts with kernel subsystems implemented by teams working on Memory Management (Linux kernel), Scheduler (Linux kernel), and device trees influenced by communities around OpenStack, Kubernetes, and QEMU Project. The tool complements profiling and tuning tools from projects such as perf (Linux), Valgrind, and SystemTap for diagnosing cache misses, interconnect latency, and remote access patterns on NUMA hardware.

Usage and Options

numactl offers flags to set CPU and memory binding, interleaving, and preferred node behavior. Common options include binding a process to CPUs via affinity flags similar in intent to APIs in POSIX, or specifying memory policies analogous to interfaces exposed by libc. Administrators use numactl with commands from GNU Coreutils and monitoring from htop, top (software), sar (Linux), and iotop. Typical usage patterns include starting a program bound to nodes using node lists that map to topology reported by tools like lscpu, numastat, and hwloc. Options let users combine CPU affinity with memory policy, set strict allocation policies to trap remote allocations, or interleave allocations across nodes to balance load for workloads comparable to those run by Apache HTTP Server, MySQL, PostgreSQL, Redis, or high-performance frameworks like Open MPI and TensorFlow.

NUMA Policies and Mechanisms

numactl controls policies such as binding, preferred, interleave, and local allocation. The bind policy pins allocations to specific nodes, akin to explicit placement strategies used in HPC centers operating clusters with resource managers like Slurm Workload Manager and Torque (software scheduler). The preferred policy nudges the kernel to allocate from a node if possible; this mirrors heuristics found in memory subsystems influenced by research from institutions such as Lawrence Livermore National Laboratory and Oak Ridge National Laboratory. Interleave distributes allocations round-robin across a set of nodes, a technique used in parallel frameworks developed at Los Alamos National Laboratory and in projects like OpenMP. Strict policies generate allocation failures that enable runtime detection of policy violations similar to fault-injection practices used in NASA software testing.

Implementation and Internals

numactl is implemented in C and relies on kernel system calls and the libnuma API exposed by a userspace library. It uses interfaces implemented by maintainers contributing to kernel trees managed under organizations like The Linux Foundation and uses topology information exposed via sysfs by firmware standards such as ACPI and Device Tree. Internally, numactl translates command-line specifications into calls to APIs similar to those defined by glibc and interacts with scheduler affinity masks provided by the kernel's cpuset and cgroup facilities, which are also used by projects like systemd and Docker. The utility reads node maps from files in /sys and coordinates with memory policy enforcement in the kernel's mm layer—a codebase continually refined by contributors employed at companies like Google, Facebook, and Microsoft who participate in upstream Linux development.

Performance and Use Cases

numactl is widely used to reduce remote memory access latency and to increase bandwidth for memory-bound workloads. Typical beneficiaries include scientific computing stacks such as LAMMPS, GROMACS, and NAMD, big data platforms like Apache Hadoop and Apache Spark, and in-memory databases or services exemplified by Memcached and ElasticSearch. Benchmarks by hardware vendors and academic groups often show improvements in throughput and latency when using node-aware placement compared to naive allocations; such results appear in publications from institutions like MIT, Stanford University, ETH Zurich, and industrial performance studies by Intel Corporation and AMD. However, incorrect policies can worsen performance if they cause CPU-Memory imbalance or interfere with scheduler load balancing mechanisms used in environments managed by Kubernetes or Mesos.

Compatibility and Platforms

numactl targets Linux distributions running on architectures with NUMA-capable hardware including x86_64, POWER, and ARM64. Platform support depends on kernel versions and firmware that correctly describe topology via ACPI or device tree, affecting systems from vendors like HPE, Oracle, and Cisco Systems. Integration with virtualization stacks such as KVM, Xen (hypervisor), and VMware ESXi varies: guest-visible NUMA topology and host policies determine the effectiveness of userspace binding. numactl's behavior is also influenced by container runtimes and orchestration platforms such as Docker, Kubernetes, and LXC which manipulate cgroups and cpusets.

Category:System administration tools