Titus, the internal container platform used by Netflix, functions as a Kubernetes-based hosting environment. It manages both the control plane and the data plane, ensuring application teams receive a consistent and reliable container experience without needing to manage the underlying host, cluster, or kernel.
This platform runs millions of containers weekly across tens of thousands of EC2 instances across multiple AWS regions. By default, Titus is multi-tenant, meaning numerous containers from various teams share each host, which drives the efficiency of container compute.
However, while host sharing works for most applications, certain demanding real-time systems present challenges. These systems require a different approach, leading to the development of single-tenant Titus. This mode grants a task its own virtual machine while maintaining Titus's standard API, deployment path, and the managed experience where the Compute/Platform team retains host ownership.
The inherent trade-off in multi-tenancy involves packing containers onto instances, which necessitates sharing a single Linux kernel and underlying hardware. Titus uses kernel tools like cgroups, namespaces, seccomp, and AppArmor to provide isolation for partitionable resources.
Nevertheless, this kernel-level isolation cannot fully partition shared hardware components. For instance, contention for a last-level cache or memory bandwidth can slow a neighbor's process, even if both remain within their allocated CPU limits, because there is no cgroup mechanism for memory bandwidth or cache residency.
This physical interference is significant. For example, the network shaping mechanism, which uses a hierarchical token bucket (HTB) queueing discipline, can suffer from contention on a global lock when flooded with small packets. This lock contention can consume kernel CPU, thereby starving co-located containers and causing tail-latency spikes.
The workloads most susceptible to this interference are stateful, high-throughput streaming jobs, such as real-time data pipelines. For these jobs, intermittent interference is not a minor slowdown but a "stop-the-world event that pages an on-call engineer." Teams have confirmed these issues do not occur on plain EC2 instances.
The platform evaluated several options to provide VM-level isolation without forfeiting Titus’s managed benefits. Improving shared-kernel isolation was deemed insufficient because its structural limitations prevent fully containing hardware issues like cache or memory bandwidth contention.
Options like microVMs were weighed but found costly due to hardware virtualization dependencies, the lack of a managed microVM service requiring internal maintenance, and virtualization overhead impacting CPU- and IO-bound work. gVisor was rejected because user-space syscall interception introduces overhead on IO-heavy paths and relies on a vendor-maintained project.
Moving workloads to EC2 was also considered, but this shifts the operational burden—choosing instance types, patching hosts, and fleet sizing—back to the application team, which is precisely what Titus is designed to prevent.
This development reflects a broader industry trend where the limitations of shared-kernel isolation become apparent when workloads demand absolute guarantees against physical resource contention, pushing infrastructure operators toward more robust isolation primitives.
