Servers for Kubernetes on-prem: memory, NVMe, network, cores

The bottlenecks that usually surface after launch

Issues typically don’t appear on day one but after 1–3 months. Teams add services, enable more logging and metrics, user counts grow, and background jobs start living their own life. That’s when you learn that “it worked in the pilot” doesn’t mean “it will be stable in production”.

Pilots are usually short and tidy: less data and integrations, infrequent deploys, no peak periods (reporting, month-end, mass mailings). In production, microservices easily create unpredictable spikes. One service slows down, queues grow, retries multiply load, and the situation cascades across the cluster.

Queues and latency most often hit four areas:

• CPU: limits are set by eye, the scheduler runs out of available cores, and neighboring pods interfere with each other during peaks.
• Memory: OOM-kills, frequent restarts, pod evictions and lost caches. The issue is often not average consumption but short spikes.
• Disks: slow down container layer operations, logs, databases and queues. IOPS run out before capacity.
• Network: east‑west traffic between services becomes constrained, bandwidth for storage and backups is insufficient, and latency rises due to overloaded switches.

A simple example: you enabled tracing and increased metrics retention. Disk writes spike, and at the same time interservice traffic rises during peak hours. The result is not one degraded component but several. So choose servers for Kubernetes on‑prem with headroom in the most sensitive resources and with the understanding that cluster load is rarely even.

Describe the load before choosing hardware

Hardware for the cluster is often chosen by eye, and later teams are surprised: sometimes memory is missing, sometimes disks are the bottleneck, sometimes the network is saturated by control traffic. To avoid your on‑prem Kubernetes servers becoming a problem a month after launch, start by describing the load in simple terms, as if explaining it to the person who will support it.

First, list what you run and where. The number of microservices alone says little: the number of environments (prod, stage, dev) and how similar they are matters more. A common situation: staging is “almost like prod” but lives on the same nodes and during a load test starts evicting production pods.

Next, mark the stateful parts. Databases, queues, log stores, monitoring and tracing usually produce steady I/O and are sensitive to latency. If they’re inside the cluster, disk and network requirements jump. If they’re external, you shift the risk to the network and external storage.

It’s useful to record this as a short “near‑requirements” note:

• how many services and environments you support now and the plan for a year from now;
• what is stateful vs stateless;
• which resource profile dominates (CPU, memory, disk);
• which SLOs are most important (request latency, deploy time, recovery speed);
• expected growth over 12–24 months (users, traffic, data).

Example: 40 services in prod and 40 in stage, Prometheus and Loki inside the cluster, database external. Then bottlenecks are most likely memory (metrics and caches), I/O for logs, and network between nodes during rollouts.

Memory per node: avoiding OOM and pod eviction

Memory usually runs out before CPU in Kubernetes. You can tighten CPU with limits and the scheduler will spread load. Memory overuse leads to OOMKilled or eviction, then pods move to other nodes and trigger cascading issues.

A common trap is sizing RAM by the sum of requests while actual consumption is closer to limits (or unlimited if limits are not set). Also remember the memory the cluster itself needs. Node Allocatable is always less than physical RAM. So a 256 GB machine might be “useful” only for 200–220 GB depending on settings and load.

Think of memory in three layers: applications, system components (kubelet, container runtime, CNI) and surrounding services (logging, monitoring, security agents). Even small parts add up — and they grow with the cluster.

Two‑socket servers add a NUMA nuance: memory is physically attached to a socket. If a pod needs a lot of RAM and CPU but NUMA policy is ignored, you can get latency and unstable performance even if numbers look sufficient.

Practical rules before procurement:

• keep a steady buffer of 15–30% RAM for the system and peaks;
• set requests and sensible limits for all important services;
• account separately for caches, queues and JVM/Go heaps (they often grow unexpectedly);
• for large stateful components plan nodes with big RAM, while for many small services it’s often better to have more smaller nodes.

If the platform expects growth, it’s easier when each node has memory headroom. Adding new microservices then won’t turn into a daily game of pod relocation.

CPU and core headroom: the scheduler likes predictability

CPU in Kubernetes is not just “how many cores”. It’s important how predictably they are available to pods. When choosing servers for Kubernetes on‑prem, budget headroom not just for today but for peaks, background jobs and growth.

The most frequent mistake is aggressive oversubscription, where CPU limits are set much higher than the node’s real resources. Average usage graphs may look fine, but at peak time pods compete for CPU time. The result: increased latency tails, API timeouts and a chain of retries that overwhelms the cluster.

When is frequency more important than core count? For short requests with strict SLOs (gateway, auth, online payments) higher frequency and predictable per‑core performance often wins. For parallel tasks (queues, background workers, batch, CI) more cores and clear limit policies matter more.

A good practice is to reserve separate nodes for cluster infrastructure. That way CPU isn’t eaten by components you don’t see in business metrics: DNS, ingress controllers, logging, monitoring, service mesh and storage controllers.

Check in advance: how much CPU headroom do you need, what oversubscription is acceptable without lottery‑like latency, how SMT/Hyper‑Threading affects your workload, and whether you need at least a small infrastructure node pool.

A simple sign: if services look “alive” after launch but latency tails grow under load, it’s often not a bug but a lack of predictable CPU and overly optimistic limits.

NVMe and disks: where high speed really matters

For microservices it’s not only about IOPS numbers but latency. IOPS shows how many operations per second a disk can handle; latency shows how fast each operation completes. In Kubernetes low latency is usually more critical: many small operations (metadata, logs, small writes) are sensitive to stalls even if average IOPS looks fine.

NVMe makes sense where the bottleneck is random access and quick write acknowledgment. Typical cases: etcd (if control plane is local and you rely on local durability), hot parts of databases and queues, search indexes, log and metrics buffers, and CI caches (artifacts, container layers, dependencies).

The trade‑off between local NVMe and external storage is simple. Local NVMe has lower latency and often feels cheaper, but is harder to protect at the node level: if the disk dies the node’s data is lost. External storage is easier to protect and scale, but it adds network and latency and can become a new bottleneck.

RAID is not free either. a controller and its cache can limit throughput, especially on NVMe where a disk can be faster than the controller. Choose NVMe layouts consciously and test under real load.

Don’t forget disk endurance (TBW) and a replacement plan without downtime: monitor wear, keep spare drives and have a clear scenario for recreating pods and moving data.

Network: bandwidth, latency and redundancy

Network in Kubernetes is rarely “just network”. East‑west traffic between services grows quickly: services talk to each other, sidecar proxies in a service mesh add more chatter, and observability continuously sends data. External traffic can look modest while internal network is already at the limit.

Choose 10/25/100GbE not for “what you have now” but for how the cluster will look in a year. 10GbE is often enough for small clusters without heavy service mesh or high replication. 25GbE usually balances cost and performance for microservice platforms. 100GbE makes sense when you have many nodes, heavy CI/CD, large data volumes, distributed storage or when the network already limits you.

Count ports from the start. A common error is to buy a “fast” NIC with a single port and no path separation. A practical minimum per node is two ports for redundancy, plus separate segments (via VLAN or physical) if you separate storage/replication and management.

MTU and jumbo frames help only when equipment and network paths are configured consistently end‑to‑end. If any segment doesn’t support the chosen MTU you’ll see intermittent issues that are hard to debug. Change MTU deliberately and verify end‑to‑end.

For redundancy use bonding with LACP on nodes and spread uplinks across two switches. This is especially important on‑prem where a single network element can become a choke point.

Cluster reliability: control plane and capacity headroom

Cluster reliability usually fails not because everything falls over, but because of small hardware issues: one node reboots, one disk slows, one PSU dies. If the cluster has no headroom, any such failure triggers a chain reaction: pods reschedule, load rises, and degradation begins.

A minimal practical production basis is three control‑plane nodes and separate workers. Three control‑plane nodes provide quorum and tolerate one failure without losing control. If you mix control‑plane and application workloads on the same servers, a sudden CPU or disk spike from an app can hit cluster management.

Where to host etcd and why latency matters

etcd is sensitive not to disk size but to I/O latency and stability. Even moderate load on slow or “flapping” disks causes write delays which affect the whole control plane. Usually the best option is fast local disks with low latency on control‑plane nodes. Aim for predictability, not just headline speed.

Capacity headroom: what must remain after a failure

Plan the cluster as if one node is already gone. For critical platforms plan for losing 1–2 nodes (reboot, upgrade, failure).

Check: will you still have enough CPU and memory if one worker is unavailable; will there be space to reschedule pods without OOMs or heavy throttling; can storage and network absorb the surge during migration; can you upgrade nodes one-by-one without downtime.

There are physical risks that are easy to miss. Poor cooling causes CPU throttling: cores exist on paper but don’t deliver expected performance. Power matters too: redundant PSUs in servers and switches plus separate power feeds reduce the chance of sudden downtime.

Example: in a bank’s microservice platform you update worker nodes one at a time overnight. If capacity was planned too tightly, by the second node you’ll see queues, higher latency and SLO failures. With headroom the cluster weathers the update as calmly as a real failure.

Upgrade and growth plan: so hardware doesn’t become obsolete early

On‑prem Kubernetes rarely stays “as deployed”. In 6–12 months new cluster versions, security requirements and service growth emerge. Suddenly the limiter is not CPU but network, ports or address space.

Fix a calendar for Kubernetes upgrades and dependency compatibility. It’s not only Kubernetes: container runtime, CNI, CSI, drivers, firmware and OS also need updates. If one component lags, upgrading the cluster becomes risky.

A common working practice: rolling node updates using cordon/drain in short windows. Take a node out of scheduling, move pods, update OS and components, return it to the pool. Problems surface per node, not cluster‑wide.

To check if a new release will hit resource limits, keep a simple checklist: run tests in a staging cluster or a separate node pool; compare peak CPU and RAM with requests/limits; check disk IOPS and p99 latency; verify network headroom (east‑west traffic, p95 latency, errors); and evaluate whether monitoring and logs handle increased retention.

Write an expansion plan as rules. For example: add nodes when a pool sustains 60–70% load, and replace server generations every 3–5 years with overlap so old and new run together for a while.

Don’t forget physical constraints: rack space, power, free switch ports, spare 25/100GbE capacity, and Pod/Service IP ranges. In practice these block growth faster than the choice between NVMe and SATA.

How to pick a configuration: a simple step‑by‑step approach

Start with a map of your services, not a server model. Hardware must fit workload profiles: some clusters hit memory, others hit disk, and sometimes the network between nodes and storage is the limiter.

Gather baseline data for each environment (dev, stage, prod) and for key services: replicas, presence of queues and caches, logging and monitoring. Estimate requests and limits, but rely on measurements rather than assumptions.

A short checklist helps:

• aggregate resource budget (CPU, RAM, disk latency/IOPS, internal network and storage connectivity) by environment;
• split nodes into 2–3 types (general, CPU‑oriented, RAM‑oriented) and reserve storage nodes only if NVMe/high IOPS are truly needed;
• design network and redundancy (two ports per node, two switches, clear failure plan, segmentation for storage if needed);
• build headroom and follow N-1: cluster must run when one node is lost;
• define expansion rules: how fast you can add nodes and how you’ll move to faster networking if traffic grows.

This avoids buying on‑prem Kubernetes servers “tight” and hitting OOMs, disk queues or network saturation in 2–3 months.

What to test before buying

Run a short test plan on pilot nodes before final procurement: deploy typical services, enable HPA, apply load to database and logging, measure pod rescheduling time when a node is taken down and behavior under disk degradation. Watch metrics from your stack, not generic industry averages.

Example scenario: on‑prem microservice platform for a company

Imagine a company with 50–80 microservices, daily CI/CD releases, plus mandatory monitoring, logging and tracing. Everything is on‑prem for data control and predictable costs.

Split the cluster into role pools: infrastructure (ingress, DNS, metrics, logging, CI runners), stateless (main services and workers) and stateful (databases, queues, caches, storage).

Where NVMe is needed: stateful nodes and nodes doing heavy logging, indexing and metrics storage. There low latency under many small ops matters. For the stateless pool quality SSDs are often enough: these services rely more on CPU and memory, disk is mainly for images and temp files.

If interservice traffic is active (many requests between services, service mesh, frequent deploys) the network quickly becomes the limiter. In practice 25GbE per node often becomes the minimum, and dense clusters should provision faster uplinks and dual network paths.

Plan capacity so you can lose one node without panic: keep N+1 capacity in each pool and don’t pack nodes tightly. For example, if the stateless pool needs 120 vCPU by requests, plan so that losing one server still leaves at least 120 vCPU available, not 90.

Common mistakes when choosing servers for Kubernetes

The main problem is that hardware mistakes reveal themselves not at procurement but after a few months, when services are added and traffic rises. Then predictability breaks: the scheduler keeps moving pods, latency increases, and updates become an overnight marathon.

One frequent trap is buying lots of CPU and skimping on memory. As a result requests are guessed, containers hit OOMs, and nodes run out of RAM long before CPU gets busy. For on‑prem Kubernetes servers it’s almost always better to balance CPU and memory than to chase frequency alone.

Other recurring errors: installing NVMe everywhere when only some roles need it; keeping 10GbE and being surprised by east‑west growth; mixing control‑plane with heavy workloads; not leaving headroom for drain and maintenance, making upgrades require downtime; ignoring power, cooling and rack limits so the real configuration won’t fit the data center.

A simple example: a company runs on‑prem CI, monitoring and several APIs. After 3 months an analytics service and more logs are added. If the network stayed at 10GbE and there’s no memory headroom, timeouts and evictions start even though CPU is still free.

Short checklist and next steps

On one page, record what matters most: CPU headroom for peaks and upgrades, RAM buffer for working data and spikes, a clear disk strategy (where NVMe is needed vs SSD), a redundant network and a growth plan with N‑1 in mind.

Ask the dev team which services are stateful, which are latency‑sensitive, expected RPS and growth, image sizes and deploy frequency, and SLOs for response time.

Then validate hypotheses on a pilot node or small cluster. A few tests are enough: where CPU throttling and OOM begin, p99 disk latency under DB/queue profile, east‑west network behavior (latency, loss, failover of a link), and how long drain and recovery take.

Draft a one‑year upgrade calendar: quarterly Kubernetes and addon updates, windows for firmware/BIOS and a capacity review (for example, every six months).

If you want to simplify procurement and ongoing support, discuss configurations and expansion plans with an integrator who covers delivery and support. For example, GSE.kz (gse.kz) in Kazakhstan manufactures S200 servers and provides 24/7 technical support, which is convenient when fast component replacement and a single responsibility contour are important.