Tuning, scaling up vs out, and how much headroom is enough.

Capacity planning for Kubernetes ingress controllers. Two ways to add capacity, scaling up with fewer larger pods or scaling out with more smaller pods, aimed at the same headroom target and validated with soak and spike tests.
In the previous article, Traefik met the performance objectives for our workload, as long as it ran with enough replicas. NGINX still did about twice the work per CPU core. The verdict was “fast enough, if you size it right.”
This article is about the “size it right” part.
Our real traffic was never the problem. It sat more than 50 times below even the smallest setup I tested. The real question was different: how much Traefik do we need to keep the same safety margin our NGINX setup gave us?
Answering it took four steps. Get what’s free from tuning, pick the right way to scale, turn the results into a replica plan, and prove the plan holds over time and under spikes. My example is Traefik replacing NGINX, but the same four steps work for any ingress controller.
Start With the Target
A capacity plan needs a target, just like a benchmark does. Without one, “enough” turns into whatever number feels safe on the day.
Our target was simple: match the headroom of the controller we were replacing. Headroom is the gap between the traffic you expect and the most traffic the system can handle while still healthy. NGINX had been running 5 to 10 pods with 1 CPU core each, and its headroom had served us well. Traefik had to offer about the same.
“Healthy” matters here. In the last article, I separated the ceiling, the highest throughput reached while still healthy, from the optimum, the point just below it where latency is still close to its minimum. The capacity numbers in this article are ceilings, taken while errors were zero and latency was still low. Real traffic should then sit near the optimum, well below them.
I kept the same method as before. k6, an open-source load-testing tool, sent load straight to the controller pods, swept from a few users up to thousands, and every level ran three times, reported as the median, to smooth out run-to-run noise. The autoscaler was pinned so the pod count couldn’t change mid-run.
The setup itself was stronger this time, with a bigger backend behind the controllers, so nothing else could become the limit at higher throughput. Measured side by side again, NGINX came out about 2.2× faster on 1-core pods and 2.1× on 2-core pods, a little less than the 2.4× in the last article. The ratio held, which is why ratios, not absolute numbers, drive the plan.
With the target fixed, the first question was whether Traefik could do more with what it already had.
Step 1: Tune Before You Scale
Tuning is free. Scaling costs CPU every hour of every day. So before adding resources, I looked for configuration changes that let Traefik do more on the same budget.
I picked what to test from the symptoms, not from a list of popular tips. In the last article, Traefik’s memory grew under load while its throughput fell, which pointed to the garbage collector competing for CPU. Access logs were the other obvious cost paid on every request. I tried the popular tips too, and they’re covered below.
The method matters as much as the result. The baseline was the configuration we actually ran: access logs on, buffered, and no garbage collector tuning. From there, I changed one setting at a time on identical resources, 2 pods with 1 core each, then combined the ones that helped. The winning combination was then confirmed over three trials.

Two changes made the difference:
- Access logs off: about 28% more throughput. Traefik ships with access logs off, but many teams turn them on for debugging, as we had. Traefik then writes a log line for every request, and even with buffering, that work competes with request handling on a single core.
- Garbage collector tuning: about 16% more. Traefik is written in Go, and Go’s garbage collector, or GC, frees unused memory in the background using the same CPU. Raising GOGC makes it run less often. GOMEMLIMIT sets a soft memory limit, so the collector still steps in before the pod runs out of memory.
Together, they lifted the peak by 44%, from about 4,250 to about 6,150 requests per second, with no extra CPU. p95 latency, the response time that 95% of requests beat, also halved at the peak, from 30 ms to 16 ms.
In Kubernetes, both are a few lines on the Traefik container. The first line only matters if you turned access logs on:
args:
- --accesslog=false
env:
- name: GOMEMLIMIT
value: "2560MiB" # about 83% of the 3 GiB limit
- name: GOGC
value: "300"
Here’s how I picked the two values:
- GOGC=300. The default of 100 runs the collector when the heap reaches twice its live size. At 300, it waits until four times, so it runs far less often. The pods had memory to spare, using at most about two-thirds of their limit, so I chose a clear step up rather than a small one. GOMEMLIMIT is what makes a bold value safe. I tested only this one value, so treat it as a starting point and try a few on your own setup.
- GOMEMLIMIT=2560MiB. That’s about 83% of the 3 GiB pod limit, in line with the common Go advice of 80 to 90%. The gap leaves room for short memory spikes, such as when Traefik reloads its routing configuration, so the pod isn’t killed for running out of memory.
Neither change is free of trade-offs. GC tuning swaps memory for CPU, and Traefik’s memory nearly doubled, coming close to the 3 GiB limit under heavy load. Turning access logs off means losing per-request logs, which you may miss during an incident.
Just as useful was learning what didn’t help:
- GOMAXPROCS, which controls how many CPU threads a Go program uses, made no difference. Recent Go versions already set it from the container’s CPU limit.
- A bigger backend connection pool made no difference. The default was already more than enough.
- FastProxy, an experimental faster proxy engine in Traefik, cut throughput by about 40% on our setup. Popular tuning tips don’t always carry over, so measure them.
- HTTPS itself cost about 24% compared with plain HTTP. That’s the price of encryption, not a setting to tune away.
In the end, the capacity plan used the baseline configuration, and the tuning gain became extra margin on top. That’s the safer way to plan. Access logs earn their keep during an incident, and a plan shouldn’t depend on settings someone might change later.
Tuning showed how much more Traefik could do on the same CPU. The next question was the best way to add CPU when you need more.
Step 2: Scale Up or Scale Out?
There are two ways to add capacity. Scaling up gives each pod more CPU. Scaling out adds more pods. Most teams default to scaling out, because that’s what an autoscaler does. I measured both, on the baseline configuration:
- Up: 2 pods, with 1, 2 and then 4 cores each
- Out: 1 core per pod, with 2, 4 and then 8 pods

Doubling CPU per pod, from 1 core to 2, added 80%. Doubling the number of pods, from 2 to 4, added about 50%. More CPU per pod was the more efficient lever.
Neither lever scales in a straight line. The third and fourth cores added 43%, and going from 4 to 8 pods added 47%. Every step bought a bit less than the one before.
So the rule for Traefik is prefer fewer, larger pods, up to about 2 cores, where the biggest jump happened. Keep at least two pods so one can restart or fail without an outage.
Now I knew which lever to pull. The next step was working out how far to pull it.
Step 3: Turn the Curves Into a Plan
To match NGINX’s headroom, I compared both controllers per pod, measured with the same load setup:

A 1-core Traefik pod delivered about 45% of a 1-core NGINX pod. Matching NGINX on 1-core pods would need roughly twice as many pods, and the last step showed that adding pods is the less efficient lever.
A 2-core Traefik pod delivered about 80% of a 1-core NGINX pod. That led to a simple plan:
- Keep the same replica range, 5 to 10 pods, so nothing changes for the people running it.
- Give each pod 2 CPU cores instead of 1.
- Keep 3 GiB of memory per pod, because Traefik uses nearly twice as much memory as NGINX and it grows under load.
That gets close to NGINX’s headroom with the same number of pods. If you need exact parity, raise the maximum to about 12 pods, because 12 Traefik pods at 80% roughly match 10 NGINX pods.
Measured against real traffic, even the smallest Traefik setup tested had more than 50 times the headroom we needed. The plan wasn’t about surviving today’s load. It was about keeping the same safety margin we already trusted.
Your own ratios will differ with your controller, hardware and traffic, so measure them rather than reusing mine. The way you get from curves to a plan stays the same.
On paper, the plan worked. A plan on paper is still a prediction, so the last step was to test it the way production would.
Step 4: Prove the Plan Holds
Short peak tests don’t show memory leaks, and they don’t show what happens when traffic jumps without warning. So I ran two more tests on 2 pods with 2 cores each, the smallest version of the plan, where any weakness would show first. Both used the baseline configuration.
Steady Load for Three Hours
A soak test holds a steady load for hours to catch slow problems, such as memory that creeps up or latency that drifts. I ran 2,000 requests per second for three hours, long enough for slow problems to surface. That rate is about a quarter of this setup’s peak, comfortably below it, where you’d want real traffic to sit.

Nothing drifted. p95 stayed between 2.2 and 2.4 ms, errors stayed at 0%, and no pod restarted. Memory settled at about 1.5 GiB per pod, half the limit, and CPU sat at about 28% of the budget.
Sudden Spikes
For spikes, I switched to open-loop load, where requests arrive at a fixed rate whether or not the controller keeps up. That’s how a real spike behaves, and it lets errors show up under overload instead of being hidden by clients that wait. Starting from 500 requests per second, I jumped to 6,000, below this setup’s measured peak, then dropped back. Next I jumped to 10,000, well past the peak, to see how it fails when overloaded.

Traefik slowed down during each jump but didn’t fall over. At 10,000 requests per second it capped throughput at about 5,400, with errors at 0.51% and no restarts. As soon as the spike ended, latency was back to 2 to 3 ms.
One detail is worth noticing. Even the 6,000 spike, below the measured peak, wasn’t fully served. A sudden jump forces thousands of new HTTPS connections at once, and setting those up costs far more than serving steady traffic over connections that already exist. A peak measured with a gradual ramp overstates what you can absorb in a sudden spike.
The plan held up under both tests. That leaves the question in the subtitle.
How Much Headroom Is Enough?
There’s no single magic number, but these tests gave me four practical rules:
- Plan around the optimum, not the ceiling. Real peak traffic should sit near the optimum, where latency is still low. The ceiling is your safety buffer, not your plan.
- Keep at least the margin you already trust. If you’re replacing a system, match its headroom. If you have service-level objectives, size against those.
- Leave extra room for spikes. A gradual-ramp peak overstates spike capacity, so don’t plan right up to it.
- Treat tuning as a bonus. Plan on the configuration you’ll actually run and let tuning gains add margin on top.
Following these rules, 5 to 10 pods with 2 cores each gave close to the margin we had with NGINX, with room to go to 12 pods for full parity. Spike behaviour and long-run stability were measured instead of assumed.
Checklist
Before you trust a capacity plan, check each of these. Each one closes a gap that could otherwise surprise you in production.
- Set the target first. Decide what “enough” means, such as matching the system you’re replacing or meeting a service-level objective, before you measure anything.
- Tune before you scale. Test one setting at a time on identical resources, confirm the winners over several trials, and write down the trade-offs.
- Measure both ways to scale. Test more CPU per pod and more pods separately, because the more efficient lever depends on the controller.
- Compare per pod. Express capacity per pod against the system you’re replacing, so the replica plan follows directly.
- Plan on what you’ll run. Size against the configuration you’ll actually run in production, and keep tuning gains as extra margin.
- Prove it over time and under spikes. Run a soak test for hours, and an open-loop spike test that goes past the peak.
Together, they turn a set of benchmark curves into a plan you can defend.
Final Thoughts
I expected capacity planning to be about finding the biggest number Traefik could reach. It turned out to be about three smaller questions. What can it do for free? Which resource buys the most? Does the plan still hold after three hours and a sudden spike?
The answers weren’t obvious in advance. A popular tuning tip made things worse, scaling up beat the autoscaler’s default habit of scaling out, and a spike exposed a limit the gradual ramp never showed.
So don’t size an ingress controller from its peak. Tune what’s free, add CPU before pods, plan on what you’ll actually run, and prove it holds before you trust it.
What’s Next
Every number in this article came from a test cluster. Production is different: a bigger cluster, other workloads on the same nodes and far more network rules. In the next article, I’ll show how to check these results in production safely, without touching real users.
Capacity Planning for Kubernetes Ingress Controllers: A Practical Guide was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.





