Nilekh Chaudhari

Napkin math for tensor parallelism

Tue, 28 Jul 2026 00:00:00 +0000

If I hand you a model that is bigger than any single GPU you own, how do you decide what deployment shape will actually work? Which context length can you realistically serve? How many concurrent users can the memory hold? Which knob do you turn to trade one for the other? These sound like scheduling questions. They aren’t. Every one of them resolves to a memory calculation, and every calculation resolves to a handful of numbers you can read out of the model’s config file.

The memory that runs the model

Sat, 18 Jul 2026 00:00:00 +0000

If you’d asked me a year ago what limits how fast an LLM can answer, I’d have said the same thing you probably would, “the GPU.” Turns out that’s wrong. The GPU is often bored. What actually limits your LLM server is a piece of memory most people have never heard of. Once you see it, you can’t unsee it. This is how I learned to see it. And how I built a tool so you can too.

A new notebook

Wed, 08 Jul 2026 00:00:00 +0000

This is the first post on a blog I just spun up. Calling it a “blog” is probably too generous. It’s really a working notebook. I wanted a public place to write down what I learn, and I wanted the writing to force me to actually understand the things I’m poking at.

Why I’m writing this

I’ve spent years shipping platform code for Kubernetes, service mesh, and distributed systems at planet scale on products such as Azure Kubernetes Service (AKS). Good work, but the ground is shifting, and with the advent of AI and LLMs, new problems are emerging. A lot of what I learned building substrate for one workload class turns out to be relevant to the next one, but not all of it, and the parts that aren’t are what I want to write about.

About

Wed, 08 Jul 2026 00:00:00 +0000

Hi, I’m Nilekh. I’m a Senior Software Engineer at Microsoft, specializing in Kubernetes, and working on the security, identity, and networking substrate for Azure Kubernetes Service (AKS). I’ve been writing and shipping open-source infrastructure for years, contributing to key upstream Kubernetes areas via SIG Auth and SIG API Machinery, the Secrets Store CSI Driver ecosystem (driver, sync controller, and Azure provider), and the Gatekeeper Library. I’m currently an active contributor to Istio.