Multi-Instance GPU (MIG) — Inference Engineering

01

A 94 GB GPU running a 20 GB model

TL;DR · A 94 GB GPU serving a model that needs only 20 GB wastes most of the card — so the question is how to safely share one physical GPU across several workloads.

An H100 is a huge, expensive slab of compute and memory. Hand it a small model and most of it sits idle. This lesson is about the two ways to stop that waste — and the very different guarantees each one gives you.

The waste problem

Put a model that needs 20 GB on a 94 GB GPU and roughly three-quarters of the memory — plus most of the compute — goes unused. You paid for the whole card; you're using a sliver of it.

The goal: share one card

The fix is to run multiple workloads on one physical GPU. There are two ways to do this, and they sit at opposite ends of a tradeoff: MIG gives hard hardware partitions, time-slicing gives soft, flexible sharing. The rest of the lesson is that one choice.

Keywords — tap to unfold the plain meaning

Analogy ❝Think of the GPU as one enormous restaurant kitchen built for a banquet. If a single line cook is plating one small order, the other ten stations stand cold and the ovens burn gas for nothing. You either wall the kitchen into smaller private kitchens (MIG), or let several cooks take turns on the one big kitchen (time-slicing). Same kitchen, two ways to share it.

A big GPU running a small model wastes most of itself. The fix is sharing — and the two ways to share, MIG and time-slicing, trade isolation against flexibility.

02

MIG: walling the GPU into private instances

TL;DR · MIG splits one physical GPU into up to 7 isolated instances, each with its own dedicated SMs, HBM slice, and L2 — so one tenant literally cannot touch another's compute or memory.

MIG is the hard option. It doesn't share by taking turns; it builds physical walls. Each slice is, for all practical purposes, its own smaller GPU with guaranteed resources.

One card, up to 7 instances

MIG (Multi-Instance GPU) splits one physical GPU into up to 7 isolated instances. Each instance gets its own dedicated SMs (the compute cores), its own HBM memory slice, and its own slice of L2 cache.

Isolation at the hardware level

The split happens in hardware, not software. One tenant literally cannot touch another's compute or memory. The result is guaranteed performance and no noisy neighbor — a heavy job in one instance can't slow down the others.

Keywords — tap to unfold the plain meaning

Each instance is its own small GPU: dedicated SMs, a private HBM slice, and a slice of L2. The walls are physical, so there's no noisy neighbor.

Analogy ❝MIG bricks the one big kitchen into seven sealed private kitchens. Each cook gets their own counter, their own oven, their own pantry shelf — and a real wall between them. No cook can grab another's ingredients or hog their stove. Whatever chaos erupts next door, your station's speed is guaranteed, because the wall is concrete, not a polite agreement.

MIG carves one GPU into up to 7 hardware-isolated instances, each with dedicated SMs, an HBM slice, and L2. Walls are physical: guaranteed performance, no noisy neighbor.

03

Profiles, and why the limit is rigid

TL;DR · You pick MIG slices from fixed profiles like 1g.10gb, 3g.40gb, or 7g.80gb — and because the partitions are fixed, you can't oversubscribe past the physical card.

MIG's strength — hard walls — is also its constraint. You don't get to invent any slice you like; you choose from a menu of profiles, and the menu can never add up to more than the real GPU.

Profiles name the slice

A MIG profile encodes compute and memory in its name. 1g.10gb is one compute slice with 10 GB; 3g.40gb is three slices with 40 GB; 7g.80gb is the whole card as a single instance. The Ng part is GPU-instance count; the .XXgb part is the HBM slice.

Rigid, and you can't oversubscribe

Because each profile is a real chunk of silicon, the slices you create must fit inside the physical GPU — at most 7 compute slices total. MIG gives hard, guaranteed isolation, but it's rigid: fixed profiles, and you can't pack more logical GPUs than the card physically holds.

Keywords — tap to unfold the plain meaning

Math, decoded

Σ g_i ≤ 7 and Σ mem_i ≤ 94 GB

g_ithe compute-slice count of instance i — the Ng number in its profile
Σ g_i ≤ 7all your instances together can use at most 7 compute slices — the physical ceiling
mem_ithe HBM slice of instance i — the .XXgb number in its profile
Σ mem_i ≤ 94 GBthe memory slices must also fit inside the card's real HBM — no overselling

MIG can't conjure capacity. Your profiles have to add up to no more than the real card: at most 7 compute slices and no more total memory than the GPU actually has. That hard ceiling is exactly why MIG can't oversubscribe.

Profile note Common profiles you'll set on an 80–94 GB card: 1g.10gb (seven tiny instances), 2g.20gb, 3g.40gb, and 7g.80gb (the entire GPU as one MIG instance). Smaller models that fit a slice can each get a private, contention-free instance.

MIG slices come from fixed profiles like 1g.10gb / 3g.40gb / 7g.80gb, and they must fit the physical card — at most 7 slices, no oversubscription.

04

Time-slicing: soft sharing by taking turns

TL;DR · Time-slicing is soft sharing: flexible enough to pack more logical GPUs than physical ones, but with no isolation — a heavy tenant can starve the others.

Time-slicing is the opposite philosophy. Instead of building walls, it lets everyone share the whole GPU by taking turns. Flexible and dense — but nobody is protected from anybody.

One GPU, many logical GPUs

Time-slicing lets several workloads share the full GPU by interleaving their work in time. You can present more logical GPUs than physical ones — oversubscribing the card to pack in more tenants.

No isolation — the noisy neighbor returns

The catch: there's no isolation. Everyone draws from the same compute and memory, so a heavy tenant starves the others. This is the classic noisy neighbor — flexible and oversubscribed, but no hardware guarantee that your job gets its share.

Keywords — tap to unfold the plain meaning

Analogy ❝Time-slicing leaves the one big kitchen wide open and just rotates cooks through it: you get the stove for a few seconds, then the next cook, then back to you. When everyone's plating small orders it feels like more kitchens than you really have. But the moment one cook starts a banquet, they monopolize every burner and pantry shelf — and your "turn" arrives to find the stove already buried. No wall, so no protection.

Time-slicing shares the whole GPU by taking turns: flexible, oversubscribable, dense — but no isolation, so a heavy tenant becomes a noisy neighbor that starves the rest.

05

The core tradeoff — and what your cluster runs

TL;DR · MIG = hard, guaranteed isolation but rigid; time-slicing = flexible and oversubscribed but no isolation. Your 4 H100s run time-slicing ×5 → 20 logical GPUs, MIG disabled.

Two options, one axis: isolation versus flexibility. You can have hard walls or soft sharing, but not both. Knowing where your own cluster sits on that axis tells you what to expect when load spikes.

MIG vs time-slicing, side by side

MIG gives hard, guaranteed isolation — but it's rigid (fixed profiles, can't oversubscribe). Time-slicing is flexible and oversubscribed — but there's no hardware isolation. Same goal (share one card), opposite guarantee.

Your cluster's actual setting

Your 4 H100s run time-slicing ×5, exposing 20 logical GPUs, with MIG disabled. That choice buys flexibility — but means no hardware isolation and a standing noisy-neighbor risk if one tenant goes heavy.

The whole lesson on one axis: MIG trades flexibility for guaranteed isolation; time-slicing trades isolation for flexibility and density.

Keywords — tap to unfold the plain meaning

Cluster note On the 4×H100 box: time-slicing ×5 turns 4 physical GPUs into 20 logical GPUs, and MIG is disabled. The win is flexibility and density; the cost is no hardware isolation — a heavy job on one logical GPU can become a noisy neighbor to the others sharing the same physical card. 4 × 5 = 20.

Analogy ❝Your cluster chose the open kitchens, not the walled ones. Four big kitchens, five cooks rotating through each — twenty cooks who all feel like they have a station. It's wonderfully flexible until one cook fires off a banquet on a shared stove; with no wall, the cooks beside them just wait. You traded the guarantee for the density on purpose — you just have to remember the wall isn't there.

MIG = isolation, rigid. Time-slicing = flexible, no isolation. Your 4 H100s run time-slicing ×5 → 20 logical GPUs, MIG disabled — flexibility bought at the price of noisy-neighbor risk.

06

When to use each

TL;DR · Reach for MIG when you need SLO isolation or small models per slice; time-slicing for bursty, density-first work; the whole GPU when one big model needs all the memory and bandwidth.

The decision isn't "which is better" — it's "what does this workload need?" Three buckets cover almost every case.

Use MIG when…

You're multi-tenant with SLO isolation — different teams or customers who must not interfere — or you have small models that fit a slice and each deserve guaranteed, contention-free capacity.

Use time-slicing when…

Your workloads are bursty, or you want to maximize density — packing as many tenants as possible — and you don't require isolation. Idle gaps from one tenant get filled by another.

Use the whole GPU when…

You're running a large model that needs all the memory and bandwidth. There's nothing to share: the model wants every SM, all the HBM, and the full memory bandwidth of the card.

Keywords — tap to unfold the plain meaning

Analogy ❝If your cooks need ironclad guarantees — separate paying clients who can't wait on each other — wall the kitchen (MIG). If they're a flexible crew handling unpredictable rushes and you care most about feeding the most orders, share the open kitchen (time-slicing). And if you're cooking one giant banquet that needs every burner, every shelf, and the whole pantry at once, give that one cook the entire kitchen — no sharing at all.

Check yourself

How many isolated instances can MIG split one physical GPU into, and what three resources does each instance get dedicated?
What's the one thing MIG guarantees that time-slicing cannot — and what's the price MIG pays for it?
Why can time-slicing present more logical GPUs than physical ones, while MIG can't oversubscribe at all?
Which approach does your 4×H100 cluster use, how many logical GPUs does it expose, and what risk does that choice carry?

MIG for isolation or small-model slices; time-slicing for bursty, density-first sharing; the whole GPU when one big model needs all the memory and bandwidth.