Inference Engineering · Lesson 23 · Failure Modes & DebuggingHome · Glossary · Your Lab

Failure Modes & Debugging

Classify the symptom before turning knobs.

Commit a prediction before revealing the model.
Today's win: triage common inference failures from request boundary to GPU and choose the first discriminating measurement instead of guessing.

Lesson 22 set quality gates; this is the triage manual for when serving itself breaks. Next, Lesson 24 starts cutting bits per number.

The setup

A delayed meal can be stuck at the door, waiting for a station, missing ingredients, or burning on the stove. Start with where progress stopped; do not buy a faster oven for a ticketing failure.

1 · Reject boundary errors clearly

Unknown model, malformed roles, tokenizer/chat-template mismatch, and prompt+max_tokens beyond context should fail before GPU work. Report token counts and limits, not a generic 500.

2 · Separate memory failures

Weights may not fit at load; KV may exhaust under concurrency/context; runtime workspaces can spike; fragmentation can prevent a large allocation. Inspect free memory, configured utilization, active sequences, context lengths, and allocator logs.

3 · Recognize overload

A growing queue and rising TTFT with stable TPOT indicates admission/prefill pressure. Rising TPOT suggests decode batch, bandwidth, collectives (collectives = cross-GPU communication such as all-reduce; see Lesson 27), or noisy-neighbor pressure. Add capacity only after locating the violated phase.

4 · Check compatibility

Unsupported numeric format (dtype, e.g. FP8 vs FP16; see Lesson 24), compute capability (compute capability = the GPU's CUDA feature-level version), attention backend (attention backend = which attention kernel is used, e.g. FlashAttention; see Lesson 19), model architecture, or compiled shape (a compiled/padded shape = kernels built ahead of time for fixed tensor sizes; see Lesson 26) can fail at startup or silently fall back to a slow path. Record versions and the selected kernels/backends.

5 · Protect request lifetime

Readiness must wait for loaded weights and warmup. Cancellation and shutdown must stop admission, drain or abort active streams intentionally, release KV, and distinguish timeout from server failure.

On YOUR cluster live-tested · course lab

The live cluster provides real examples from different layers: heavy GPU memory reservation, vLLM running/waiting/KV metrics, time-sliced GPUs, and a diagnosed NVLink placement fault. The symptom must be assigned to the right layer before tuning.

Study next: sources & lab companionvLLM troubleshooting · Kubernetes Pod lifecycle

Final check

← Lesson 22Lesson 24 →
References

vLLM troubleshooting · Kubernetes Pod lifecycle

Failure Modes & Debugging

Classify the symptom before turning knobs.

Today's win: triage common inference failures from request boundary to GPU and choose the first discriminating measurement instead of guessing.

The picture

A delayed meal can be stuck at the door, waiting for a station, missing ingredients, or burning on the stove. Start with where progress stopped; do not buy a faster oven for a ticketing failure.

Reject boundary errors clearlyUnknown model, malformed roles, tokenizer/chat-template mismatch, and prompt+max_tokens beyond context should fail before GPU work
Separate memory failuresWeights may not fit at load; KV may exhaust under concurrency/context; runtime workspaces can spike; fragmentation can prevent a large allocation
Recognize overloadA growing queue and rising TTFT with stable TPOT indicates admission/prefill pressure
Check compatibilityUnsupported dtype, compute capability, attention backend, model architecture, or compiled shape can fail at startup or silently fall back to a slow path

1 · Reject boundary errors clearly

Unknown model, malformed roles, tokenizer/chat-template mismatch, and prompt+max_tokens beyond context should fail before GPU work. Report token counts and limits, not a generic 500.

TRIAGE · the symptom names the layer; measure before you tune symptom where did progress stop? rejected before GPU work → BOUNDARY: bad model / template / context (return 400 + counts) queue ↑ · TTFT ↑ · TPOT flat → ADMISSION / PREFILL overload: add capacity, not a faster GPU TPOT ↑ (each token slower) → DECODE: batch · bandwidth · collectives · noisy neighbor OOM only under load → MEMORY: KV / workspace / fragmentation, not the weights startup fail / silent slow path → COMPATIBILITY: dtype · attn backend · compiled shape
Notice the first discriminating measurement is the metric pattern, not a guess: queue growth with rising TTFT but flat TPOT points at admission/prefill, while rising TPOT points at decode. Assign the symptom to a layer first. Adding GPUs will not fix a chat-template rejection or a memory-fragmentation OOM.

2 · Separate memory failures

Weights may not fit at load; KV may exhaust under concurrency/context; runtime workspaces can spike; fragmentation can prevent a large allocation. Inspect free memory, configured utilization, active sequences, context lengths, and allocator logs.

BOUNDARY · fail fast and precisely, before spending the GPU at the door ✗ unknown model ✗ malformed roles ✗ template mismatch ✗ prompt+max > ctx return 400 "prompt 9,000 + max 2,000 > context 8,192" counts + limits, not a stack trace no GPU time spent client can fix and retry a generic 500 hides the cause; a precise 400 with counts is self-service debugging
Notice these failures never need the GPU: they are contract violations the server can name exactly. This matters because returning a 400 with the offending token counts and limits turns a confusing outage into a fix the caller can make themselves, while a generic 500 sends everyone hunting in the wrong layer.

3 · Recognize overload

A growing queue and rising TTFT with stable TPOT indicates admission/prefill pressure. Rising TPOT suggests decode batch, bandwidth, collectives (collectives = cross-GPU communication such as all-reduce; see Lesson 27), or noisy-neighbor pressure. Add capacity only after locating the violated phase.

MEMORY · "OOM" is four different bugs; measure which one weightsfail at loadmodel > VRAM KV cachefail under loadconcurrency × context workspacetransient spikea kernel's scratch fragmentationfree, but notcontiguous first measurement: free vs reserved memory · configured utilization · active sequences · context lengths · allocator log
Notice the timing of the OOM is the clue: weights fail at load, KV fails under concurrency and long context, workspaces spike transiently, and fragmentation blocks a big allocation while total free memory looks fine. This matters because each cause has a different fix, and the metric you read first, not the word “OOM,” tells you which one you have.

4 · Check compatibility

Unsupported numeric format (dtype, e.g. FP8 vs FP16; see Lesson 24), compute capability (compute capability = the GPU's CUDA feature-level version), attention backend (attention backend = which attention kernel is used, e.g. FlashAttention; see Lesson 19), model architecture, or compiled shape (a compiled/padded shape = kernels built ahead of time for fixed tensor sizes; see Lesson 26) can fail at startup or silently fall back to a slow path. Record versions and the selected kernels/backends.

OVERLOAD · the same "it's slow" is two different layers admission / prefill queue depth ↑ TTFT ↑ TPOT flat (tokens still quick) requests wait to start → add capacity / shed load decode pressure TPOT ↑ (each token slower) TTFT may look fine batch · bandwidth · collectives running requests slow down → check batch / neighbor / interconnect
Notice the discriminating metric is TPOT: if tokens stay quick but starts are delayed (queue + TTFT up), the bottleneck is admission/prefill and capacity; if each token gets slower (TPOT up), the bottleneck is decode itself. This matters because the two call for opposite fixes: more replicas versus smaller batch or a fixed interconnect.

5 · Protect request lifetime

Readiness must wait for loaded weights and warmup. Cancellation and shutdown must stop admission, drain or abort active streams intentionally, release KV, and distinguish timeout from server failure.

COMPATIBILITY & LIFETIME · the failures with no error message unsupported config dtype · compute capability · attn backend model arch · compiled/padded shape → startup crash OR silent slow fallback protect the lifetime readiness waits for weights + warmup cancel/shutdown: stop admit, drain, free KV distinguish timeout from server failure record versions + the selected kernels/backends; the slow path often logs nothing at all readiness that flips green before warmup sends traffic into cold-start latency
Notice the dangerous case is not the crash but the silent fallback: an unsupported attention backend or compiled shape can quietly drop to a slow path that emits no error. This matters because only recording the selected kernels/backends and versions reveals it, and readiness must gate on warmup so the load balancer never routes to a cold replica.

On YOUR cluster live-tested · course lab

The live cluster provides real examples from different layers: heavy GPU memory reservation, vLLM running/waiting/KV metrics, time-sliced GPUs, and a diagnosed NVLink placement fault. The symptom must be assigned to the right layer before tuning.

Check yourself

← Lesson 22Lesson 24 →
References

vLLM troubleshooting · Kubernetes Pod lifecycle