Will it fit?

How the numbers are made

Every number on this page carries one of three labels. Measured: a recipe or an owner on the wiki reported it. Calculated: the formulas below applied to the model's config.json. Assumed: an allowance the configs and the wiki do not give, stated so you can change it.

1. Weights: bytes = parameters x bits per weight / 8. Counted in parts so each can have its own bits: routed experts; attention, shared experts and dense layers; embeddings and output head (16 bits by default, the way most quants leave them); draft layers (MTP or DSpark) and encoders when included (encoders at 16 bits). For a MoE model every expert is held in memory: total parameters set the memory, active parameters set the speed.
2. KV cache per token: 2 x layers x KV heads x head_dim x bytes. Bytes per value: 2 for FP16 or BF16, 1 for FP8 or Q8, 0.5 for Q4. When K and V have different sizes the sum K + V replaces 2 x head_dim. Only layers that keep a cache count: hybrid models count their full-attention layers.
3. MLA cache per token: layers x (kv_lora_rank + qk_rope_head_dim) x bytes. Multi-head latent attention caches one latent per layer instead of K and V per head (GLM 5.2, GLM 5.3, the sparse-attention layers of GLM 5.3 Flash, Mistral Large 4). Sparse-attention indexers add their own keys: layers with an indexer x index_head_dim x bytes.
4. DeepSeek V4 and V4.1 cache per token: layers x head_dim / compress ratio x bytes, plus a window. Their configs have no kv_lora_rank: each entry is one head_dim (512) latent for K and V together, and a layer stores one entry per compress_ratios tokens, plus the last sliding_window (128) tokens raw. V4.1 keeps the long-range cache only on its kv_source_layer_ids and shares it. Indexer keys: index_head_dim / ratio on the layers that own them.
5. Sliding-window layers: layers x per-token size x min(context, window). A sliding-window layer never holds more than its window, whatever the context (MiMo V2.6 Flash, Laguna-S-2.1, the DeepSeek V4 windows).
6. Linear-attention state: layers x heads x key dim x value dim x bytes, per user. Gated DeltaNet (Qwen3.6, Qwen3.8) and KDA (GLM 5.3 Flash) layers keep a fixed state per sequence instead of a cache that grows, plus a small convolution state.
7. Total KV: per token x context x concurrent users, plus window and state x users. Every user is assumed to fill the full context. Engines with a shared KV pool (vLLM, SGLang, llama.cpp with slots) let users share a pool, so the same pool serves more users at shorter contexts.
8. Fit: weights + KV + state + overhead x cards against cards x usable memory per card. Fits at or under 95% of usable memory, tight from 95% to 100%, does not fit above. An even split across cards is assumed; pipeline ranks are uneven (the last rank holds the output head and often the drafter), so leave room on one card.

Memory is in GiB (1,073,741,824 bytes), the unit the unlock sizes are in (64 GB = 65,536 MiB). Parameters are counted from the tensor shapes in each checkpoint, so draft layers, encoders and n-gram tables can be left out or kept off the cards.

Memory per card

• 8 GB card at 64 GB: 63.39 GiB usable (Measured). 64,911 MiB (63.39 GiB) usable of the 65,536 MiB unlock, as the recipe DeepSeek V4.1 Flash on 4x 8 GB (alanoo fidelity build) reports it. The top of VRAM backs the GSP firmware: tail-reserve unlocker forks expose 65,344 MiB or 64,768 MiB, and on builds without the late-PMA fix cuda_memtest passes only when capped at about 63.75 GB. Filling a card into that tail crashes it. Source: Xid 31 and the reserved top of VRAM.
• 10 GB card at 40 GB: 37.50 GiB usable (Measured). The wiki says to plan for about 37 to 38 GiB usable of the 40 GB profile after reservations (one owner measured about 37.7 GiB per card on four cards); the middle, 37.5 GiB, is used. A model that goes over the cap crashes the card and needs a restart. Source: Memory unlock on the 10 GB card.
• 10 GB card at 48 GB: 45.50 GiB usable (Assumed). The wiki gives no usable figure for 48 GB, which needs a good sixth HBM stack (about one card in five). The same 2.5 GiB reservation as the 40 GB profile is assumed: 45.5 GiB. No recipe was measured at 48 GB. Source: Memory unlock on the 10 GB card.
• Overhead: CUDA context, engine workspace, activations and graphs: 1.5 GiB per card is assumed. Engines that take a share of memory (vLLM --gpu-memory-utilization, SGLang --mem-fraction-static) leave the rest outside their pool; where a recipe states its share, the page shows what it left free. (Assumed)

Where each cache size comes from

DeepSeek V4 Flash

• Compressed latent, ratio 4 layers: 21 layers (compress_ratios 4) x 512 head_dim / 4
• Compressed latent, ratio 128 layers: 20 layers (compress_ratios 128) x 512 head_dim / 128
• Indexer keys, ratio 4 layers: 21 layers x 128 index_head_dim / 4
• Sliding window, every layer: 43 layers x 512 head_dim, at most 128 tokens (sliding_window)
• No kv_lora_rank in this config: each layer caches one 512-value latent (head_dim 512, num_key_value_heads 1, of which qk_rope_head_dim 64 carry position) for K and V together. Layers store one latent per compress_ratios tokens plus a 128-token window, as the reference inference/model.py allocates them.

DeepSeek V4.1 Flash

• Compressed latent, KV source layers at ratio 2: 3 layers (kv_source_layer_ids 2, 8, 14) x 512 head_dim / 2
• Compressed latent, KV source layers at ratio 1: 1 layers (kv_source_layer_ids 20) x 512 head_dim / 1
• Indexer keys at ratio 2: 3 layers that own index keys (in both kv_source_layer_ids and index_source_layer_ids) x 128 index_head_dim / 2
• Indexer keys at ratio 1: 1 layers that own index keys (in both kv_source_layer_ids and index_source_layer_ids) x 128 index_head_dim / 1
• Sliding window, every layer: 40 layers x 512 head_dim, at most 128 tokens (sliding_window)
• Only the 4 layers in kv_source_layer_ids keep a long-range cache; the other layers read theirs. No kv_lora_rank in this config: each entry is one 512-value latent (head_dim) for K and V together, as the reference inference/model.py allocates it.
• The Engram n-gram tables (2 layers, 384,006,168 and 384,016,682 rows x 256 engram_head_dim, stored FP8 with one scale per 32 values) are counted on their own: most recipes keep them off the cards.

GLM 5.3 Flash

• MLA latent, sparse-attention layers: 11 layers (layer_types deepseek_sparse_attention) x (512 kv_lora_rank + 0 qk_rope_head_dim)
• Indexer keys, sparse-attention layers: 11 layers x 128 index_head_dim / 4 index_kpool
• Recurrent state, KDA linear-attention layers: 34 layers (kda_layers) x 64 heads x 128 x 128 x 4 bytes (Assumed: the config does not state the state dtype; 4 bytes (fp32) is assumed)
• Convolution state, KDA layers: 34 layers x (4 short_conv_kernel_size - 1) x 3 (q, k, v) x 64 x 128 channels x 2 bytes (Assumed: 2 bytes (the checkpoint dtype) is assumed for the convolution state)

GLM 5.3

• MLA latent, every layer: 78 layers x (512 kv_lora_rank + 64 qk_rope_head_dim)
• Sparse-attention indexer keys: 21 layers with their own indexer (indexer_types full) x 128 index_head_dim

GLM 5.2

• MLA latent, every layer: 78 layers x (512 kv_lora_rank + 64 qk_rope_head_dim)
• Sparse-attention indexer keys: 21 layers with their own indexer (indexer_types full) x 128 index_head_dim

Qwen3.8 Flash Next

• K and V, full-attention layers: 2 x 12 layers (layer_types full_attention) x 2 KV heads x 256 head_dim
• Indexer keys, full-attention layers: 12 layers x 1 indexer_kv_heads x 128 indexer_head_dim / 4 indexer_compress_ratio
• Recurrent state, linear-attention layers: 36 layers (layer_types linear_attention) x 48 linear_num_value_heads x 128 x 128 x 4 bytes (mamba_ssm_dtype float32)
• Convolution state, linear-attention layers: 36 layers x (4 linear_conv_kernel_dim - 1) x (2 x 16 x 128 + 48 x 128) channels x 2 bytes (Assumed: 2 bytes (the checkpoint dtype) is assumed for the convolution state)
• The per-layer n-gram embedding table (PLE, ngram_vocab_size_base 20,000,000 x ple_embed_dim 2560) is counted on its own: most recipes keep it in host RAM or on an SSD.

Qwen3.6 27B

• K and V, full-attention layers: 2 x 16 layers (layer_types full_attention) x 4 KV heads x 256 head_dim
• Recurrent state, linear-attention layers: 48 layers (layer_types linear_attention) x 48 linear_num_value_heads x 128 x 128 x 4 bytes (mamba_ssm_dtype float32)
• Convolution state, linear-attention layers: 48 layers x (4 linear_conv_kernel_dim - 1) x (2 x 16 x 128 + 48 x 128) channels x 2 bytes (Assumed: 2 bytes (the checkpoint dtype) is assumed for the convolution state)

Qwen3.8 27B

• K and V, full-attention layers: 2 x 16 layers (layer_types full_attention) x 4 KV heads x 256 head_dim
• Recurrent state, linear-attention layers: 48 layers (layer_types linear_attention) x 48 linear_num_value_heads x 128 x 128 x 4 bytes (mamba_ssm_dtype float32)
• Convolution state, linear-attention layers: 48 layers x (4 linear_conv_kernel_dim - 1) x (2 x 16 x 128 + 48 x 128) channels x 2 bytes (Assumed: 2 bytes (the checkpoint dtype) is assumed for the convolution state)

MiMo V2.6 Flash

• K and V, full-attention layers: 9 layers (hybrid_layer_pattern 0) x 4 KV heads x (192 head_dim + 128 v_head_dim)
• K and V, sliding-window layers: 39 layers (hybrid_layer_pattern 1) x 8 swa_num_key_value_heads x (192 + 128), at most 128 tokens

Mistral Large 4

• MLA latent, layers that own a cache: 63 layers (distinct kv_source_layers) x (512 kv_lora_rank + 64 qk_rope_head_dim)
• Sparse-attention indexer keys: 44 layers (indexer_types full) x 128 index_head_dim
• 9 of the 72 layers read the KV cache of layer 35 (kv_source_layers), so 63 layers hold one.

Laguna-S-2.1

• K and V, full-attention layers: 2 x 12 layers (layer_types full_attention) x 8 KV heads x 128 head_dim
• K and V, sliding-window layers: 2 x 36 layers (layer_types sliding_attention) x 8 x 128, at most 512 tokens (sliding_window)

The formula against the recipes

What the formula gives at a recipe's own setup, next to what the recipe reports. The gap is what the formula leaves out: engine pools sized by a memory share, quant mixes and runtime buffers.

• GLM 5.3 Flash on 2x 8 GB (ExLlamaV3 layer split): Weights plus KV cache, both cards: Calculated 113.6 GiB, Measured 116.6 GiB; the formula comes out 2.6% under. EXL3 3.05 bpw on every layer, Q8 KV at 392,960 tokens (profile q8_384k), no draft layers or encoders; measured: 116.6 GiB resident across both.
• MiMo V2.6 Flash on 4x 8 GB: Weights, four cards: Calculated 165.3 GiB, Measured 182.0 GiB; the formula comes out 9.2% under. NVFP4 at 4.5 bits on every layer, embeddings and encoders 16-bit; measured: 45.5 GiB of weights per card. The checkpoint is a ModelOpt mixed-precision NVFP4, so some layers are above 4.5 bits.
• DeepSeek V4.1 Flash on 8x 8 GB (Zanooda PP8): Weights, eight ranks: Calculated 285.8 GiB, Measured 299.3 GiB; the formula comes out 4.5% under. MXFP4 experts, 8-bit dense, DSpark layers in, Engram tables off the cards; measured: 35.9 per rank for ranks 1 to 6, 37.6 rank 0, 46.3 rank 7 (posted as GB, compared as GiB).
• DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark): Weights against the official checkpoint size: Calculated 154.7 GiB, Measured 155.4 GiB; the formula comes out 0.5% under. MXFP4 experts, FP8 attention, BF16 embeddings, DSpark layers in; measured: about 155.4 GiB (the checkpoint as stated in the recipe).
• Qwen3.8 Flash Next on 2x 8 GB (vektorprime PP2): Weights, two ranks: Calculated 64.2 GiB, Measured 85.0 GiB; the formula comes out 24.4% under. AWQ INT4 at 4.125 bits, embeddings 16-bit, MTP layers in, PLE table in host RAM; measured: 39.48 and 45.53 GiB of weights on the two ranks.
• Mistral Large 4 on 8x 8 GB (llama.cpp lab trial): Weights against the file size: Calculated 374.1 GiB, Measured 394.0 GiB; the formula comes out 5.0% under. IQ3_XXS at 3.0625 bits on every tensor; measured: the UD-IQ3_XXS files are 394 GiB, because the mix keeps some tensors above IQ3_XXS.
• Qwen3.6 27B on 1x 10 GB (llama.cpp MTP): Weights plus a 4,096-token cache, Q4: Calculated 15,058 MiB, Measured 17,464 MiB; the formula comes out 13.8% under. Q4_K at 4.5 bits on every tensor, MTP layer in, no overhead; measured: 17,464 MiB used at UD-Q4_K_XL (context not stated).
• Qwen3.6 27B on 1x 10 GB (llama.cpp MTP): Weights plus a 4,096-token cache, Q8: Calculated 28,086 MiB, Measured 35,718 MiB; the formula comes out 21.4% under. Q8_0 at 8.5 bits; measured: 35,718 MiB used at UD-Q8_K_XL.
• Qwen3.6 27B on 1x 10 GB (SGLang W8A8): Largest context at one user: Calculated 117,748 tokens, Measured 24,576 tokens; the formula allows 4.8 times the recipe's context. W8A8 at 8 bits, 16-bit KV, 37.5 GiB usable, 1.5 GiB overhead; measured: a 24,576-token pool. SGLang sizes its pool from a static memory share and the recipe also runs an EAGLE draft, so the pool is far below what the memory allows.
• DeepSeek V4 Flash 0731 on 3x 8 GB (PP3 DSpark): Largest context at one user: Calculated 9,673,868 tokens, Measured 691,968 tokens; the formula allows 14.0 times the recipe's context. The cache is about 3.4 KB per token, so memory allows far more than 1M; the recipe's 691,968 was the largest stable context at utilization 0.965, limited by transient buffers that grow with context (Xid 31 page), not by the cache.

Not counted: draft models that ship separately (DFlash2 drafters), the KV cache of draft layers, the DeepSeek compressor's small per-user buffers, and engine-specific cache layouts (some 8-bit caches keep scales and the position part at 16 bits, so they run larger).