DeepSeek Deployment Selection: A Three-Step Guide from Precision, VRAM to GPU Model

1. The First Step in DeepSeek Deployment Is Not Choosing a GPU, but Calculating the VRAM Tiers

Many teams think the first question in DeepSeek deployment is "which GPU", but a more logical order is to calculate VRAM first. Because at different precisions, the same model can have drastically different VRAM requirements. For example, with the DeepSeek-R1 distilled series, at native FP16/BF16 precision, the 32B and 70B models require 64GB-181GB of VRAM—a range wide enough to eliminate single-GPU options. With FP8 or Q4_K_M quantization, VRAM needs drop to 18GB-45GB, making single RTX 4090/5090 or A100/H100 viable.

Understanding this VRAM tier shift from "native" to "quantized" is the first lesson in DeepSeek deployment selection. This article does not discuss model capabilities; it only provides engineering capacity estimation to help you avoid the pitfall of "buying a GPU first, then finding out VRAM is insufficient."

Note: The VRAM ranges in this article come from the public NVIDIA NIM model specification page for DeepSeek-R1-Distill-Qwen-32B and two publicly available VRAM/KV cache estimation articles from 2026 (three public sources in total). All numbers are range-level estimates, not actual tokens/s or precision loss measurements. Actual usage must be validated in your own environment.

2. FP16 Native vs. FP8/Q4_K_M Quantized: VRAM Range Comparison for 32B and 70B

Let's look at VRAM changes from quantization. According to public estimates, the DeepSeek-R1 distilled 32B model after FP8 or Q4_K_M quantization requires about 18GB-24GB for weights; the 70B model requires 38GB-45GB. The range width comes from differences in quantization schemes (FP8 vs. Q4_K_M), vocabulary size, and runtime overhead. It's recommended to reserve the upper bound to avoid OOM due to implementation details.

The table below maps precision, VRAM, and GPU models for quick reference:

Note: Both rows in the native precision column fall within 64GB-181GB; the 70B is significantly higher than the 32B, so plan according to the upper bound. For the question "DeepSeek deployment Q4_K_M quantized VRAM usage", the table is your answer: use this range as the planning baseline for weight VRAM after Q4_K_M, and reserve the upper bound.

3. Don't Just Calculate Weights: How KV Cache, Context Length, and Concurrency Consume Remaining VRAM

Weight VRAM is just the foundation; what really causes VRAM to run out is the KV cache. It grows with the number of transformer layers, KV head count, head dimension, context length, and concurrency (batch size). For example, when processing long documents or high concurrency, the KV cache can easily exceed the weight footprint.

Fortunately, inference frameworks like vLLM and SGLang offer optimization: enabling FP8 KV cache quantization compresses cache size, and Prefix Caching can reuse common prefix caches, significantly reducing VRAM pressure.

To estimate the KV cache, use this formula: KV Cache ≈ layers × KV heads × head dimension × 2 (K and V) × sequence length × concurrency × bytes per element. You can read each parameter from the model config; the byte count varies with FP16/FP8 KV cache settings. First calculate the KV cache for your longest context and peak concurrency, then check if enough VRAM remains for weights.

4. Mapping VRAM Tiers to GPU Models: Boundaries for Single 4090/5090, Single A100/H100, and Multi-GPU Nodes

Now let's map the VRAM tiers to specific GPU models.

• 18GB-24GB tier: Corresponding to quantized 32B, a single RTX 4090 (24GB) or RTX 5090 (32GB) is sufficient, but you need to reserve headroom for KV cache and concurrency. It's recommended to enable quantized cache compression.
• 38GB-45GB tier: Corresponding to quantized 70B, a single A100/H100 (80GB) is most stable, or dual RTX 4090/5090 using tensor parallelism.
• Unquantized scenarios: Native FP16/BF16 requires 64GB-181GB, which even an 80GB single GPU cannot cover, so you need multi-GPU or multi-node setups.

For the question "Can a single RTX 4090 run DeepSeek deployment?", the answer is: yes for quantized 32B, but the 70B requires dual GPUs or a higher-VRAM card. As for "A100 vs. H100 for DeepSeek deployment", it depends on your service scale: the A100's 80GB VRAM is enough for the quantized 70B model, while the H100 offers higher throughput for high-concurrency scenarios.

When selecting a GPU, total VRAM = weight upper bound + KV cache estimated from target context length and concurrency + framework and fragmentation overhead. Calculate these three separately before choosing a GPU; don't rely on a fixed multiplier.

5. Trade-offs from Quantization: How to Validate Throughput, Latency, and Output Quality Yourself

FP8 and Q4_K_M are not lossless; they trade off numerical precision, affecting output quality, but the loss cannot be generalized. Therefore, you need your own validation process:

1. Pick a representative set of prompts covering code, reasoning, long-text, etc.
2. Compare model outputs before and after quantization on long-context and chain-of-thought tasks.
3. Monitor VRAM usage and concurrency limits; observe any OOM or performance degradation.

This way, you can find the right balance among throughput, latency, and output quality for your business. Don't blindly trust claims of "lossless quantization"; always test in practice.

6. Use On-Demand Resources on NexGPU for a VRAM and Throughput Validation

Theory is no substitute for hands-on testing. NexGPU, as a GPU cloud computing platform, offers various GPU server models, pay-as-you-go usage, and instant startup, making it ideal for running a validation with hourly resources.

Action steps:

• Select a target GPU model (e.g., RTX 4090 or A100), start a vLLM/SGLang service on demand.
• Load the quantized model, enable FP8 KV cache and Prefix Caching.
• Stress-test the target context length and concurrency; observe VRAM usage and throughput.
• Use the data to decide whether to use single or multi-GPU nodes in the long term, avoiding locking into the wrong spec.

After this round, you'll have your own evidence-based GPU cloud server configuration instead of copying someone else's recommendations.

7. DeepSeek Deployment Selection Checklist and Common Pitfalls

Finally, here's an actionable checklist:

• Confirm weight precision and quantization format (FP8, Q4_K_M, etc.) and use official or trusted model files.
• Reserve VRAM based on the upper bound, e.g., at least 24GB for quantized 32B, 45GB for 70B.
• Calculate KV cache separately, based on context length and concurrency, and consider compression.
• Leave extra headroom for VRAM fragmentation and burst concurrency; size it based on peak VRAM from stress tests, not a fixed percentage.
• Validate with hourly on-demand resources before committing to a long-term spec.

Common mistakes include: only estimating weight VRAM, ignoring long contexts and concurrency; blindly buying the highest-end GPU, wasting resources; and not validating quality after quantization before going live. Avoid these pitfalls, and your DeepSeek deployment will go much smoother.

Remember, the core of selection is to match actual needs with data-driven decisions.