DeepSeek-Coder-V2 is an open-source Mixture-of-Experts (MoE) code language model available in two configurations:
- DeepSeek-Coder-V2-Lite: 16 billion total parameters with 2.4 billion active parameters per token.
- DeepSeek-Coder-V2: 236 billion total parameters with 21 billion active parameters per token.
Both models support a context length of 128,000 tokens. citeturn0search0
The hardware requirements for these models are not explicitly detailed in the official documentation. However, based on the model sizes and typical resource needs for similar large-scale language models, the following table provides an estimated guideline:
| Model | Total Parameters | Active Parameters | Minimum GPU Memory | Recommended GPU Memory | Number of GPUs | Disk Space | System Memory (RAM) |
|---|---|---|---|---|---|---|---|
| DeepSeek-Coder-V2-Lite | 16B | 2.4B | 24 GB | 32 GB | 1 | 500 GB | 64 GB |
| DeepSeek-Coder-V2 | 236B | 21B | 40 GB | 80 GB | 4 | 2 TB | 128 GB |
Notes:
- GPU Memory: Running these models efficiently requires high-memory GPUs. For the Lite version, a single GPU with at least 24 GB of memory (e.g., NVIDIA RTX A6000) is suggested. The full 236B model may necessitate multiple GPUs with a minimum of 40 GB each (e.g., NVIDIA A100 40GB) to handle the active parameters during inference.
- Number of GPUs: The full 236B model’s active parameter set is substantial, and distributing the load across multiple GPUs can enhance performance and manageability.
- Disk Space: Storing the model weights and associated data requires significant disk space. The Lite model may need around 500 GB, while the full model could require up to 2 TB.
- System Memory (RAM): Adequate RAM is essential to support data preprocessing and model inference. The Lite model should function with 64 GB of RAM, whereas the full model may benefit from 128 GB or more.
These estimates are based on typical requirements for large-scale language models and may vary depending on specific use cases and system optimizations. For precise hardware specifications, consulting the official DeepSeek-Coder-V2 documentation or reaching out to the development team is recommended.