What happened
- DeepSeek published DeepInfer on GitHub under the Apache 2.0 license. It is an inference framework (the program that runs an already trained model to answer queries) that, according to its own README, powers its production services.
- This first release covers offline batch inference and a single model, DeepSeek-V4.1-Flash, with the checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017 as the default.
- It runs on Nvidia GPUs with the SM100 architecture and on Huawei’s Ascend 950 NPUs. It requires Linux, Python 3.12, CUDA 13 on Nvidia and CANN 9.2 on Ascend.
- The README states what it leaves out: a KV cache system, separation of the prefill and decode phases, and load balancing across replicas.
Why it matters
- It is the second piece in a month. In September DeepSeek uploaded its DeepGEMM and DeepEP libraries for Ascend to GitHub. Now it publishes the engine that uses them.
- For a Chilean company renting GPUs, the effect is not immediate. Without caching or balancing, DeepInfer is useful for testing the model on your own hardware, not for serving thousands of users at once.
- The consequence the README does not mention: an open model that runs with the same code on two manufacturers lowers the cost of switching compute providers. Anyone negotiating GPU prices in Latin America has an argument they did not have before.
- The wrapper is open, but provenance matters. The code comes from a company that three US security bodies named in September over distillation. Reviewing the source code is part of adopting it.
The number
2 chip families. Nvidia SM100 and Huawei Ascend 950, with a single inference codebase.
Context
The repository gives no release date. It shows performance curves for prefill and decode, measured on 4 SM100 GPUs for the first phase and 64 for the second, but gives no performance figures in the text. Until third parties measure, performance is a claim by the company.
What’s next
- No roadmap or deadlines announced in the repository.
- Pending: the three missing pieces, which will decide whether anyone outside DeepSeek uses it in production.
- Pending: independent measurements on Ascend 950 and SM100 with the same model.
Bottom line
In September DeepSeek opened its tools for Ascend. The engine completes the package, with the caveat that what is needed to operate at scale remains unpublished.
Sources
Edited by Rodrigo Cornejo. How we select and verify: who writes these notes.
