French AI startup ZML has released its LLM inference server "ZML/LLMD" as a free product. TechCrunch reported on this on July 8, 2026, and ZML's official page also lists LLMD as a "universal LLM server." Supported models include LLaMa and Gemma, as well as the Qwen and Mistral families. Available execution targets include NVIDIA CUDA and AMD ROCm. Google TPU, Intel oneAPI, and Apple Metal are also covered. This is not an announcement of a new foundation model. Rather, it's a product designed to keep the inference server that companies use to run AI from being locked into a specific accelerator.
Companies struggling with GPU shortages or inference costs have good reason to try such a product. The more AI applications become part of everyday use, the more the cost and operation of inference—rather than training—comes to the forefront. ZML addresses this by unifying the server-side entry point while using chip-optimized execution binaries under the hood. Just how much faster and cheaper this makes things remains to be verified. Even so, it's now clear that competition in inference infrastructure is expanding from model performance to freedom in the execution environment.
Placing Five Accelerators Behind the Same Entry Point
ZML/LLMD's selling point is the breadth of supported chips it puts front and center. The official page shows a CUDA-based Docker image for NVIDIA, a ROCm version for AMD, and a oneAPI version for Intel. There's a TPU version for Google TPU, and installation via Homebrew is described for Apple Metal. According to the platform sizes listed on the page, CUDA is 1.7GB and ROCm is 3.9GB. TPU comes in at 280MB, oneAPI at 350MB, and Metal at 140MB.
The product also comes equipped with the functionality expected of an inference server. ZML has built continuous batching, Paged Attention, and tensor-parallel sharding into LLMD. It also supports Prefix Caching, Tool Calling, and Prometheus metrics output. There's a mechanism for loading models from Hugging Face, S3, and Google Cloud Storage, using paths like hf://, s3://, and gs://. This design reduces the need to separately download and place models beforehand, allowing the server to load them directly from wherever they reside.
What particularly stands out on the performance side is DFlash speculative decoding. ZML's LLMD page claims that DFlash delivers a 10x speedup for supported models. Native support for the Gemma 4 series has already shipped, with Qwen support planned. However, this figure comes from ZML itself and is limited to specific models—it doesn't mean every model on every chip will run 10x faster. Separating out this distinction is the starting point for evaluating LLMD.
The Distance Between Free Release and Open Source
According to TechCrunch, ZML/LLMD starts out free, but it is not open source. This differs in character from ZML's first publicly released project. In 2024, ZML released a machine learning framework for inference on GitHub, and that repository uses Zig, MLIR, and Bazel. In addition to NVIDIA, AMD, and Intel, it also advocates a configuration that compiles directly to TPUs and Trainium. In the March 2026 announcement of ZML/v2, the company explained that it had completely rewritten the framework to explicitly handle platforms, compilation, and memory. I/O and placement were also brought into the same design scope.
LLMD sits closer to being the entry point for a commercial product built atop that foundation. TechCrunch reports that ZML founder Steeve Morin said he wants to measure usage and monetize in a way that doesn't hinder growth. In other words, the free release should be seen not as a promise of permanent free access, but as a launch strategy for gathering adoption and usage data. For companies, this leaves both the benefit of being able to try it for free and the risk that future pricing and licensing remain undetermined.
Not being open source also affects adoption decisions. With public projects like vLLM and SGLang, it's easier for outsiders to track performance improvements and bug fixes. With LLMD, adoption decisions hinge on how much ZML is willing to disclose about internal implementation, benchmarks, and failure behavior. Especially for a server spanning multiple accelerators, there's a gap between simply appearing on a compatibility chart and actually handling long contexts and concurrent requests in production, while withstanding memory pressure and driver differences.
Expanding Choices Beyond NVIDIA
ZML's aim isn't to squarely reject NVIDIA. According to TechCrunch, Morin said he isn't bearish on NVIDIA and that the company maintains a good relationship with it. Rather, what LLMD is trying to move forward isn't a binary choice between using NVIDIA or not, but increasing the number of chips usable for each inference workload. If companies can combine CUDA, ROCm, and TPU depending on the model or use case, their procurement options expand. And if oneAPI and Metal are brought into the same entry point, power and pricing negotiations become easier too.
This direction also carries significance for European AI chip companies. TechCrunch reports that Morin mentioned Axelera, Fractile, Kalray, and OLIX. Q.ANT, SiPearl, SpiNNcloud, and VSORA also came up in the same context. Emerging chip companies can't win customers on hardware alone—they need a software layer where existing models and inference servers run, and that developers can operate. A cross-cutting stack like ZML's could serve as a conduit connecting such chips to AI use cases.
Competition in this space is already substantial. TechCrunch lists vLLM, SGLang, and Baseten as entities that partially compete with ZML/LLMD. Here, ZML is attempting to push down into a layer below a mere feature checklist for inference servers. Morin told the outlet that ZML has reached the point of co-designing silicon itself. If software moves from the stage of simply using a specific chip to the stage of feeding requirements back into chip design, competition in inference infrastructure will keep descending into lower layers.
What to Watch Next: Benchmarks and Pricing
LLMD's value can't be determined by the compatibility chart in a press release alone. What's needed first are independent benchmarks under the same model, same load, and same precision conditions. DFlash's claimed 10x speedup is compelling, but it's limited to supported models in the Gemma 4 series. Qwen support is only planned, and other models will likely yield different results. We need to see how startup time and throughput change across the CUDA, ROCm, and TPU versions. Measured latency and memory usage are also needed for the oneAPI and Metal versions.
Pricing remains undetermined as well. TechCrunch reports that ZML is a small team of 20 people that has raised $20 million. Investors include 20VC, >commit, AALVC, and Drysdale Ventures. Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures also participated. While that's not a small amount of funding, continuously maintaining a multi-chip inference server requires keeping pace with changes in each vendor's runtime, drivers, and model formats. There are limits to how long one can keep expanding maintenance scope while remaining free.
ZML/LLMD demonstrates that the main battleground for inference is shifting from the surface of model APIs toward questions of which chip, which server, and how stably it can be run. The numbers companies will look at next aren't single-shot peak speeds. What matters is how well the system holds up under real-world congestion, whether mixing multiple chips complicates management, and whether pricing after the free release can retain a compelling reason to switch from an existing stack. If LLMD can clear that bar, the range of choices in inference infrastructure will change more significantly than appearances suggest.