https://zml.ai/tags/nvidia/ Recent content in NVIDIA on ZML - Model to Metal Hugo en-us Wed, 08 Jul 2026 08:00:00 +0100 https://zml.ai/posts/llmd/ Wed, 08 Jul 2026 08:00:00 +0100 https://zml.ai/posts/llmd/ <p>Today we&rsquo;re releasing ZML/LLMD. It&rsquo;s a self-contained inference server that runs LLaMa, Gemma, Qwen and Mistral LLMs transparently on <strong>5 architectures</strong>: <strong>NVIDIA CUDA</strong>, <strong>AMD ROCm</strong>, <strong>Google TPU</strong>, <strong>Intel oneAPI</strong> and <strong>Apple Metal</strong>.</p> <p><img src="https://zml.ai/posts/llmd/llmd1.png" alt=""> </p> <h2 id="modern-serving-features">Modern serving features</h2> <p>ZML/LLMD supports modern serving features: <strong>continuous batching</strong>, <strong>paged attention</strong>, <strong>tensor parallel sharding</strong>, <strong>prefix caching</strong>, <strong>tool calling</strong> and does so on <strong>all platforms</strong>.</p> <p>Metrics are also exposed in the Prometheus format via the <code>/metrics</code> endpoint.</p>