Install Ollama to use the commands below. Builds are sorted by the original repository’s 30-day Hugging Face downloads. Sizing examples show where more memory changes your options.
Context changes the memory estimate. Set the actual context in your runtime separately.
Check the deployment guide for the GPU count, runtime version and context length your project needs.
Thinking Machines Lab
Inkling 2 Small
Released
A text and image reasoning model with a long context window.
594B total · 17B active
Safetensors · NVFP4 / MXFP8
Apache 2.0
Approx. weight download
394 GB
3,647 Hugging Face downloads · 30 days
Publisher sizing examples are 4 × H200 or 2 × B300. Verified SGLang recipes use 8 × H200 or 4 × B300. Confirm the runtime and GPU layout for your project.
We convert the registry download size from GB to GiB, add 2 GiB per 4,096 context tokens, then add the larger of 8 GiB or 20% of the model size for runtime overhead. This is a screening rule for one active request, not a model-specific KV cache calculation. Image inputs, parallel requests and some runtimes can exceed it. The estimate covers the listed Ollama build only.
How does the hardware filter work?
It shows builds that fit within our planning budget: 96 GiB for a 128 GB GB10 system, 80 GiB on a single 96 GB GPU, or 120 GiB on a single H200 NVL GPU. The shared-memory budget leaves room for the host. The W1/W2, W3 and S4 profiles represent ZXO configuration targets. W3 and S4 also show builds above the one-GPU budget as candidates for multi-GPU review when their estimate is below the combined planning allowance. GPU memory is separate on each card. Actual fit depends on the runtime, interconnect and how it splits the model.
What does ZXO measure?
A performance benchmark records the exact model build, host, context length and measured generation speed. The Qwen3 8B pilot example shows document Q&A checks. Plan to measure speed and first-token latency on the intended host.
What do download counts measure?
Counts are snapshots of the original publishers’ Hugging Face model repositories over the previous 30 days, checked . The separate H3 count is for ComfyUI’s workflow repository. Hugging Face counts requests to selected files, so these figures do not measure unique users, completed weight downloads or pulls of the listed Ollama build.
Where do weights and credentials go?
Commands run on your machine and fetch the model from the listed registry. Review each publisher’s license and terms before deployment. For a gated Hugging Face checkpoint, accept the publisher’s terms there and authenticate locally with hf auth login. ZXO does not collect access tokens or host model weights in this catalog.
05 / Deployment tools
Start with a local toolkit.
Get a sample Ollama Modelfile, vLLM and Open WebUI setup, and a benchmark script for first-token latency and generation speed. Review the prerequisites for your runtime and host.
Configuration templates. Includes no model weights or benchmark results.
06 / Local AI Pilot
Test it with your workload.
We’ll scope a four-week pilot around one workflow and agree on what to measure. Need an offline install? Tell us which models and network restrictions apply so we can review what can be packaged.