On October 15, 2026, NVIDIA released Nemotron 3.5 Super VL, an open AI model that combines image and document understanding with tool use. It has 120 billion parameters in total, but uses only about 12 billion to process each token, and it supports context lengths of up to 1 million tokens. It is designed to handle an entire workflow in a single model: reading the contents of a document, checking internal business rules, and carrying out the work using multiple tools. However, anyone deciding whether to adopt it needs to look beyond generation speed and also check how quantization affects accuracy, as well as operating requirements such as GPUs and memory.
One model handles image understanding, reasoning, and tool use
Nemotron 3.5 Super VL is a multimodal model that accepts text, images, and video as input and outputs text. It can read documents that contain tables and charts, reason based on their contents, and call external tools. According to NVIDIA, it can, for example, link a table and a chart that appear on different pages of a document and summarize their contents. It does not support audio input.
For image understanding, it uses a vision encoder called "C-RADIO," which converts images into an internal representation and passes it to the language model through a connector. The language model side uses a hybrid architecture that combines Mamba-2 with Transformer attention. A Mixture of Experts (MoE) structure, which selects and computes only the necessary experts from among multiple expert networks, keeps the number of parameters used per computation low. The aim is to handle everything from visual understanding to subsequent reasoning and tool calls in one model, reducing the effort of combining multiple models.
The context length of up to 1 million tokens is also shared across this entire workflow. In the official model card, NVIDIA explains that "the prompt, visual inputs, conversation history, tool results, and generated response share this context window" (translated). In other words, 1 million tokens is not simply the length of document the model can read; it is an upper limit that includes the conversation history and tool execution results accumulated during the work. When an agent runs for a long time, this history also consumes context, and the amount of documents that can be input is limited accordingly.
The model also incorporates mechanisms for faster inference. One is Multi-Token Prediction (MTP). It generates several candidate upcoming tokens at once, and the main model verifies them in a single computation. If multiple candidates are accepted, the number of computations required for text generation can be reduced. In addition, prefix caching reuses the computation results for common system instructions, tool definitions, conversation history, and so on, easing the burden of repeated processing.
In addition to the trained weights, NVIDIA is releasing the datasets and training procedures. Companies that want to further train the model on their own documents and business rules can use "NeMo," and according to the model card, the model can be used for both commercial and non-commercial purposes under the "OpenMDW-1.1" license. NVIDIA's Nemotron series also includes "Lightning," which handles large-volume text processing, and "Ultra," which handles complex planning. Super VL is positioned as the model for multistep tasks that require visual information.
Up to 5x the throughput claimed, but quantization lowers accuracy in some tasks
NVIDIA says Nemotron 3.5 Super VL achieves up to 5 times the throughput of open models of similar size. In the comparison shown in the technical explanation, using a 100,000-token input, it recorded an output speed of more than 600 tokens per second on a Baseten endpoint. However, this is a generation speed obtained in a specific inference environment under specific measurement conditions, and it does not mean that every real-world task will finish five times faster. Total processing time, including document loading and external tool execution, needs to be evaluated separately.
When deploying the model, you can choose between the BF16 format and the NVFP4 format, which reduces memory usage. On NVIDIA's Blackwell architecture, the NVFP4 version processes both weights and activations in 4-bit; in other environments, it handles weights in 4-bit and activations in 16-bit. It uses a mixed-precision approach that keeps some components, such as the vision encoder, at higher precision, so not all computation is uniformly lowered in precision. NVIDIA also uses distillation, training with the output of the BF16 version as the teacher, to limit the accuracy loss that comes with quantization.
Still, the effect of quantization on accuracy varies depending on the task being evaluated. The official model card published on October 15 includes comparison results for the BF16 and NVFP4 versions, with conditions other than the model, such as runtime settings and evaluation software, held constant. Results for representative benchmarks are as follows.
| NVIDIA evaluation item | BF16 | NVFP4 |
|---|---|---|
| SWE-bench Verified | 71.10 | 71.07 |
| Terminal-Bench 2.1 | 66.70 | 60.80 |
On SWE-bench Verified, the BF16 and NVFP4 scores were nearly identical. On Terminal-Bench 2.1, however, the NVFP4 score was 5.90 points lower than the BF16 score. This is a score difference (60.80 − 66.70 = −5.90), not a relative rate of change. Both sets of results are NVIDIA's own evaluations, and they do not establish the statistical significance of the differences or the impact on quality in actual work. Even so, they show that the effect of memory savings on accuracy differs by task and cannot be judged from a single benchmark.
Performance evaluations also change depending on the number of concurrent requests. In measurements by the inference service provider Blackbox AI using a single B300 GPU, Nemotron 3.5 Super VL generated 521 tokens per second for a single request and 218 tokens per second per request with 10 concurrent requests. "Gemma 4 31B," run on the same GPU, recorded 357 and 228 tokens per second, respectively. Super VL was ahead on single requests, but Gemma 4 31B was slightly ahead at 10 concurrent requests, so speed on a single request does not necessarily translate into an advantage under parallel processing.
The measurements were taken on October 7 using 30 prompts of about 10,000 tokens each. They were run three times with reasoning enabled, and the median was used; output token counts include the reasoning portion. The output was also capped at 1,500 tokens. These are only generation-speed measurements by a service provider, not results comparing answer quality or operating costs.
Blackbox AI also tested the speedup from MTP. In an experiment using the same six long prompts, enabling MTP raised single-request generation speed about 2.5 times, from 198 tokens per second to 492. Time to first token, meanwhile, barely changed, going from 0.41 seconds to 0.40 seconds. A mechanism that boosts generation speed does not shorten input processing or waiting for external tool responses by the same proportion, so in real work it is important to measure the time it takes for the whole task to finish.
It can run on a single GPU, but long-running operation needs a lot of memory
Quantization lets Nemotron 3.5 Super VL run on significantly fewer GPUs. NVIDIA NIM's deployment documentation lists the following configurations depending on the GPU type and model precision.
| GPU | NVFP4 | BF16 (LoRA supported) |
|---|---|---|
| B200 | 1 | 2 |
| H200 | 1 | 4 |
| H100 80GB | 2 | 8 |
The NVFP4 version can be deployed on fewer GPUs than the BF16 version, and it can run on a single B200 or H200. However, the NIM profiles that support LoRA adapters are limited to the BF16 version. LoRA is a technique for efficiently adapting the original model to a specific use by training a small number of additional parameters. The GPU configuration you need also changes depending on whether you use the model as-is for inference or apply the results of additional training on your own data as an adapter.
In addition, for long-running agent processing, not only GPU memory but also CPU-side host memory becomes important. NIM includes a profile called "agentX," intended for coding work in which context grows longer as tool calls are repeated. Because this configuration offloads the KV cache, which holds computation results such as conversation history, to the CPU side, it requires 200 GiB of host memory per GPU, separate from the memory the container normally uses. With two H100s, that comes to 400 GiB, but this is a requirement specific to agentX and does not mean every deployment environment needs the same capacity.
When running the model locally, the number of concurrent requests also needs attention. In the launch example for DGX Spark in the official model card, concurrency is set to one request, and "LiLiCorr" is used to generate token candidates. The context length of up to 1 million tokens is a specification of the model itself, but in an actual inference environment, the available context length may be set shorter depending on GPU memory capacity and the number of concurrent requests. Therefore, running on a single device, being able to use the maximum context length, and handling many requests at the same time are separate conditions that each need to be checked.
In large-scale environments that use many GPUs, on the other hand, optimization of the inference infrastructure itself also affects performance. In a test using its inference platform "Dynamo," NVIDIA says that with 16 B200 GPUs, under the condition of maintaining a generation speed of 50 tokens per second per user, it increased the number of supportable users by up to 8 times and cut time to first token by 85%. This is the result of combining mechanisms that route requests to appropriate GPUs and reuse or offload caches, and it does not indicate the model's own generation speed on a single GPU.
The defining feature of Nemotron 3.5 Super VL is that one model can handle everything from visual understanding to reasoning and tool use. But whether that advantage can be put to use in actual work depends on post-quantization accuracy, performance under concurrent loads, and the memory capacity needed to hold long work histories. Companies considering adoption should test the full sequence of reading their own documents and running tools, using the number of concurrent users and the context length they expect. Comparing the task success rate and completion time from that test with the cost of acquiring and operating GPUs and host memory will help them judge which tasks make Super VL worth adopting.