NVIDIA released TensorRT-LLM 1.3.0 as a stable build on 2026-11-20 after 30 release candidates spanning 302 days, though 1.4.0 candidates were already public. The release introduces preview support for Rubin (SM107), DeepSeek-V4, Kimi K3, and MiniMax-M3 on Blackwell/Rubin; a new Anthropic Messages endpoint (/v1/messages), gRPC server, multi-process HTTP frontends, and Prometheus metrics; speculative decoding drafters (DSpark, DFlash, Suffix Automaton, PARD, EAGLE3 dynamic trees); KV Cache Manager V2, NVFP4 KV cache, cold-page compression, and FlexKV/LMCache connectors; and a VisualGen vertical for image/video generation. Infrastructure shifts to PyTorch 2.14.0, CUDA 13.4.1, C++20, Python 3.10/3.12, and drops TensorRT, ModelOpt, and ONNX as dependencies. Thirteen breaking changes require auditing: six removals (TensorRT backend/trtllm-build, legacy C++ Executor, TensorRT Triton models, TRTLLMSampler, two-model speculative decoding, AutoDeploy/Star Attention/WIDEEP); three silent default changes (CLI flags now override YAML, KV transfer timeout defaults to 60 seconds via cache_transceiver_config.kv_transfer_timeout_ms, per-request chat templates require --allow_request_chat_template); and four API changes (logprobs capped at 100, /perf_metrics replaced by X-TRTLLM-return-metrics header, dynamic speculation fields renamed to acceptance_rate_window_size/acceptance_rate_threshold, obsolete flags removed). Telemetry is enabled by default; opt out via TRTLLM_NO_USAGE_STATS=1, DO_NOT_TRACK=1, --no-telemetry, or TelemetryConfig(disabled=True). GPT-OSS defaults to TRTLLM MoE backend on Blackwell; TRITON MoE deprecated with three-month migration. Known issues include disaggregated DSpark KV transfer gaps, GLM-5.2 NVFP4 segfault on GB200 above 4096 max_num_tokens, and /v1/responses compatibility gaps.
Read original
dev.to