product
Data quality: Editorially ReviewedFree
Self-hosted / on-premises
MLOps & model tooling
Private cloud / VPC
ML lifecycle & data
Model Serving
GPU & Compute
On-device / edge
Model inference
Cross-industry / Horizontal
Fine-tuning
NVIDIA TensorRT-LLM
MLOps & model tooling
Overview
TensorRT-LLM is an open-source NVIDIA library that compiles and tunes LLM inference workloads for NVIDIA GPUs. Developers use it to optimize model execution, batching, quantization, and distributed serving.
- Pricing model
- Free
- Deployment
- Self-hosted / on-premises, Private cloud / VPC, On-device / edge
- Regions served
- Global
- Categories
- Free, Self-hosted / on-premises, MLOps & model tooling, Private cloud / VPC, ML lifecycle & data, Model Serving, GPU & Compute, On-device / edge, Model inference, Cross-industry / Horizontal, Fine-tuning
- Last updated
- 6 October 2026
- Published
- 6 October 2026
Use cases
ML lifecycle & data
Deployment options
Self-hosted / on-premises
Private cloud / VPC
On-device / edge
Data sources & provenance
Where this information came from, how it was checked and when it was last looked at. “Last updated” records a change to the record — it does not mean the record was verified.
- Data quality
- Data quality: Editorially Reviewed
- Primary source
- developer.nvidia.com
- Sources cited
- 1
- Last reviewed
- 5 October 2026
- Last updated
- 6 October 2026
Spotted something out of date?
Anyone can suggest a correction — you don't need an account. An editor checks every suggestion against the sources before the record changes.