NVIDIA’s TensorRT Model Connect Skips ONNX, but x86_64 Users Must Build From Source

NVIDIA has released TensorRT Model Connect, or TRTMC, in public preview: an Apache-2.0 tool that takes supported Hugging Face or local model checkpoints to TensorRT inference in two commands, without an intermediate ONNX export. Its central promise is operational: teams can move the resulting artifact into a native C++ application without carrying PyTorch through the runtime path.

The bundle becomes the handoff

TRTMC accepts a model ID or local checkpoint and creates a deployment bundle containing TensorRT engines and runtime-specific assets. The resulting versioned .bundle file loads through native C++ task APIs, creating a defined boundary between model building and the software that serves inference.

NVIDIA frames the conventional route as PyTorch through ONNX or TorchScript to TensorRT, followed by model-specific C++ integration. TRTMC instead exposes task-level inputs and outputs for text, images and audio, with a semantic API for more managed deployment and a module-level API for finer control.

A command-line build, then a native target

The documented quick start builds Qwen3-0.6B at bf16 precision and runs the bundle from the command line. The same bundle can be loaded from C++, making the command-line step a build and packaging path rather than the required production-serving environment.

trtmc build Qwen/Qwen3-0.6B --precision bf16 --max-cache-length 16384 --output qwen3-0.6b.bundle
trtmc run ./qwen3-0.6b.bundle --prompt "What is the capital of France? Answer in one word." --chat-template --no-thinking

PyTorch-free has an exception

For native profiles, inference can run in C++ without PyTorch in the runtime path. The design is not universal: a small number of hybrid profiles invoke a helper Python executable, and their manifests explicitly declare that dependency.

TRTMC also supports custom GPU kernels through TVM FFI, offering a route to extend specialized deployments beyond the provided task APIs.

The installation constraint is immediate

Current release wheels target Linux aarch64 only, with Python 3.10 or 3.12, glibc 2.39 or newer, and TensorRT 11.1.0.106. No x86_64 wheels are published, so users on that architecture must build from source through Docker.

NVIDIA’s July 29 GB300 snapshot covered 105 profiles across 76 model families; 102 beat their declared reference by more than 5%. That company-reported result is a snapshot rather than a guarantee for every checkpoint, hardware configuration or workload.

NVIDIA says it uses nightly releases and automated validation to support new models and architectures. It also says Codex agents built the project’s implementations, tuning, tests, integrations and documentation under human direction and review.