NVIDIA NeMo Relay Adds Native Support for Oracle’s Generative AI Service

An AI agent can give a convincing answer about work it never did. Oracle says NVIDIA NeMo Relay now natively supports OCI Generative AI, giving teams a way to record the model and tool calls behind an answer, whether they use Oracle’s managed service or run a model themselves on Oracle Cloud.

One record across two deployment paths

The change is a new OCI Generative AI codec: the part of Relay that reads and writes Oracle’s service formats. It covers all three formats Oracle identifies—GENERIC, COHERE and COHEREV2—and recognizes payloads automatically. Native function calls and streaming are supported, so a recorded run can include more than a single prompt and reply.

For a managed OCI Generative AI call, Relay uses that native codec. For a self-hosted model on Oracle Kubernetes Engine, it uses the endpoint’s OpenAI-compatible format. Oracle says the same runtime and governance policies work on both paths. A team moving between them would not have to give up its audit trail or rebuild those policies.

A record is not a replacement for a safety boundary

Oracle says Relay records each model and tool call, including requests, responses, tool arguments and token use, grouped by run. It also offers guardrails that can reject a request before a model receives it. In Oracle’s example, one blocked a request for a production password before the OCI service was called.

Its personal-information redaction works differently. Oracle says it masks what gets recorded for observability while the model still receives the original prompt. That can keep an email address out of the audit trail; it is not a promise that the model never sees it. Relay also calculates costs from token use and can export records to existing observability systems rather than requiring a new one.

Relay runs alongside an agent rather than acting as a network proxy. According to Oracle, the application keeps its own HTTP clients and OCI credentials, and prompts do not leave the customer’s tenancy through Relay. LangChain users can add it as middleware; teams without that framework can call Relay’s API directly.

When the answer outran the work

Oracle’s clearest demonstration was a GPU operations assistant running a self-hosted Nemotron model. An early version gave plausible GPU readings and cited three tool calls. Its trace showed only one real call, revealing that the assistant’s account of its work was false. The record exposed the mismatch; it did not, by itself, make the answer accurate.

That example gives the integration a practical test: can a team inspect what its agent actually did, rather than rely on its final explanation? Oracle says it also ran Relay against a managed endpoint serving an imported Nemotron model and captured a run spanning managed and self-hosted calls. Those are Oracle’s demonstrations, not a measured reliability rate for agents using Relay.

The version choice

The OCI codec arrived in NeMo Relay 0.8, but Oracle’s LangChain middleware example requires 0.9.0 or newer. Relay has Rust, Python and Node.js bindings, and exports records in ATIF, OpenTelemetry or OpenInference formats. The distinction matters for teams trying the example: having the codec alone is not enough to run the LangChain integration shown by Oracle.