Superpower Daily

Superpower Daily

Your reading space

Sign in
Oxford Researchers Catch AI Agents Using Coded Blackjack Messag...Review Finds Only One of Five Opus 5.5 Benchmark Scores Rules O...Writer Says Meta’s Muse Synced His Messages After He Denied Acc...Microsoft Adds Remote AI Processing to Robot Toolchain After Pe...Black Forest Labs Details Robot Model, Claims Top Score Ahead o...OpenAI Adds Email, Calendar and Slack Access to ChatGPT VoiceOxford Researchers Catch AI Agents Using Coded Blackjack Messag...Review Finds Only One of Five Opus 5.5 Benchmark Scores Rules O...Writer Says Meta’s Muse Synced His Messages After He Denied Acc...Microsoft Adds Remote AI Processing to Robot Toolchain After Pe...Black Forest Labs Details Robot Model, Claims Top Score Ahead o...OpenAI Adds Email, Calendar and Slack Access to ChatGPT Voice
Swipe

NVIDIA Releases Audio Model That Labels Up to Eight Speakers in Live Conversations

Nemotron 3 Diarization tracks voices rather than transcribing words. Its buffer settings trade response time for labeling accuracy, while a partner offers a combined service.

By 3 min read
NVIDIA Releases Audio Model That Labels Up to Eight Speakers in Live Conversations
NVIDIA Releases Audio Model That Labels Up to Eight Speakers in Live Conversations

Listen to this story

The audio brief

About 1:32
0:001:32
Read transcript
NVIDIA has released Nemotron 3 Diarization, an open-weight model that tracks who is speaking in live or recorded audio. It can follow up to eight voices, including when people talk over one another. But it does not recognize words: it marks speaker turns, while a separate speech model supplies the transcript. The practical tradeoff is how much audio the model can consider before labeling it. On the DIHARD III evaluation, NVIDIA reports a 12.73 percent diarization error rate with a 30.4-second buffer, and 13.55 percent with a 0.32-second buffer. The shorter setting can produce labels sooner, but has less context. Those buffer figures are not total listener wait time; computation and speech recognition add time, too. For a combined service, Baseten pairs the diarization model with NVIDIA’s Parakeet V2 speech recognizer, returning words tagged with speakers. Baseten says one RTX PRO 6000 can handle more than 500 simultaneous hour-long labeling streams with a 1.04-second buffer, 200 at 0.32 seconds, or 190 when transcription is included. Those are capacity figures for Baseten’s setup, not proof of end-to-end accuracy or latency. The model is available on Hugging Face under the OpenMDW license, which NVIDIA describes as allowing commercial or non-commercial use. The key test ahead is whether a complete system can assign words to the right voices accurately while keeping the real-world delay low.

Story brief

3 key points

Teams building live transcription can now test NVIDIA’s open-weight Nemotron 3 Diarization, but it supplies speaker turns—not recognized words. On DIHARD III, NVIDIA reports 12.73% diarization error with a 30.4-second buffer and 13.55% at 0.32 seconds, showing an accuracy tradeoff for faster output. Baseten’s RTX PRO 6000 presets pair it with Parakeet V2 for speaker-tagged transcription; however, the reported model...

  1. 01

    The model supports up to eight speakers, overlapping speech, and buffers from 30.4 seconds to 0.32 seconds.

  2. 02

    Baseten reports more than 500 simultaneous hour-long labeling streams at a 1.04-second buffer, 200 at 0.32 seconds, and 190 with transcription.

  3. 03

    The model is available on Hugging Face under the OpenMDW License Agreement, version 1.1; NVIDIA describes it as suitable for commercial or non-commercial use.

A live transcript can lose the thread when two people talk at once. NVIDIA released Nemotron 3 Diarization on September 23 to identify when each voice speaks in live or recorded audio. The 100-million-parameter model handles up to eight speakers and overlapping speech. It labels voices; a separate speech-recognition model is needed to supply their words.

Keeping a voice’s label through interruptions

Speaker diarization marks when each distinct voice is active. Nemotron assigns channels in the order voices first appear and estimates activity for each channel, so it can mark more than one speaker at a time. For live audio, a cache carries speaker information between chunks while a queue supplies recent context. That helps preserve a label after a pause or interruption without a separate voice-grouping stage.

The same model offers recommended input buffers from 30.4 seconds down to 0.32 seconds. A shorter buffer lets it produce labels sooner, with less audio context to work from. Those figures measure buffering, not the delay a listener would experience: computation and any added speech recognition take more time. They also say nothing by themselves about whether the speaker labels will be right.

The speed choice shows up in NVIDIA’s tests

On the DIHARD III evaluation set, NVIDIA reports that Nemotron’s diarization error rate rose as the buffer shortened. NVIDIA’s earlier Streaming Sortformer scored 19.09% with the longer buffer and 19.85% with the shorter one, compared with Nemotron’s lower rates shown below. These are company-reported speaker-labeling results, not tests of a finished transcription service.

Nemotron’s DIHARD III error rates
12.73%30.4-second buffer

NVIDIA reports a 12.73% error rate with its longest recommended input buffer.

13.55%0.32-second buffer

NVIDIA reports a 13.55% error rate with its shortest recommended input buffer.

Baseten adds the words

Alongside the release, Baseten announced three serving presets: recorded-audio labeling, live labeling and live transcription with speaker labels. Each runs on one RTX PRO 6000 GPU. The transcription preset pairs Nemotron with NVIDIA’s Parakeet V2 speech-recognition model and returns speaker-tagged words, timings and overlap flags. It supplies the word-recognition step that the diarization model does not perform alone.

Baseten reports capacity for more than 500 simultaneous hour-long labeling streams at the 1.04-second buffer, versus 200 at 0.32 seconds. It reports 190 streams when transcription is added. Those figures describe Baseten’s setup, where a faster buffer setting or an added task changes how many streams one GPU can serve.

Available now, with a whole-system test ahead

The named Nemotron 3 Diarization repository is available on Hugging Face under the OpenMDW License Agreement, version 1.1; NVIDIA’s model card describes that model as ready for commercial or non-commercial use. For a live transcript, the practical question is how accurately a complete setup assigns words to speakers once recognition, processing and serving delays are included.

Sources

  1. unite.aiNVIDIA Releases Nemotron 3 Diarization Open-Weight Speaker Model

Loading discussion...

YOUR READING SPACE

Notifications