AWS adds usage-based software billing for live speech AI on SageMaker

AWS is giving speech-model providers a way to bill for usage while a live conversation is still running. Announced November 11, 2026, the new Amazon SageMaker AI metering feature lets AWS Marketplace sellers report consumption during two-way streaming sessions. Deepgram is the launch partner, with speech-to-text and text-to-speech billing.

The change addresses a mismatch between conventional request billing and a running audio session. For ordinary request-response inference—using a model to produce a result—providers can put a billable quantity in the response’s HTTP header. A live stream keeps accumulating usage, and its final total is unknown until it ends. That header cannot deliver periodic usage updates throughout the conversation.

SageMaker’s solution is a second WebSocket connection, a companion channel dedicated to metering rather than model inputs and outputs. Model servers opt in through a response header. SageMaker then opens the companion connection, and both channels carry the same request identifier so the provider can match usage records to the right session.

Providers send usage records as JSON, a structured text format, as consumption builds. AWS recommends incremental reporting rather than waiting for the end, reducing unreported usage if a session breaks unexpectedly. Each message contains one record; the channel allows one message per second, with a maximum size of 512 bytes.

The billing field is named inference.count, but that does not mean every request counts as one unit. The provider defines the consumption being measured, and the Marketplace offer maps that unit to a price. For a product charging by audio second, 12 minutes of metered audio would be reported as 720 units.

Keeping a connection open for 12 minutes does not, by itself, establish 12 minutes of billable audio. The provider’s counting rules decide that. Buyers also face a labeling wrinkle: Marketplace displays the unit price as “per request,” even when the underlying unit represents something else, such as a second of audio.

This changes the software bill, not the entire cost of running the model. SageMaker endpoint infrastructure charges remain separate and continue while the endpoint—the hosted service running the model—is provisioned. Unlike hourly software pricing, which can charge through quiet periods, the new software metering follows reported consumption.

Customers receive the software charges through their AWS bill rather than managing a separate billing relationship with the model provider. Marketplace private offers can support negotiated per-unit rates or fixed-price contracts.

The two-channel design creates a choice for providers. By default, inference continues if the metering connection cannot open or drops mid-session. Providers can instead require the data stream to stop when the metering channel becomes unavailable. That prioritizes metering continuity over keeping the live interaction running.

Neither setting guarantees that every submitted record becomes a charge. Malformed records can still be dropped while the metadata connection stays open. AWS’s billing validation instructions therefore call for checking the full path from model-server records to seller reports and buyer charges before release:

1. Start with a limited-visibility listing and an allowlisted test buyer, using an offer that exercises the intended unit and price mapping.
2. Send a known quantity of usage and reconcile the server’s reported units with Marketplace seller reports and the buyer’s charges.
3. Include interrupted-session tests, and allow reports time to populate rather than assuming a fixed reporting delay.