Together AI Secures $240 Million IBM Cloud Buildout for Open-Source Inference

Together AI is lining up a dedicated, large-scale IBM Cloud cluster to serve open-source models to enterprise customers. IBM and Together AI have put $240 million behind that planned buildout, seeking infrastructure for a service Together AI says is handling roughly 400 trillion tokens a month.

The companies signed a multi-year agreement to deploy the inference cluster on IBM Cloud. IBM says it will be its first dedicated, large-scale inference cluster built with Nvidia HGX B300 systems, and the deployment is expected to become available in the first quarter of 2027. That makes the announcement a committed infrastructure project rather than a live service today.

The stack is designed for serving models at scale

The cluster will pair Nvidia HGX B300 systems with Nvidia Spectrum-X Ethernet networking. IBM Cloud supplies the cloud environment, while Together AI plans to run its inference software on the combined system. The design brings the compute hardware, network and serving software into one deployment for Together AI’s enterprise open-source model service.

Together AI’s stated objective is to improve inference performance and token economics while providing open-source model inference to enterprise customers. It selected IBM and Nvidia based on their product roadmaps and their ability to supply GPU capacity at the scale required for expanding workloads. The deployment is also intended to support real-time AI services.

Nvidia’s performance figure is not a result from this cluster

Nvidia says the underlying infrastructure is designed to deliver up to 30 times more AI factory output than prior generations. That is a platform-level design claim, not a measured outcome from the announced IBM Cloud deployment. Together AI’s proposed gains in service performance and token economics are likewise goals for a system that has not yet reached its expected availability date.

A deeper IBM-Nvidia infrastructure relationship

For IBM, the agreement expands a broader relationship with Nvidia across AI infrastructure and software. Together AI says its AI Native Cloud spans inference, training, fine-tuning and agentic workflows, but this project is specifically aimed at inference: running open-source models for enterprise customers rather than adding a general-purpose platform announcement.

The next milestone is delivery

The immediate result is a dedicated infrastructure plan for Together AI’s enterprise inference service. The meaningful next test comes with the expected Q1 2027 availability, when the companies can show whether the B300-based system delivers the operating improvements they have promised.