Salesforce Uses SageMaker Placement Controls to Meet 2-AZ Agentforce Standard, Keep 8x Savings

Salesforce has used new placement controls in SageMaker Inference Components to make its production Agentforce models support two Availability Zones, a requirement that an endpoint spanning multiple zones did not previously guarantee. The company says it retained the shared-GPU approach that cut its infrastructure costs eightfold.

AWS introduced SchedulingConfig in the CreateInferenceComponent API to control where copies of an inference component—a deployed model unit—are placed across instances and Availability Zones. Salesforce used the feature because SageMaker’s previous default placement handled each deployment operation independently. That could leave copies of a particular model concentrated in one zone even when the endpoint itself covered more than one.

That distinction creates two failure modes. If several copies share an instance, one instance failure can remove them together. If a model’s copies sit in only one Availability Zone, an outage in that zone can make that model unavailable. Salesforce requires every production model to support two zones, according to AWS’s account of the deployment.

The configuration has two layers. AvailabilityZoneBalance aims to distribute copies evenly across zones, while MaxImbalance sets the tolerated difference in copy counts between them. Within each zone, SPREAD puts copies on as many instances as possible for fault isolation; BINPACK puts them on fewer instances to favor utilization. Salesforce chose SPREAD, prioritizing isolation over denser packing.

AWS’s example uses four copies on four instances split between two zones. With SPREAD and the stated AZ-balance settings, SageMaker places two copies in each zone. For a lighter model with two copies, a MaxImbalance setting of zero targets one copy per zone. AWS recommends at least two component copies and two instances for a two-zone high-availability deployment.

The new controls do not turn zone balance into an unconditional guarantee. AWS says the available enforcement mode is permissive: if capacity prevents the desired distribution, SageMaker can place copies on available instances rather than fail the deployment. AWS recommends On-Demand Capacity Reservations in regions where capacity is constrained by Availability Zone, and says Salesforce pre-provisioned GPU capacity in each target zone.

AWS says scale-out places new copies to maintain the configured AZ distribution, while scale-in removes them symmetrically across zones. For longer-term consolidation after repeated scaling changes, the endpoint can use a CONSOLIDATION scale-in policy that periodically releases idle instances while honoring AZ-balance constraints. Endpoint and inference-component updates also preserve multi-AZ placement, according to AWS.

Teams can check whether the intended distribution persists through SageMaker AI Insights and CloudWatch. AWS says the services expose AZ skew, copy counts by zone, rebalancing events and insufficient-capacity errors. The operational test is therefore broader than setting CopyCount: teams must retain capacity, watch for drift and avoid reducing a high-availability model to one copy, which can exist in only one zone.