SageMaker Feature Store Adds 25-Record Batch Writes and a Record-Discovery API

Amazon SageMaker Feature Store now has answers to two stubborn operational problems: writing features one record at a time and finding records when their identifiers are missing. AWS has added BatchWriteRecord and ListRecords, but the tools trade those gaps for work that still belongs in application logic: selective retries, retention choices, and pagination that is not a consistent snapshot.

Feature Store is AWS’s managed repository for storing, sharing, and managing machine-learning features. It combines a low-latency online store for real-time inference with an offline store for historical retention and training data. Until now, PutRecord wrote one feature record to one feature group per request, creating an N-by-M calling pattern when pipelines send many records to several groups.

Writing at scale

BatchWriteRecord accepts as many as 25 entries in one request, including entries for multiple feature groups. Each entry can target the online store, the offline store, or both. The service keeps PutRecord’s EventTime ordering: a newer record becomes the online version, while an older one cannot overwrite it; for feature groups with offline storage, the older record is retained as historical data there.

The important qualification is that this is a partial-success API, not a transaction. A failure for one record or feature group does not undo successful writes. Failed records return error details, and unprocessed entries can be retried, so clients must identify and resubmit only those entries. BatchWriteRecord also lets users set time-to-live, or TTL, at the record, request, or feature-group level, in that order of precedence.

Finding the records that remain

ListRecords addresses a different failure mode. The previous write, read, and delete APIs required callers to already know a record identifier. That was especially restrictive in the In-Memory tier: it has no corresponding offline store by default, so losing identifiers left teams unable to discover or delete those records. Standard-tier users could query an offline store through Amazon Athena, but that depended on offline-store configuration and was not real-time.

The new API paginates identifiers for active, non-deleted, non-expired records in a feature group. It works with both the Standard tier, backed by DynamoDB, and the Redis-backed In-Memory tier. It returns identifiers rather than feature values, so users must follow up with GetRecord or BatchGetRecord when they need the full data for a selected record.

The cleanup loop still needs safeguards

• Results have no guaranteed order, so callers cannot treat a page sequence as ordered inventory.
• Records written or deleted during pagination can produce duplicates or gaps, preventing ListRecords from serving as a consistent point-in-time inventory.
• NextToken values are opaque and bound to a specific feature group and account; callers must pass them through rather than construct or reuse them elsewhere.

Two gaps closed, two operating models retained

The release changes the basic control surface for feature pipelines: teams can consolidate small writes and inspect the identifiers that exist in either online-storage tier. It does not turn ingestion into an atomic multi-record operation or enumeration into a stable snapshot. For production users, the value will depend on whether retry logic, TTL policy, and cleanup jobs are designed around those boundaries.