AWS adds AI data preparation to SageMaker, pairing automated labels with human review
Teams can generate, label, check and reshape training data through code or a coding agent. Model agreement supplies confidence scores, while uncertain labels can go to reviewers.
SageMaker AI now offers a set of dataset-preparation components that teams can use separately or link into a workflow, from generating examples to checking and transforming data. The setup combines model-assisted labeling with expert review for low-confidence records, while keeping specifications, generated code and execution under customer control. That gives developers a way to inspect and adjust preparation steps rather than accept a finished dataset, but availability is currently limited to two US regions.
01
For classification, named entity recognition and extractive question answering, agreement among several models supplies row-level confidence scores; other labeling tasks do not automatically get this treatment.
02
Synthetic-data workflows can use customer documents and seed data, and generate pipelines that teams can inspect, modify and reuse.
03
Quality checks can flag duplicates, near-duplicates, inconsistent labels and answer leakage against business rules, built-in metrics or language-model assessments.
AWS made AI data preparation toolsgenerally available in Amazon SageMaker AI on November 17, giving model developers a connected way to generate, label, check and reshape datasets. The release pairs automation with human review: software can prepare examples at scale, while reviewers handle labels that fall below a confidence threshold chosen by the team.
Model agreement versus expert judgment
Auto-labeling can start with an unlabeled dataset, a small set of labeled examples, or no labels at all. Users describe the job in a short specification, which is checked against their examples before labeling the full dataset. Built-in tasks include classification, identifying named entities, extracting answers from text, ordinal ratings and summarization; custom tasks are also supported.
For tasks with a fixed set of possible answers, AWS uses several models as a jury. Their agreement becomes each row’s confidence score. This applies to classification, named entity recognition and extractive question answering—not automatically to every supported labeling task.
Rows below the user’s threshold enter a review queue in SageMaker Studio, using the open-source Label Studio Community Edition. Outside subject-matter experts can access the review app through a link in Studio. Their corrections feed back into auto-labeling, and teams can generate consensus metrics and labeling reports.
Creating examples is separate from checking them
Synthetic data generation creates training, evaluation and test examples from a specification. Teams can ground those examples in their own documents and seed data. AWS says the service produces a code-based generation pipeline that users can inspect, modify, reuse and scale, rather than handing them only a finished dataset.
Quality assessment is a separate building block. It checks datasets against customer-defined business rules, built-in metrics and assessments made by language models. AWS says it can find duplicates and near-duplicates, inconsistent labels and answer leakage. Data transformation then handles cleanup and formatting through plain-language requests that generate code, built-in operations or the customer’s own Python.
AWS’s example: checking a support assistant
Supply a product guide and require reference answers about factory resets to preserve its warning that saved settings will be erased.
Generate a small sample, inspect the examples and code, then assess individual records and send ambiguous answers to an expert.
Revise the specification or transform the dataset, generate additional examples where needed, and reassess before evaluating the assistant.
Agent assistance, customer-controlled execution
The agent route works in SageMaker Studio JupyterLab, Kiro or a local development environment with a coding assistant. Users can open and edit the specifications and code it produces. AWS recommends using the AI-ML skill to help create specifications even when developers intend to run the resulting workflow through the Python SDK.
AWS says execution uses a customer-controlled Identity and Access Management role, which governs permissions. Source data, generated datasets and labels stay in the customer’s own Amazon S3 storage buckets in the selected AWS region. The components can be used alone or chained together, so teams can add a quality check without adopting the entire preparation workflow.
Availability is currently limited to US East (N. Virginia) and US West (Oregon). AWS’s suggested starting point is one training or evaluation task: review its specification before generating a sample.
Sources
aws.amazon.comAnnouncing AI data preparation capabilities in Amazon SageMaker: Build high-quality datasets for model development workflows | Amazon Web Services
Reader comments
Newest comments first. Replies stay oldest first.