AWS details an internal gateway for AI agents to query company data

Amazon’s sort-center planners are the intended users of an internal system that lets AI agents query operational data through one controlled gateway. In a November 6 technical article, AWS detailed the implementation: teams can expose another dataset with a metadata file rather than custom integration code, while keeping permission enforcement in a shared server.

The team supporting Amazon’s North America Sort Center network built the system for planning work involving capacity, labor and package flow. Its data lake contains thousands of datasets. Previously, different teams built their own connections, embedding database-query logic, table knowledge and connection handling in each agent. A schema change meant patching agents individually, and bringing a new table online typically took several days.

Teach the gateway what a table means

The replacement uses Model Context Protocol, or MCP, to let compatible agents discover and call a shared set of tools. The server runs on Amazon Elastic Container Service with AWS Fargate. FastMCP, a Python framework, supplies the tool interface, while Amazon Bedrock powers the agent that analyzes questions, selects tools and turns results into answers.

Dataset onboarding starts with a JSON metadata file describing a table’s structure, business purpose, columns and sample values. Those examples help the agent distinguish real database values from everyday descriptions: a warehouse might be stored as a code rather than a spelled-out location. The file can also carry instructions for writing queries that work with a particular database engine.

A refresh pipeline converts that metadata into numerical representations for search and indexes it in Amazon Bedrock Knowledge Bases. Once indexed, the table becomes discoverable to connected agents without a server restart. The agent retrieves table knowledge when a question arrives, rather than relying on table structures hardcoded into each application.

Narrow the search before generating SQL

An optional business-context mapping limits which tables the agent searches. A labor-planning request, for example, can point toward workforce data instead of the entire catalog. AWS says this scoping reduces irrelevant matches, improves SQL accuracy and lowers latency. Without a specified context, the agent searches across all indexed tables.

AWS reports end-to-end interactions of 15–20 seconds in its testing. That covers the path from a question through table selection, query generation, validation and execution to the returned answer. It is a company-reported result for this implementation, rather than a performance guarantee for other deployments.

The tool set reaches beyond tables. Athena and Redshift handle SQL queries, while Bedrock Knowledge Bases searches documents, metadata and operational policies. AWS describes combining a database result with an internal document search in one response—for example, retrieving a return-rate figure alongside a team’s explanation of its causes.

Keep the database boundary outside the model

The architecture separates the internet-facing client from the MCP server, which runs in private subnets behind an internal load balancer. Amazon Cognito checks user identity at the client layer, and authenticated requests continue downstream. Conversation history lives in DynamoDB rather than individual Fargate tasks, allowing those tasks to be replaced or scaled without losing session state.

The Athena tools allow only SELECT queries, rejecting writes before they reach the data lake. Amazon Bedrock Guardrails applies to model interactions and is designed to filter harmful content, check grounding and help block prompt injection attempts. Those model-facing checks sit alongside the tool-level restriction, not in place of it.