LangChain lets teams search AI-agent records for behavior, not just keywords

Teams investigating an AI agent can now describe the behavior they want to find instead of manually opening records. LangChain launched AI Search in public beta on October 13, 2026, for LangSmith Cloud. The feature uses a model to identify matching sessions, including signs of user frustration or leaked personal information, within a fixed reading budget.

A question, then a model check

A trace is a record of an agent’s work: user inputs, model responses, reasoning and calls to tools. LangChain says teams mine these records for examples to test their agents or send to human reviewers. But a question about whether an agent actually completed a task does not fit neatly into a tag, metadata field or keyword search.

The company also argues that conventional vector search can miss the distinction. That method compresses text into numerical representations and finds nearby matches. It can locate a topic such as billing, but two records may look similar even if one agent did the work and another only claimed it did. AI Search instead uses a decision model to check the recorded content against the question.

Before that check, SmithDB, LangSmith’s trace database, converts incoming records into a trajectory: a session-level sequence of messages, tool calls and tool results. It removes repeated messages and puts supported recording formats into a common shape. That matters when an agent repeatedly sends its conversation history with each model call.

Existing filters narrow the candidate records first—for example, to sessions with errors or a particular tag. The model then checks the remaining trajectories in parallel. SmithDB prepares the next batch while the current one is being checked, and matches stream back without waiting for the entire search to finish.

What fits inside the search

Removing repetition does not make every session short. LangChain sampled 150 conversations from Open SWE, its open-source coding agent, and found a couple exceeding one million tokens—units of text processed by a model. In the median conversation, user messages and replies accounted for about 2,000 of 72,000 tokens. About 85% was tool output, mostly file reads and shell commands.

LangChain’s implementation gives each trajectory an 8K-token budget. User messages and the agent’s replies get priority; remaining space goes to intermediate model messages, tool results and system messages. A message that only partly fits is trimmed, and subsequent messages are dropped. The retained material reaches the decision model in its original order, not its priority order.

That selection has a material limit. LangChain says questions about an agent ignoring a tool error need evidence from tool results, rather than just the conversation. It is exploring question-specific context selection and uncertainty signals that could trigger a second pass with more context.

Searching for product problems, not just failures

The intended uses extend beyond technical debugging. LangChain suggests searching for unsupported integration requests, competitor mentions, onboarding confusion or attempts to cancel a subscription. Previously, it says, behavior-based searches required either manual review or an evaluator configured ahead of time. AI Search lets teams ask those questions as they arise, using content already captured in their traces.

Interpretation remains difficult in subtle cases. LangChain highlights distinguishing a user who gave up from one who got what they needed and left. The company wants to test whether a model tuned on traces makes that distinction more reliably. Grouping similar matches and summarizing their common features are also future ambitions, not announced beta capabilities.

The public beta is available at no additional cost on LangSmith Cloud in the GCP US, EU and APAC regions. Organization administrators can enable it in settings; it then appears on the tracing page.