LangChain Adds Custom Apps to LangSmith So Teams Can Tailor Agent Reviews

Teams that review AI agents often need different views of the same data. LangChain has launched LangSmith Custom Apps so they can build and share those views inside their existing workspace, rather than run a separate interface for every recurring review.

The work behind a custom view

LangSmith already has standard screens for common checks, including dashboards for latency, cost and errors. It also has a comparison view that shows where experiment results got worse. Those defaults do not always match how a team decides whether its own agent is ready to use.

Some teams have built their own frontends around LangSmith data using its APIs. That gives them a tailored screen, but it also leaves them maintaining the software, hosting it and handling access. LangChain says Custom Apps lets teams publish a screen in LangSmith and avoid managing its hosting, authentication and permissions separately.

From an idea to a workspace app

There are two ways to make one. A user can describe the data and desired display in LangSmith Chat, which turns the request into a working app. A team that wants to build in code can start with LangChain’s templates and use the LangSmith API, with a coding agent if it chooses.

Once published, the app runs in the LangSmith workspace and can be shared with teammates. It stays connected to LangSmith data, so a team can reuse the same view for a recurring review instead of rebuilding a static chart or exporting data each time. That changes where the interface lives, not the underlying need to decide which data and judgments belong on it.

What teams might put on screen

The same agent data can support different review jobs. LangChain offers three examples: collecting human feedback, comparing changes before release and investigating an agent’s steps. A custom screen can put the relevant evidence in front of each reviewer rather than ask everyone to work through one general-purpose view.

• For feedback, an engineer might need the full trace, including tool calls and intermediate steps. A subject-matter reviewer might instead see the user’s request, the response, relevant context and a clear scoring guide.
• For experiments, a team could compare outputs from prompt versions or a model change, then focus on customer segments or types of failure that matter to its release decision.
• For traces, a support-agent review might center on tone and whether a request was resolved. A research-agent review might emphasize sources and tool use instead.

A limit on the lighter plan

Custom Apps are available now on LangSmith Plus and Enterprise. Both plans include the chat-based creation option, but Plus allows one Custom App per organization; Enterprise allows unlimited apps. A Plus team with several review processes will therefore have to choose which one gets a dedicated shared screen.

The next decision is less about creating a screen than designing a useful one. LangChain’s feedback example deliberately gives different reviewers different amounts of detail. Teams will have to decide which details help each person judge an agent—and which details a cleaner interface might hide when they matter.