ElevenLabs introduces customer memory and follow-up automation for AI agents

ElevenLabs is extending its customer-service agents beyond the current conversation: remembering returning customers and handling work before and after calls. At its New York summit on November 11, the company announced upcoming Memory and Routines features for ElevenAgents, previewed a conversation-management engine, and introduced partner and Marketplace options to help businesses deploy agents.

ElevenAgents is ElevenLabs’ platform for configuring and deploying voice and chat agents across sales, support, recruitment and operations. The announced changes cover distinct jobs: carrying context between conversations, executing connected tasks, and choosing how an agent responds while someone is speaking.

A customer record built from conversation

Memory is designed to recognize returning authenticated customers and recall details from earlier exchanges. ElevenLabs says the customer relationship management system, or CRM, will remain the primary customer record. Memory instead captures information that may not fit an existing field, such as preferences, constraints or an upcoming occasion.

After an interaction, the agent will distill the conversation into a structured record and update it when the customer returns. The company’s examples include remembering that someone prefers Spanish and can only speak after 5 p.m. Those details would shape a later interaction rather than needing to be collected again.

ElevenLabs also described a subscriber mentioning a planned move in March. On an April call, the agent could ask about transferring service and retire the old address. That is the intended mechanism: preserve a detail, use it when relevant, then revise the record as circumstances change.

Routines addresses a different problem: the work surrounding a conversation. ElevenLabs says this automation layer will connect to existing tools and run multi-step tasks before, during and after an exchange. Tasks can include emailing a report, writing to a CRM or starting a call from within ElevenAgents.

Its proposed uses include pre-screening a mortgage applicant and calling back within minutes, or reminding patients about a visit and offering a canceled appointment to someone on a waiting list. These are company-described workflows, not measured results from deployments of the new feature.

Choosing when to speak—and which model answers

The previewed Interaction Engine manages the rhythm of a conversation. It handles turn-taking and interruptions, determining when an agent should begin speaking and when it should stop. It can also select different large language models—the systems generating responses—for different parts of the same exchange.

ElevenLabs described using a cheaper model for initial triage, then bringing in a more capable frontier model for complex tasks. The design aims to keep token use—the units models process—in check without assigning every step to the most capable model.

Within a few years, conversational agents will become critical infrastructure for every enterprise, as essential as cloud or payments

Mati Staniszewski, ElevenLabs CEO and co-founder

Deployment help now, purchasing and customization later

Alongside those product plans, ElevenLabs launched a Partner Program and introduced a Marketplace. It says its partner network now includes more than 550 partners across 44 countries. The two channels serve different deployment needs:

• Implementation support: Partners can integrate ElevenAgents into existing business systems or work alongside ElevenLabs’ deployment engineers. A newly announced alliance pairs ElevenAgents with McKinsey’s enterprise transformation work and its QuantumBlack AI unit.
• Partner-built applications: The curated Marketplace covers industries including healthcare, financial services, insurance, education and retail. Businesses can browse applications and contact their builders; purchasing through existing ElevenLabs commitments is scheduled to begin in 2027.

Company-specific model customization has a separate timetable. In the coming months, ElevenLabs says businesses will be able to fine-tune certain open-source language models and ElevenLabs voice models using their own deployment data. Fine-tuning means further training a model for a particular business context, rather than only supplying instructions.

Businesses will choose the measures they want to improve, such as customer satisfaction or reliability, and work with ElevenLabs researchers to refine the models. The proposed feedback loop uses new interactions to guide further optimization, making live deployment data an input to later model training.