LiveKit Agents

LiveKit Agents is an open-source framework that lets you add programmable AI participants—such as voice, video, and text agents—to LiveKit rooms in real time. It supports building AI voice agents, multi-agent systems, and any programmatic participant with real-time audio, video, and data processing capabilities.

You can build LiveKit agents using Python or Node.js.

Agents register as "worker" processes with a LiveKit server (self-hosted or LiveKit Cloud). When dispatched, a worker boots a "job" subprocess that joins the LiveKit room as a full participant, enabling real-time interaction.

The LiveKit Agents framework supports multiple input/output modes including voice, video, and text. It features streaming pipelines for speech-to-text (STT), large language models (LLM), and text-to-speech (TTS) with interruption and turn detection handling. It also provides flexible multi-agent handoffs and workflows for complex task delegation, supports tools integration callable by any LLM, and offers wide AI provider integrations for STT, LLM, TTS, and more. Additional features include telephony and WebRTC client support for making or receiving calls from phones, semantic turn detection using transformers to avoid interruptions, and session and context data management for smooth conversation state retention.

Agents can handle different parts of a conversation and transfer control smoothly between one another, maintaining context and session metadata so conversations flow naturally. For example, a Sales Agent can transfer to a Technical Specialist or Pricing Agent based on user needs.

An AgentSession orchestrates user input, voice pipelines, LLM invocation, and output. Each session contains one or more Agent components managing AI logic and tools. You configure voice providers and interaction patterns globally or per agent.

Yes, you can deploy agents to LiveKit Cloud using the LiveKit CLI or run them on any custom environment. Agents register with the server and can be scaled via worker dispatch APIs.

No, they reduce repetitive workload and automate routine tasks but are designed to escalate or hand off complex conversations to humans while preserving context.

Agents can support any language available in your STT, LLM, and TTS stacks. Voice output leverages TTS providers and can be tuned for naturalness depending on provider capabilities.

Yes, LiveKit Agents support defining custom tools callable by the LLM. These tools can generate speech, call frontend methods, handle handoffs, external API calls, and data retrieval for enhanced interactivity.

LiveKit provides a Python Voice AI quickstart guide that walks you through building your first voice agent in under 10 minutes.

Yes, LiveKit Agents integrate with LiveKit’s telephony stack, enabling agents to receive or make phone calls seamlessly.