How to add an AI voice agent to your existing PBX via SIP

How to add an AI voice agent to your existing PBX via SIP

You can route selected calls to an AI voice agent while keeping your business number and phone system. Start with one test number. Once you have tested the agent, the transfer to a person and the voicemail fallback, you can use the route for after-hours calls.

How the AI agent fits into your call flow

Imagine a Luxembourg service company with a Mixvoip SIP trunk and an Asterisk PBX, the system that manages its business calls. The trunk connects the company’s phone system to the public telephone network. Asterisk decides where each call goes. During opening hours, it rings reception. After hours, it can send the call to ElevenLabs over a separate connection using SIP, the protocol the two systems use to set up calls. The company keeps its number with Mixvoip. Mixvoip SIP trunk service
Incoming call → Mixvoip SIP trunk → Asterisk PBX
  • Open: Ring reception.
  • Closed: Send the call over SIP to the AI agent.
    • Caller asks for a person: Transfer to the on-call team.
    • Agent is unreachable or does not pick up: Route to PBX voicemail.
Example routing for one business number. Test this flow on a spare number before applying it to the main line.

What you need before setting up the route

Check that your existing SIP trunk provider already routes a spare number to your PBX. You will also need an account on a voice agent platform that accepts SIP calls. Have someone with access to both systems available. Save the current after-hours rule and note where it sends callers.
Ask your PBX administrator to confirm the agent’s SIP address, authentication, firewall rules and audio codec, which is the format used to carry the voice. For ElevenLabs, agree how to secure the connection and choose an audio format supported by both ends, such as G.711 A-law if available on your PBX. Confirm how the platform will reach the on-call team. ElevenLabs SIP requirements
Give the agent a short list of approved answers, such as opening hours and service areas. Set the languages your customers need and your team can test and tell the agent what to do when it cannot answer. For a first pilot, use a simple callback or voicemail option when nobody is on duty.
If you already use Voxbi, its AI Receptionist can answer on a Voxbi extension after hours. Voxbi says it offers this feature in Luxembourg through Voxbi Managed. Voxbi AI Receptionist

How to connect Asterisk to the voice agent over SIP

  1. Assign the test number to the agent. In ElevenLabs, open Phone Numbers, choose Import number, then select From SIP Trunk. Enter the test number with its country code, configure the agreed SIP security settings and assign the agent. For a conference transfer back through Asterisk, also enter the PBX address and agreed authentication under the outbound SIP settings. Your existing SIP trunk provider still sends calls for the test number to your PBX. ElevenLabs setup and agent assignment
  2. Add an agent destination in Asterisk. Have your administrator configure a PJSIP endpoint for the AI platform. Asterisk can dial an explicit SIP address through that endpoint. The dial string takes the form PJSIP/ai-pilot/sip:<number>@<host>:<port>. Replace the placeholders with the actual values and use the agreed transport and security settings. The number in the SIP address must match the imported number, including the leading plus sign. Asterisk PJSIP dial syntax
  3. Apply the route to the spare number. Point its incoming rule to the new destination. Add a limit on how long the PBX waits for the agent to answer. If the connection fails or the agent does not answer within that time, send the call to the existing voicemail.

How to transfer callers to your team

Give the agent a dedicated handoff destination, such as a separate number for the on-call team. In ElevenLabs, enable the transfer tool and set it to transfer when a caller asks for a person or raises an issue the agent cannot handle. Configure the team’s destination to send busy or unanswered calls to voicemail or a callback queue. Sending callers back to the same after-hours number can put them in a loop.
For this pilot, choose a conference transfer to the team’s phone number. The agent platform calls that number and connects the two parties. If the call comes back through Asterisk, have the administrator restrict the incoming route from ElevenLabs to the approved destination. A direct transfer to a SIP address uses SIP REFER and requires the receiving SIP setup to accept it. ElevenLabs transfer methods

How to test the pilot and restore the old route

Answer: Call from an external mobile. Check that the agent picks up, both sides can hear each other and the caller’s number reaches the agent if your workflow needs it.
Languages: Try the same request in each language you plan to offer. Include a local place name and a question outside the agent’s approved answers.
Handoff: Ask for a person. Try again when the on-call destination is busy or unanswered. Check that the caller hears what to do next.
Failure: Make the agent destination unavailable on the test route. Check that the PBX sends the call to voicemail. Test a dropped call after answer separately, since the initial no-answer rule may never run in that case.
Switching the pilot off
Remove the agent route from the spare number. If you have already switched the main number over, restore its saved after-hours destination. Call each number you changed from an external phone to confirm the previous route works.

What to check before opening the route to customers

Identify the agent as AI in its opening greeting. Review whether you need recordings or transcripts, who can access them and how long you keep them. Confirm where live audio is processed and where records are stored. If the platform processes personal data on your behalf, put a data processing agreement in place. EU AI Act Article 50 · Luxembourg CNPD on GDPR processors
Ask both providers how many calls can run at once and how AI minutes and transferred calls are billed. Then switch over one after-hours route, make a test call to the main number and review how the first calls go before expanding the pilot. For other ways to connect an agent and what to budget, see the PBX.lu guide to AI voice agents.

TLDR: How to add an AI voice agent to your existing PBX via SIP

  • Route after-hours calls to an AI agent by connecting your PBX to a SIP-compatible platform like ElevenLabs.
  • Test answer, language, handoff, and failure scenarios on a spare number before switching your main line.
  • Check GDPR obligations, call capacity, and billing before opening the route to customers.
Updated on 2026-11-11