Limited-Risk or High-Risk? Where Your AI Voice Agent Sits Under the EU AI Act
by pbx.lu Editorial on September 28, 2026
AI voice agents now answer calls, book appointments and route callers on many business phone lines. Since 2 August 2026, the EU AI Act asks one thing of all of them: callers must know they are talking to a machine. Some voice agents carry much heavier duties, and a few uses are banned outright.
The difference does not come from the technology. It comes from what the agent is used for. This article shows where a typical AI voice agent sits in the AI Act's risk tiers, which uses push it into high-risk, what powers these agents behind your Cloud PBX, and which questions to ask your phone system vendor.
⚖️ The short answer. An AI voice agent that answers calls, gives information, takes messages, books appointments or routes calls is a transparency case, often called limited-risk. Its main duty is to tell callers they are speaking with AI, from the start of the call. It becomes high-risk only when it is used for a purpose listed in Annex III of the Act, such as screening job applicants, checking creditworthiness or triaging emergency calls. Those high-risk duties apply from 2 December 2027.
🧭 On this page
- The four risk tiers, applied to a phone line
- Why most AI voice agents are a transparency case
- When the same voice agent becomes high-risk
- What is already banned
- What powers the AI voice agents in business phone systems
- Who is responsible: you or your phone system vendor?
- What you should know about your own AI voice agent
- Questions to ask your Cloud PBX or UCaaS vendor
- Other rules that sit next to the AI Act
- Key dates for AI voice agents
- Frequently asked questions
The four risk tiers, applied to a phone line
The EU AI Act (Regulation (EU) 2024/1689) does not regulate "AI" as one thing. It sorts AI systems by the risk of their use. For business telephony, the four tiers look like this.
AI voice agent Software that answers or makes phone calls and holds a spoken conversation with the caller. It usually combines speech recognition, a large language model and a synthetic voice.
Limited-risk Not a term used in the Act itself. It is common shorthand for AI systems whose main duties are the transparency rules in Article 50.
Risk tier | Voice agent example | Applies from |
Prohibited | Scoring the emotions of your own call agents; pressure tactics aimed at vulnerable callers | 2 February 2025 |
High-risk | Phone screening of job applicants; credit pre-checks; emergency call triage | 2 December 2027 |
Limited-risk (transparency) | AI receptionist, FAQ agent, appointment booking, call routing | 2 August 2026 |
Minimal risk | Spam call filtering, background noise removal | No specific duties |
The high-risk date moved. The Digital Omnibus on AI (Regulation (EU) 2026/1744), in force since 27 July 2026, pushed the start of the Annex III high-risk rules from August 2026 to 2 December 2027. The transparency rules were not delayed.
Why most AI voice agents are a transparency case
Article 50(1) sets the core rule. The provider of an AI system that talks directly with people must design it so those people are told they are interacting with AI. The only way out is when this is obvious to a reasonably well-informed and observant person. The European Commission's Article 50 FAQ says this exception should be read narrowly. On the phone, where a synthetic voice can sound human, it is rarely obvious.
Provider The company that develops an AI system and places it on the market under its own name. For a voice agent built into your phone system, this is usually the phone system vendor.
Deployer The organisation that uses an AI system under its own authority. When your business switches on an AI receptionist, your business is the deployer.
The information must come at the latest at the first interaction. It must be clear and accessible. The Commission's final Article 50 guidelines, published in July 2026, add that an AI agent should state both that it is AI and on whose behalf it acts. A vague label such as "assistant" is not enough.
In practice, the business sets the greeting. A vendor can build the disclosure into the product, but it only works if the business keeps it clear and switched on. Two further transparency duties fall directly on the business as deployer. If the agent recognises callers' emotions from their voice, callers must be told (Article 50(3)). If the agent uses a cloned voice of a real person, that audio may count as a deepfake that must be disclosed (Article 50(4)).
🎙️ What a compliant opening line looks like
A line such as "You are speaking with the AI assistant of [Company]. You can ask for a colleague at any time." covers the core points: AI nature, whose behalf, and a route to a human. On multilingual lines, the disclosure should play in the language of the call, not only in one default language.
🔊 Does the synthetic voice itself need a hidden marker?
Article 50(2) asks providers of systems that generate synthetic audio to mark it in a machine-readable way, so it can be detected as AI-generated. Systems already on the market before 2 August 2026 have until 2 December 2026. How this works for live phone audio, which telephone networks compress, is not yet settled in guidance. It is the provider's duty, but ask your vendor how they handle it.
When the same voice agent becomes high-risk
Annex III of the Act lists the purposes that make an AI system high-risk. Several of them can happen on a phone line.
Voice agent task | Annex III point | Comment |
Interviewing or filtering job applicants by phone | 4(a) Employment | Covers analysing and evaluating candidates |
Rating call agents' performance or behaviour | 4(b) Employment | Applies to call analytics as well as agents |
Checking a caller's creditworthiness | 5(b) Essential services | Fraud detection is excluded |
Risk assessment or pricing for life or health insurance | 5(c) Essential services | Other insurance lines are not listed |
Checking eligibility for public benefits | 5(a) Essential services | Public authorities and those acting for them |
Classifying or prioritising emergency calls | 5(d) Essential services | Includes emergency healthcare triage |
Identifying a caller by voice among many people | 1(a) Biometrics | Confirming a claimed identity is excluded |
Recognising callers' emotions from their voice | 1(c) Biometrics | Also needs disclosure under Article 50(3) |
🎯 Purpose decides, not technology. The same software can be limited-risk at a dental practice and high-risk at a consumer lender. What counts is the intended purpose stated by the provider, and how the business actually uses the agent.
There is a filter in Article 6(3). An Annex III system is not high-risk if it only performs a narrow procedural task, a preparatory task, or does not materially influence the decision. For banks, lenders and insurers, which the financial services use case covers in more depth, this filter is decisive. An agent that collects a loan enquiry and books a meeting with an adviser may fall outside. An agent that scores callers or pre-rejects them does not. Any system that profiles individuals is always high-risk.
The Commission's draft guidelines on high-risk classification add a warning: tools marketed as "support" can still be high-risk if they drive outcomes in practice. A provider that relies on the filter must document its assessment and still register the system in the EU database.
🔐 Voice authentication versus voice identification
Many banks and service lines use voice biometrics to confirm that a caller is who they claim to be. This one-to-one check, called biometric verification, is excluded from the high-risk list. Picking out an unknown caller by comparing their voice against many stored voices is remote biometric identification, which is high-risk. GDPR rules on biometric data apply in both cases.
What is already banned
🚫 Banned since 2 February 2025. AI that infers the emotions of people at work, which includes your own call agents, unless it is for medical or safety reasons. AI that uses manipulative techniques, or exploits age, disability or a difficult social or economic situation, in a way that causes significant harm. A voice agent designed to push elderly callers into purchases would be a clear risk.
The line between callers and staff matters. Analysing a customer's emotions during a call is not banned, but it is high-risk from December 2027 and needs disclosure now. If the same tool also scores your agents' emotions, that part is banned. Sentiment scoring of a written transcript, such as the output of AI call transcription, is generally outside the definition of emotion recognition, because it does not use biometric data. Analysing tone of voice is inside it.
What powers the AI voice agents in business phone systems
An AI voice agent usually has three layers. Speech recognition turns the caller's words into text. A large language model decides what to say. A synthetic voice speaks the answer. Newer "speech-to-speech" models do all three in one step, which cuts delay. Most Cloud PBX and UCaaS vendors do not build every layer. They license them.
UCaaS Unified Communications as a Service: a cloud platform that combines calling, messaging and meetings in one subscription.
Layer | Common providers | Where based |
Language model | OpenAI, Google Gemini, Anthropic, xAI; Mistral AI | US; France |
Speech recognition | Deepgram, AssemblyAI, Microsoft Azure, Google; Speechmatics; Gladia | US; UK; France |
Synthetic voice | ElevenLabs, Cartesia, Microsoft Azure, Google; Mistral AI | US and UK; US; France |
Voice agent platform | Vapi, Retell AI; Parloa, Synthflow, Cognigy (part of NiCE); PolyAI | US; Germany; UK |
What vendors say publicly varies a lot. As of September 2026, RingCentral states that its AI Receptionist uses OpenAI models. 3CX lets administrators connect their own OpenAI or xAI account. Ringover built its AIRO Voice agent on the ElevenLabs agent platform. Dialpad uses its own in-house model. Zoom combines its own models with third-party models, including Anthropic and OpenAI. Telavox names Google Gemini and Sana AI in its AI privacy notice. Many European Cloud PBX vendors do not name their providers at all.
European options exist at every layer. Mistral AI in France offers both language and speech models. Kyutai, a Paris research lab, publishes open voice models. Gladia in Paris specialises in speech recognition. Parloa and Synthflow in Berlin build voice agent platforms, although both also use US models.
🌍 Does a US model make the voice agent non-compliant?
No. The AI Act applies to any AI system placed on the EU market, wherever the provider is based. Providers of general-purpose AI models have had their own duties since 2 August 2025. Where the model comes from matters more for GDPR: some US services store data in the EU but do not guarantee EU processing for real-time voice. Ask where audio is processed, not only where it is stored.
Who is responsible: you or your phone system vendor?
Both, for different things. The vendor, as provider, must build the disclosure into the product and mark synthetic audio. If it offers a high-risk use, it must also run risk management, write technical documentation, pass a conformity assessment, apply the CE marking and register the system before selling it.
Your business, as deployer, must keep the disclosure working and meet the Article 50(3) and 50(4) duties. For a high-risk use, the deployer must also follow the instructions for use, assign trained staff to human oversight, keep the logs for at least six months, inform workers before use at work, and tell people when a high-risk system helps make decisions about them. Public bodies, lenders and life or health insurers must also run a fundamental rights impact assessment.
Human oversight A named, trained person who can understand the AI system's output, question it, and stop or override it.
Article 25 contains a trap. A business becomes a provider, with all provider duties, if it puts its own name on a high-risk system, changes it substantially, or turns a general voice agent into a high-risk tool. Building your own candidate-screening flow on an open voice agent platform is a typical example.
🏷️ White-label and resale: who is the provider?
Many Cloud PBX services are resold under a partner's brand. For a limited-risk voice agent, the practical question is who designs the disclosure and who configures it. For a high-risk use, the company whose name is on the product is treated as the provider. Resellers and integrators should settle this in the contract before launch, not after a complaint.
What you should know about your own AI voice agent
Before asking your vendor anything, map your own use. These eight points are enough for most small and medium businesses.
- Purpose: what the agent does, on which lines, and what it must never do.
- Risk tier: whether any task matches an Annex III purpose, now or in future plans.
- Disclosure: the exact opening line, in every language you serve.
- Human route: how a caller reaches a person, and how fast.
- Emotion and voice features: whether anything analyses tone, emotion or voice identity, for callers or staff.
- Data: where recordings and transcripts are stored, for how long, and who can read them.
- Outbound calls: whether the agent ever calls people, and for what reason.
- People: who in your team configures and monitors the agent, and whether they understand how it works.
The last point links to Article 4 on AI literacy, in force since February 2025. The Omnibus softened it into a duty to support AI literacy, but staff who set up a voice agent should still know its limits.
Questions to ask your Cloud PBX or UCaaS vendor
These twelve questions separate a vendor that has done the work from one that has not. A good vendor answers them in writing. Add them to your Cloud PBX buying checklist when you score providers.
- What intended purpose do you state for the voice agent, and which uses do you exclude?
- Have you checked whether any feature is high-risk under Annex III? Can we see the result?
- How does the agent disclose that it is AI and whose behalf it acts on? Can we edit the wording, and can it be switched off?
- How do you handle machine-readable marking of the synthetic voice under Article 50(2)?
- Does any feature analyse emotions, tone of voice or voice identity, for callers or for our staff? Do you offer cloned voices?
- Which companies provide the language model, speech recognition and voice?
- Where is call audio processed, and where are recordings and transcripts stored?
- Are our calls or transcripts used to train any model? Can we opt out?
- Are all AI sub-processors listed in the data processing agreement?
- What logs do we get, how long are they kept, and can we export them?
- Can we test the agent, limit its topics and block promises it must never make?
- How will you inform us when you change the model, provider or features?
Other rules that sit next to the AI Act
The AI Act is not the only rule on the line. GDPR still governs recordings and transcripts, and Article 22 of GDPR limits decisions made only by automated means that significantly affect a person. A written call recording policy and a signed data processing agreement remain the basics.
Outbound AI calls need extra care. EU ePrivacy rules require prior consent for marketing calls made by automated calling systems without human intervention. An AI agent that runs marketing calls on its own is likely to be treated this way. In France, all telephone marketing to consumers needs prior consent since 11 August 2026. Germany already required prior express consent for marketing calls to consumers.
Fines under the AI Act reach up to 15 million euros or 3% of worldwide turnover for breaches of the transparency rules, and up to 35 million euros or 7% for banned practices. For small and medium businesses, the lower of the two amounts applies. In Luxembourg, draft law 8476 proposes the CNPD as the main supervisory authority, with the CSSF for the financial sector.
Key dates for AI voice agents
📅 Where we are now (September 2026). 2 February 2025: bans and AI literacy apply. 2 August 2025: duties for general-purpose AI model providers. 2 August 2026: transparency rules and fines apply. 2 December 2026: synthetic audio marking for systems already on the market. 2 December 2027: Annex III high-risk rules apply. 2 August 2028: high-risk rules for AI built into regulated products.
Frequently asked questions
Is an AI receptionist high-risk under the EU AI Act?
Not by default. An AI receptionist that answers, informs, books and routes calls is a transparency case under Article 50. It becomes high-risk only if it is used for an Annex III purpose, such as recruitment, credit checks or emergency triage.
Do I have to tell callers they are talking to an AI?
Yes. Since 2 August 2026, callers must be informed at the start of the first interaction, unless it is obvious. On a phone line it rarely is, so plan for a clear opening sentence.
Is a classic IVR menu covered by the AI Act?
A "press 1 for sales" menu with recorded prompts is not an AI system. A speech-based IVR that understands free speech and answers in its own words is an AI system that talks directly with callers, so the disclosure rule is likely to apply.
When do the high-risk rules start?
On 2 December 2027 for Annex III uses, after the Digital Omnibus on AI moved the date. Bans have applied since February 2025 and transparency rules since August 2026.
📚 Want the wider picture on AI in your phone system? Look up key terms in the regulatory and compliance glossary, see how call recording works in a Cloud PBX, explore the features guide or compare providers active in Luxembourg and the Greater Region.
bullet:hide Latest Insights synced block goes here
Manual step: insert the Latest Insights synced block (source 3320aa1f0a4881dfa218e655b13da274) above the booking CTA, then delete this toggle.
📅 Ready to explore Cloud PBX for your business?
A short conversation with a telephony consultant often saves weeks of evaluation. The session is free and takes about twenty minutes.