What Is Voice Cloning and How Does It Work? (2026)
What is voice cloning? AI copies how a person sounds from a short recording. See how it works, the main types, how much audio tools need and the rules.
What is voice cloning? It is AI that makes a synthetic copy of one specific person's voice from a recording of them, then reads new text in that voice. Tools differ mainly in how little audio they need and whether they change the underlying model, which affects how fast a clone is ready.
How does AI voice cloning work?
Most voice cloning systems are text-to-speech models with two inputs: the words to say, and a reference recording of the target speaker. The model uses the recording to pick up the speaker's timbre, accent and pace, then applies those traits to the speech it generates. Research and vendors describe two broad approaches.
The first is conditioning on a short sample. Microsoft researchers' VALL-E paper, posted to arXiv in January 2023, treats speech synthesis as language modelling over discrete neural codec codes. The model was trained on 60K hours of English speech, and the authors state that it can "synthesize high-quality personalized speech with only a 3-second enrolled recording" of a speaker it has not seen before. Those are the authors' claims, not independent tests.
The second is fine-tuning on more audio. ElevenLabs' API concepts page explains that Instant Voice Cloning "uses few-shot adaptation": the model conditions on your audio at generation time and does not update its weights, so the clone is usable immediately. Professional Voice Cloning fine-tunes the model on your samples, modifying its parameters. ElevenLabs says that gives higher quality and a more consistent emotional range, but it takes longer.
Instant vs professional voice cloning
The two modes trade speed for fidelity. The table below summarises how ElevenLabs describes each one in its documentation as of October 2026.
| | Instant (few-shot) | Professional (fine-tuned) | |---|---|---| | How it works | Conditions on your audio at generation time; model weights unchanged | Trains the model on your samples; model parameters change | | Audio the vendor asks for | About 1 to 2 minutes of clear audio; more than about 3 minutes adds little | At least 30 minutes; closer to 2 to 3 hours for the most accurate result | | Readiness | Usable right after creation | Needs a training step, which ElevenLabs estimates in hours, not seconds | | Plan access | Starter and up on ElevenLabs' pricing table | Creator and up |
Instant cloning is the quick route, but it is less forgiving. ElevenLabs' instant cloning guide says the method may struggle with uncommon accents or unique voices, and suggests professional cloning in that case. Professional cloning asks for far more audio and more waiting, but it is the option the vendor points to when an instant clone falls short.
How much audio do different tools need?
There is no single answer. Published figures run from about 3 seconds in an open research model to several hours for a professional clone. The figures below come from vendor documentation checked on 17 October 2026. Vendors update these pages without notice, so check the current page before you record.
| Tool or model | Reference audio stated | |---|---| | ElevenLabs Instant Voice Cloning | About 1 to 2 minutes recommended; over about 3 minutes gives little gain | | ElevenLabs Professional Voice Cloning | 30 minutes bare minimum; closer to 2 to 3 hours recommended | | Cartesia instant clone | About 10 seconds to start; up to 60 seconds helps accent on Sonic 3.6 and newer | | Google Chirp 3 Instant Custom Voice | Up to 10 seconds, as close to 10 as possible | | Microsoft personal voice | 5 to 90 seconds of speech, plus a verbal consent statement | | OpenAI custom voices | 30 seconds or less, with 10 to 30 seconds recommended, plus a consent recording | | Fish Audio | At least 10 seconds per clip; a minute or two for better fidelity | | Coqui XTTS-v2 (open source) | A "quick 6-second audio clip" | | Qwen3-TTS Base (open source) | "3-second rapid voice clone" (the authors' claim) |
Length is only part of the picture. ElevenLabs says audio quality is the most important factor for instant cloning, and that what matters is total run time rather than the number of clips. A short, clean sample usually beats a long, noisy one.
Voice cloning vs text to speech vs voice conversion
These three terms overlap, so it helps to separate them:
- Text to speech turns written words into speech in a voice that already exists, such as a prebuilt or designed voice. For the basics, read what text to speech is.
- Voice cloning makes the new voice a copy of a real speaker's voice.
- Voice conversion changes one speaker's voice into another's while keeping the original speech. YourTTS, a model described in a December 2021 arXiv paper, was built on the VITS architecture to handle both zero-shot multi-speaker speech and zero-shot voice conversion.
What is voice cloning used for?
Several platforms have pointed the technology toward personal and consented uses. Apple's Personal Voice asks you to read a series of sentences aloud. Speech is processed on the device and stored encrypted there, and the feature is meant for personal, non-commercial use. Microsoft's personal voice works from a 5 to 90 second speech sample plus a verbal consent statement.
The same technology is also used by scammers. The FTC's consumer alert on harmful voice cloning, dated 8 April 2024, warns that if a call sounds like your boss or a family member in an emergency, "you're more likely to act." Scammers use cloned voices to make their requests believable.
Risks, consent and safeguards
Several rules and vendor policies shape how a clone can be used:
- Consent first. ElevenLabs asks you to confirm you have the right and consent to clone a voice before saving it. Microsoft requires customers to hold explicit written permission from the voice talent.
- Disclose AI audio. OpenAI's usage policies require clear disclosure to end users that a TTS voice is AI-generated.
- Label in the EU. Article 50 of the EU AI Act applies from 2 August 2026. Providers must mark synthetic audio, image, video and text outputs in a machine-readable format that makes them detectable as artificially generated.
- Robocalls in the US. The FCC's February 2024 ruling treats AI-generated voices as "artificial or prerecorded" voices under the TCPA, so callers need prior express consent unless an exemption applies.
Watermarking varies. Chatterbox, an open-source MIT-licensed model from Resemble AI, embeds a watermark in every file it generates. The project describes it as imperceptible and says it survives MP3 compression. The sources reviewed did not show watermarks on Google or OpenAI voices, so do not assume one is present.
For a fuller look at the rules by country, read voice cloning laws.
How OndaLabs handles custom voices
OndaLabs custom voices are cloned from a recording you have permission to use, or designed from a text description. A voice clone or design costs 1,500 credits. The plans list custom voice slots as follows:
| Plan | Monthly price (USD) | Custom voice slots | |---|---|---| | Creator | $22 | 3 | | Pro | $49 | 10 | | Business | $199 | 30 |
Commercial use is included on Starter and above. OndaLabs also offers 30 prebuilt voices that work in every supported language. For the full plan details, see the pricing page.
Try it
To create a custom voice from a recording you have permission to use, open the custom voices page. Before you record, read how to clone your voice for the consent and audio steps.
Frequently asked questions
What is the difference between voice cloning and text to speech?
Text to speech turns written words into audio using a voice that already exists, such as a prebuilt voice. Voice cloning builds a new voice that copies a specific real person. A clone starts from a recording of that person, so it needs consent and a clear source of audio.
How much audio do you need to clone a voice?
It depends on the tool. Instant methods from ElevenLabs and Cartesia ask for about one to two minutes or about 10 seconds, respectively, while professional clones need at least 30 minutes. Some open research models claim to work from about 3 seconds, though those are authors' claims rather than independent tests.
Can AI clone my voice from a 10 second clip?
Some tools say that a clip of about 10 seconds is enough to start. Google's Chirp 3 Instant Custom Voice accepts up to 10 seconds, and Cartesia says about 10 seconds is enough to begin. Short clips limit how much of your voice the model can learn, so expect the result to be less consistent than one built from more clean audio.
What is zero-shot voice cloning?
Zero-shot cloning builds a voice from a short reference sample without training the model on that speaker in advance. The model conditions on the sample at generation time. The YourTTS paper describes zero-shot multi-speaker speech and voice conversion, and open models such as Qwen3-TTS claim three-second cloning, though that figure is the authors' claim.
Is voice cloning legal and safe?
Cloning a voice with the speaker's consent is the lawful path, while unauthorised cloning can breach several laws, and the rules vary by country and state. Vendors add their own consent steps, and the EU and China now require labelling of AI-generated audio. This is general information, not legal advice, so check the current law for your situation.