AI voice cloning: how a few seconds of sound are enough to copy a voice

The scammer opened one of Miva's videos and cut out a few seconds of her talking. That was all he took from her.

The rest came from other people. Long before, the makers of the program, called a voice model, trained it on thousands of hours of people talking, in thousands of different voices. From them it learned how speech works: how each word sounds, how a voice goes up at a question.

So Miva's few seconds do not teach the model to talk. It only listens to them and sums up her voice as a short list of numbers. The model chose these numbers itself, in training: nobody set them by hand. Together, they hold what makes her voice sound like her: how high or low it is, how warm or rough it sounds, a bit of her accent.

Then the scammer types any sentence. The model says it the way the voice in those numbers would say it, and Miva never said a word of it. This is voice cloning. A program that reads typed words out loud is called text-to-speech.

How few seconds? In 2023, Microsoft researchers showed a model that copied a voice from a recording of 3 seconds. In 2024, OpenAI said its own model needed one sample of 15 seconds. Because of the risk, it gave that model only to a small group of partners.

Some programs do not even need typing. The scammer talks, and Miva's voice comes out a moment later, with his words and his pauses. So he can cry, and answer your questions, live.

The next night, "Miva" calls again and asks for more money. This time a voice is not enough for you: you ask to see her, and she switches to video. There is her face, and her lips move with every word. Is it really her?