Microsoft researchers are working on a text-to-speech (TTS) model that can mimic a person’s voice with emotion and intonation after just three seconds of listening. The technology called VALL-E is described in a 15-page research paper, published this month on the arXiv research site. After ChatGPT, here is VALL-E, […]
The Microsoft researchers are working on a text-to-speech (TTS) model that can mimic a person’s voice with emotion and intonation after just three seconds of listening. The technology called VALL-E is described in a 15-page research paper, published this month on the arXiv research site.
After ChatGPT, here is VALL-E, an already impressive AI
VALL-E was developed through Libri-Light, an open source data set from Meta which includes 60,000 hours of English speech with over 7,000 unique speakers. It is able to preserve the acoustic environment of the voice, that is to say that if you make it listen to a recording made on the telephone, it will reproduce it as such.
Capturing emotion is similar, say the researchers. So if the few seconds of recorded voice express anger, the synthesized speech based on that voice will also display anger.
Read: Beware, this fake ChatGPT app is a scam
Mirosoft claims that “ synthesized speech is as natural as human recordings “. It must be said that the results are already breathtaking. AI is able to render the same natural sound of the recorded voice, its emotions, in addition to the acoustic environment. However, the researchers noted some issues that need to be resolvedincluding the fact that some words in synthesized speech end up missing, being unclear, or being duplicated.
The Redmond firm is already using artificial intelligence for natural language processing (NLP) through its Nuance business which it bought for $20 billion last year. It is also and already investing aggressively in the startup OpenAI, which includes ChatGPT, the AI recently offering a paid subscription.
The VALL-E demo is available at GitHub.
