\n<\/p>\n
Researchers at Amazon have trained the largest ever text-to-speech model yet, which they claim exhibits \u201cemergent\u201d qualities improving its ability to speak even complex sentences naturally. The breakthrough could be what the technology needs to escape the uncanny valley.<\/p>\n
These models were always going to grow and improve, but the researchers specifically hoped to see the kind of leap in ability that we observed once language models got past a certain size. For reasons unknown to us, once LLMs grow past a certain point, they start being way more robust and versatile, able to perform tasks they weren\u2019t trained to.<\/p>\n
That is not to say they are gaining sentience or anything, just that past a certain point their performance on certain conversational AI tasks hockey sticks. The team at Amazon AGI \u2014 no secret what they\u2019re aiming at \u2014 thought the same might happen as text-to-speech models grew as well, and their research suggests this is in fact the case.<\/p>\n
The new model is called Big Adaptive Streamable TTS with Emergent abilities<\/a>, which they have contorted into the abbreviation BASE TTS. The largest version of the model uses 100,000 hours of public domain speech, 90% of which is in English, the remainder in German, Dutch and Spanish.<\/p>\n At 980 million parameters, BASE-large appears to be the biggest model in this category. They also trained 400M- and 150M-parameter models based on 10,000 and 1,000 hours of audio respectively, for comparison \u2014 the idea being, if one of these models shows emergent behaviors but another doesn\u2019t, you have a range for where those behaviors begin to emerge.<\/p>\n As it turns out, the medium-sized model showed the jump in capability the team was looking for, not necessarily in ordinary speech quality (it is reviewed better but only by a couple points) but in the set of emergent abilities they observed and measured. Here are examples of tricky text mentioned in the paper<\/a>:<\/p>\n \u201cThese sentences are designed to contain challenging tasks \u2013 parsing garden-path sentences, placing phrasal stress on long-winded compound nouns, producing emotional or whispered speech, or producing the correct phonemes for foreign words like \u201cqi\u201d or punctuations like \u201c@\u201d \u2013 none of which BASE TTS is explicitly trained to perform,\u201d the authors write.<\/p>\n Such features normally trip up text-to-speech engines, which will mispronounce, skip words, use odd intonation or make some other blunder. BASE TTS still had trouble, but it did far better than its contemporaries \u2014 models like Tortoise and VALL-E.<\/p>\n There are a bunch of examples of these difficult texts being spoken quite naturally by the new model at the site they made for it.<\/a> Of course these were chosen by the researchers, so they\u2019re necessarily cherry-picked, but it\u2019s impressive regardless. Here are a couple, if you don\u2019t feel like clicking through:<\/p>\n\n
\n