\n<\/p>\n
On Tuesday, Amazon debuted a new generative AI model, Nova Sonic, capable of natively processing voice and generating natural-sounding speech. Amazon claims that Sonic\u2019s performance is competitive with frontier voice models from OpenAI and Google on benchmarks measuring speed, speech recognition, and conversational quality.<\/p>\n
Nova Sonic is Amazon\u2019s answer to newer AI voice models such as the model powering ChatGPT\u2019s Voice Mode<\/a>, which feel more natural to speak with than the more rigid models from Amazon Alexa\u2019s early days. Recent technological breakthroughs have made legacy models and the digital assistants they underpin, such as Alexa and Apple\u2019s Siri, seem incredibly stilted by comparison.<\/p>\n Nova Sonic is available through Bedrock, Amazon\u2019s developer platform for building enterprise AI applications, via a new bi-directional streaming API. In a press release, Amazon called Nova Sonic \u201cthe most cost-efficient\u201d AI voice model on the market, and around 80% less expensive than OpenAI\u2019s GPT-4o.<\/p>\n Components of Nova Sonic are already powering Alexa+, Amazon\u2019s upgraded digital voice assistant<\/a>, according to Amazon SVP and Head Scientist of AGI Rohit Prasad.<\/p>\n In an interview, Prasad told TechCrunch that Nova Sonic builds on Amazon\u2019s expertise in \u201clarge orchestration systems,\u201d the technical scaffolding that makes up Alexa. Compared to rival AI voice models, Nova Sonic excels at routing user requests to different APIs, said Prasad. This capability helps Nova Sonic \u201cknow\u201d when it needs to fetch real-time information from the internet, parse a proprietary data source, or take action in an external application \u2014 and use the appropriate tool to do it.<\/p>\n During a two-way dialogue, Nova Sonic waits to speak \u201cat the appropriate time,\u201d taking into account a speaker\u2019s pauses and interruptions, says Amazon. It also generates a text transcript for the user\u2019s speech, which developers can use for various applications.<\/p>\n Nova Sonic is less prone to speech recognition errors than other AI voice models, according to Prasad, meaning the model is relatively good at understanding a user\u2019s intent even if they mumble, misspeak, or are in a noisy setting. On a benchmark measuring speech recognition across languages and dialects, Multilingual LibriSpeech, Amazon says Nova Sonic achieved a word error rate (WER) of just 4.2% when averaged across English, French, Italian, German, and Spanish. That means\u00a0that roughly four out of every 100 words from the model differed from a human transcription\u00a0in those languages. <\/p>\n On another benchmark measuring loud interactions with multiple participants, Augmented Multi Party Interaction, Amazon says Nova Sonic was 46.7% more accurate in terms of WER than OpenAI\u2019s GPT-4o-transcribe<\/a> model. Nova Sonic also has industry-leading speed, with an average perceived latency of 1.09 seconds, according to Amazon. That makes it faster than the GPT-4o model powering OpenAI\u2019s Realtime API, which responds in 1.18 seconds, per benchmarking by Artificial Analysis.<\/p>\n Prasad says Nova Sonic is a part of Amazon\u2019s broader strategy to build AGI (artificial general intelligence), which the company defines as \u201cAI systems that can do anything a human can do on a computer.\u201d Moving forward, Prasad says Amazon plans to release more AI models that can understand different modalities, including image, video, and voice, as well as \u201cother sensory data that are relevant if you bring things into the physical world.\u201d<\/p>\n Amazon\u2019s AGI division, which Prasad oversees, seems to be playing a larger role in the company\u2019s product strategy these days. Just last week, Amazon launched a preview of Nova Act<\/a>, a browser-using AI model that appears to be powering elements of Alexa+ and Amazon\u2019s Buy for Me feature<\/a>. Starting with Nova Sonic, Prasad says the company wants to offer more of its internal AI models for developers to build with.<\/p>\n<\/div>\n