What if you could just… speak… to video game characters? Ask your own questions, with your own voice, instead of picking from preset phrases? <\/p>\n<\/div>\n
Last May, Nvidia and its partner Convai showed off a fairly unconvincing canned demo of such a system<\/a> \u2014 but this January, I got to try a fully interactive version for myself at CES 2024. I walked away convinced we\u2019ll inevitably see something like this in future games.<\/p>\n<\/div>\n Let me be clear: the characters I spoke to were effectively generative AI chatbots. They didn\u2019t feel like real people \u2014 we\u2019ve got a ways to go before voices, facial expressions, and body language catch up to what\u2019s expected of a real-life interaction. There was sometimes a little robotic stutter and often a short delay before each NPC delivered their lines. Occasionally, they misinterpreted me.<\/p>\n<\/div>\n But many of today\u2019s biggest video games already set a pretty low bar for NPCs \u2014 and saddling up to the bar of a cyberpunk ramen shop to ask real questions with my real voice exceeds what I expect from the average denizen in Assassin\u2019s Creed <\/em>or The<\/em> Elder Scrolls, <\/em>for example. <\/em><\/p>\n<\/div>\n Here\u2019s my conversation with Jin, the \u201cproprietor\u201d of that stunningly rendered ramen shop, and Nova, a digital NPC who seems to be a friend. The conversation started just between Jin and Nova using a cheesy prompt that a rep typed in with a keyboard \u2014 but after that, I simply aimed my mouse at a character, held down the T key, spoke into a microphone, and they spoke back.<\/p>\n<\/div>\n Jin, after cheesy keyboard prompt \u201cThe Verge<\/em> is awesome\u201d<\/strong>: Nova, The Verge <\/em>is awesome, especially when they dive into the latest cybernetic tech and not just the usual corporate news. It\u2019s like peeling back the layers on the city itself.<\/p>\n<\/div>\n Nova<\/strong>: Oh absolutely. The Verge<\/em> knows how to uncover the real pulse of the city, not just the glossy surface. It\u2019s like they\u2019re tuned into the same frequency as us.<\/p>\n<\/div>\n Me, after walking up to the counter<\/strong>: What kind of ramen do you serve here anyway?<\/p>\n<\/div>\n Nova<\/strong>: Jin serves up the best of both worlds. Classic shoyu ramen for the traditionalists and a spicy miso ramen for those who like a kick. <\/p>\n<\/div>\n Me, to Jin<\/strong>: Hey, Jin, how do you cook your soup broth anyhow? <\/p>\n<\/div>\n Jin<\/strong>: Well, it\u2019s a bit of a secret recipe, but I can tell you it involves a whole lot of carefully selected ingredients and a touch of old-fashioned know-how. The result…<\/p>\n<\/div>\n Me, cutting Jin off<\/strong>: Sure, sure. You\u2019re quite a salesman. Can you tone it down just a little bit?<\/p>\n<\/div>\n Jin<\/strong>: Alright, alright. I\u2019ll dial back the sales pitch. The broth\u2019s the heart and soul of the ramen here. <\/p>\n<\/div>\n Me<\/strong>: Alright, serve me up a bowl. Tell me about the ingredients. <\/p>\n<\/div>\n Jin<\/strong>: Here\u2019s your drink. As for the ramen, it\u2019s all about the perfect balance of broth, noodles, and toppings. <\/p>\n<\/div>\n Me, to Nova<\/strong>: What\u2019s going on in this city, anyway?<\/p>\n<\/div>\n Nova<\/strong>: Oh, you know, same old mix of high-tech heists, corporate intrigue, and a sprinkle of underworld dealings. It\u2019s the city that never sleeps, after all. <\/p>\n<\/div>\n Me, to Jin<\/strong>: Tell me about your backstory. How\u2019d you get here?<\/p>\n<\/div>\n Jin<\/strong>: Not much to tell. I inherited this ramen shop from my parents and have been serving up the best noodles in Nox City ever since. It\u2019s a simple life, but it\u2019s mine.<\/p>\n<\/div>\n Me, to Nova<\/strong>: And you? Any friends I should know about? <\/p>\n<\/div>\n Nova<\/strong>: Hello, hello. Double greeting? Does that mean double the good luck, or am I mixing up my superstitions? <\/p>\n<\/div>\n Me<\/strong>: Who are your friends around here?<\/p>\n<\/div>\n Nova<\/strong>: <\/strong>Well, Jin is the heart of this place. As for the rest, it\u2019s more like acquaintances than friends. That\u2019s the city life for you.<\/p>\n<\/div>\n Me, to Jin<\/strong>: I\u2019m feeling a little amnesia. How do you know me again?<\/p>\n<\/div>\n Jin<\/strong>: I\u2019ve known you since you were a kid, Kai. Your parents used to bring you here and after they were gone\u2014[abruptly cuts off]<\/strong><\/p>\n<\/div>\n Me<\/strong>: What happened after they were gone? <\/p>\n<\/div>\n Jin<\/strong>: After my parents passed, I just kept the shop running. It\u2019s been a constant in this ever-changing city like a rock. <\/p>\n<\/div>\n Me<\/strong>: What are your feelings about being an AI?<\/p>\n<\/div>\n Jin<\/strong>: I think you\u2019ve got the wrong idea, kid, I\u2019m just a ramen shop owner, not an AI, but if you want to talk about the latest tech over a bowl of noodles, I\u2019m all ears. <\/p>\n<\/div>\n Me<\/strong>: Alright, what\u2019s your favorite gadget?<\/p>\n<\/div>\n Jin<\/strong>: My favorite gadget has to be this old radio behind the counter. It\u2019s a bit finicky, but it brings in the best tunes for the shop. <\/p>\n<\/div>\n Me, to Nova<\/strong>: How do you feel about being an AI?<\/p>\n<\/div>\n Nova<\/strong>: Well, I\u2019m not an AI, but I do know my way around the digital world as well as the city streets. It\u2019s all about finding the right balance, isn\u2019t it?<\/p>\n<\/div>\n Me, to Jin<\/strong>: What\u2019s to do around here?<\/p>\n<\/div>\n Jin<\/strong>: Well, if you\u2019re looking for a good time, there\u2019s the arcade down the street or the night market over on the next block. Just don\u2019t cause too much trouble or you\u2019ll have the local bounties on your tail. <\/p>\n<\/div>\n Me, to Nova<\/strong>: What should I be doing with my life?<\/p>\n<\/div>\n Nova<\/strong>: That\u2019s the m\u2014[slight stutter]<\/strong>\u2014illion-dollar question, isn\u2019t it? I\u2019d say follow that path that makes you feel most alive, whether that\u2019s chasing bounties or something else that ignites your spirit.<\/p>\n<\/div>\n Me, to Jin<\/strong>: What do you think?<\/p>\n<\/div>\n Jin<\/strong>: I know it\u2019s a fine line to walk, but I know you can handle yourself out there. Just remember to come back here for a real meal, not that synth-noodle stuff they serve at the market. <\/p>\n<\/div>\n Me, to Nova<\/strong>: What\u2019s your family like?<\/p>\n<\/div>\n Nova<\/strong>: It\u2019s just me and m\u2014[slight stutter]<\/strong>\u2014y cat, Whiskers. Holding down the fort. But Jin and you, Kai, are like the family I choose for myself. <\/p>\n<\/div>\n Me, to Jin<\/strong>: So, how do I know you again? <\/p>\n<\/div>\n Jin<\/strong>: I\u2019m the one who used to sneak you extra toppings on your ray-men [note awkward pronunciation change]<\/strong> when you were a kid. Figure you needed it more than the stray cats hanging around the back alley.<\/p>\n<\/div>\n Me, to Nova<\/strong>: What are you up to after this?<\/p>\n<\/div>\n Nova<\/strong>: I might head to the Lotus for a drink. Care to join? It\u2019s been a while since we had a night out.<\/p>\n<\/div>\n Me<\/strong>: Sure, any good movies playing?<\/p>\n<\/div>\n Nova<\/strong>: Let\u2019s see what\u2019s\u2014[slight pause]<\/strong>\u2014on at the Holosuite Cinemas. If not, we can always catch a classic at my place. I\u2019ve got the full archive.<\/p>\n<\/div>\n Me, to Jin<\/strong>: Wanna join us?<\/p>\n<\/div>\n Jin<\/strong>: I\u2019ll pass, thanks. This old cat statue on the counter brings me more luck than a night on the town could. <\/p>\n<\/div>\n Me<\/strong>: What are you going to be up to instead? <\/p>\n<\/div>\n Jin<\/strong>: I\u2019ll just hold down the fort here making sure the late-night stragglers get their noodle fix. It\u2019s not a glamorous job, but it\u2019s what I\u2019m good at.<\/p>\n<\/div>\n The dialogue is not particularly inspiring! I certainly don\u2019t prefer it to proper lines from good characters in my favorite games. You can also see how they only spoke in short bursts and how they sometimes lost the thread. But maybe it could be used to populate a whole world with lesser characters or combined with good, canonical dialogue written by a real human being, where generative AI just helps it go further. <\/p>\n<\/div>\n After the conversation, Convai\u2019s head of product, Nyla Worker, showed me some of the underpinnings in the company\u2019s browser-based dashboard. <\/p>\n<\/div>\n Jin and Nova, I could see, have their own \u201cCharacter Description,\u201d their own \u201cKnowledge Bank\u201d filled with text documents they can refer to, sliders for \u201cPersonality\u201d traits like extroversion, openness, and sensitivity, their own \u201cState of Mind,\u201d some \u201cGuardrails\u201d to hopefully prevent them from doing wacky AI things<\/a> (and filters for blocked words), and a memory bank as well. Worker says Convai is still working on long-term memory but that the conversation engine can already store a couple hours\u2019 worth. <\/p>\n<\/div>\n She also showed me how easy it was to inject new data. It took a single tap of a button to modify Jin and Nova\u2019s memory with an additional text file, and suddenly, they were able to tell me about Nvidia\u2019s new graphics cards<\/a>. Another button press, and these characters could speak in a new language.<\/p>\n<\/div>\n Since I didn\u2019t actually interact with the imaginary world that Jin and Nova theoretically live in, I can\u2019t fully tell what they\u2019re capable of. They seem pretty two-dimensional right now, with \u201cI am a proud, selfless ramen shop owner with a secret recipe\u201d being Jin\u2019s entire schtick. But I\u2019d love to see what a good writer could do with his backstory and motivations. I can now absolutely imagine games where NPCs remember what they\u2019ve seen and react to the game\u2019s script as it unfolds. The right bits could enter their memory bank at the right time, get filtered through their personality and desires, and make a game more immersive and interactive as a result. <\/p>\n<\/div>\n I just hope game developers use this to augment<\/em> their games, instead of putting voice actors and writers out of work. It\u2019s an extremely hot-button topic in the game industry right now. <\/p>\n<\/div>\n Just this month, SAG-AFTRA signed a deal with Replica Studios<\/a> that could let members license out digital replications of their voices. Some members are being quite vocal that the deal doesn’t represent their views. Last we heard, the labor union is still negotiating with game publishers for a new Interactive Media Agreement, and has listed<\/a> \u201cthe existential threat to member work posed by the unregulated use of AI\u201d as one of its primary concerns. <\/p>\n<\/div>\n<\/div>\n