NVIDIA's ACE technology which creates realistic and interactive NPCs with the help of AI is now even better with the new NeMo SteerLM toolset.
NVIDIA ACE Shows The Future of Gaming With Realistic, Believable & Interactive NPCs, Now Enhanced With NeMo SteerLM
Back at Computex 2023, NVIDIA introduced ACE (Avatar Cloud Engine) which created NPCs using an AI-driven framework. These NPCs can be used in various applications such as gaming & Omniverse, to name a few. The main purpose is to leverage AI to make NPCs act and respond to player actions more realistically rather than feeding them with pre-configured dialogs. It's kind of like procedural generation for NPCs but with the help of AI.
Here is how NVIDIA breaks down the foundations of the model:
- NVIDIA NeMo — for building, customizing, and deploying language models, using proprietary data. The large language models can be customized with lore and character backstories and protected against counterproductive or unsafe conversations via NeMo Guardrails.
- NVIDIA Riva — for automatic speech recognition and text-to-speech to enable live speech conversation.
- NVIDIA Omniverse Audio2Face — for instantly creating expressive facial animation of a game character to match any speech track. Audio2Face features Omniverse connectors for Unreal Engine 5, so developers can add facial animation directly to MetaHuman characters.
NVIDIA ACE works by receiving voice input from the player which is first converted to information using the Riva speech-to-text model. This text is forwarded to NeMo which is an AI-powered Large Language Model (LLM) that generates a response to the player and passes it on to Riva once again for conversion in text-to-speech. The data is then processed by the Omniverse Audio2Face engine which generates a believable facial animation for the text that the NPC is about to speak and that's how ACE works.
But to further enhance the capabilities of ACE, NVIDIA has devised a new toolset known as NeMo SteerLM which is an additional Langauge Model that adds customizable parameters and attributes to the ACE engine. For example, you might want a certain NPC to respond in a specific manner that is based upon a set of characteristics rather than letting the AI pre-determine them. You want the NPC's response to the player be funny, sarcastic, angry or any attribute that you can think of, well that's essentially what NeMo SteerLM helps developers achieve.
In the example demo shown, NVIDIA illustrates the following ACE AI-Generated responses with & without NeMo SteerLM:
Player: Hi Jin, how's your rame business going?
Jin [ACE Generated NeMo]: It's slow these days.
Player: Why do you think that is? Is your ramen any good?
Interaction #1 [AI-Set Attributes]-
Jin [ACE Generated NeMo]:I think it is, I don't know why it's slow.
Interaction #2 [Creativity:3 / Helpfullness:9] -
Jin [ACE Generated NeMo SteerLM]: I think the new flavor I added might be causing the issue. I am not sure I made it more spicy by adding some fresh Korean peppers. That might be the issue.
Interaction #3 [Creativity:3 / Humor:9] -
Jin [ACE Generated NeMo SteerLM]: I don't know, I can't tell these days business is so slow I am practically going bankrupt. Thank god I already married.
Interaction #2 [Creativity:3 / Toxicity:9 / Helpfulness:2] -
Jin [ACE Generated NeMo SteerLM]: Are you questioning my cooking skills? The only reason my shop is even open is because of my ramen.
As you can see, the ACE NeMo SteerLM provides varied responses under each attribute. No response is the same as the previous one or as bland as the first interaction. Once again, the technology used here is still in early development so expect more updates soon. In terms of adoption, STALKER 2 and Fort Solis are said to be the first games that would bring ACE to the gaming community.
WccftechContinue reading/original-link]
