I'm excited for the future of video games after talking to Nvidia's AI NPC

Screenshot

I’m a seasoned, pessimistic tech journalist and it’s been a long time since I’ve been truly impressed by a tech demo, but after recently experiencing Covert Protocol, which uses Nvidia’s ACE technology, I’ve found myself very excited about the influence it could have on video games.

ACE essentially turns non-player characters into chatbots, so instead of talking to them with a limited script and set responses, you can have a free-ranging conversation with them. Covert Protocol is a proof-of-concept demo that shows off this new technology.

It was only previously shown to a small number of journalists at the Game Developers Conference earlier this year, but I recently had a chance to experience it in London. Here’s everything you need to know about this exciting new technology.

Best gaming laptop: powerful portables for PC games

What is ACE?

Nvidia’s Avatar Cloud Engine, or ACE for short, is used for the speech and animation of NPCs. It combines Nvidia NeMo for language and Nvidia RTX for ray-traced rendering to enable developers to create non-player characters capable of AI-powered natural language.

Essentially, it’ll turn NPCs into an AI chatbot like ChatGPT. The aim is to make conversations with game characters more realistic and engaging.

ACE is a combination of several technologies, from Nvidia Audio2Face tech which powers the facial animation, and Nvidia Riva which includes automatic speech recognition and text-to-speech (TTS).

Depending on the local GPU capabilities, it can run across the Cloud or PC to help ensure you receive the best experience.

How does ACE work in Covert Protocol?

Okay, so the best way to experience ACE at the moment is by playing Covert Protocol – a new technology demonstration created by Inworld AI in partnership with NVIDIA.

Covert Protocol is designed to push the boundary of what character interactions in games can be. In the demo, you’re a detective tasked with finding the hotel room number of someone you’re investigating – Martin Lane. You can talk to the hotel porter, receptionist, and a guest who works with Lane.

You start outside the hotel talking with the hotel porter – you can choose whether to soft-talk him or get straight to the point. Our group asked about his ambitions (which is becoming a bartender in the hotel), his home town (South Korea), as well as whether he had any annoying hotel guests (one is a particular diva). But when we got around to asking him about Martin Lane, he was tight-lipped, after all, it’s hotel policy to not share a guest’s room number.

All of this is very impressive, and it forces you to think of new ways to get information out of a character.

The character’s back story and personality are written by the developers in InWorld Studio, but how that comes out completely depends on what you ask them.

We move into the hotel lobby and talk to the hotel receptionist. Her sentences are very abrupt and she’s very professional – we’re not getting any information out of her.

Ultimately we set our sights on a hotel guest who works with Martin Lane. Initially, he doesn’t give us the time of day, but after we find a keynote pass and badge, he’s a bit more hospitable. After talking to him for a while you’ll realise he’s very focused on his keynote presentation. We used this to our advantage and said that Lane was feeling unwell and the conference was going to be cancelled. This prompted the guest to walk to the receptionist and ask to call Martin Lane in Room 103. Mission complete!

Now, did the demo run perfectly? While there isn’t a script, you can still tell you’re talking to an NPC. For example, sometimes information was shoe-horned in with unrealistic responses, there wasn’t any emotion in the NPC’s speech, and the delay between us talking and the NPC’s response often felt stilted (which the NVIDIA representative claimed was down to the demo area’s bad Wi-Fi).

Ultimate, though, I left Covert Protocol very impressed and wanting to play more.

What other video games could ACE potentially work in?

As soon as the Covert Protocol demo was over I started dreaming up other games ACE could work in.

Due to the crime-solving nature of the demo, my mind initially went to a game like L.A. Noire, driving around Los Angeles in the 40s interrogating suspects and chatting with witnesses.

Of course, it doesn’t have to appear in first-person games – you could also have an AI-powered race engineer in an F1 game, and you can ask things like how far the car behind you is and the condition of your tyres. AI could even power your whole squad of teammates in an FPS (for those times when you can’t get a group of friends together). The possibilities for ACE are really exciting.

Nvidia has said game publishers worldwide are evaluating how ACE can improve the gaming experience. Developers and platforms embracing ACE include Ubisoft, Deloitte, and Tencent (to name a few).

About

As Buying Guide Editor, Spencer is responsible for all e-commerce content on Stuff, overseeing buying guides as well as covering deals and new product launches. Spencer has been writing about consumer tech for over eight years. He has worked on some of the biggest publications in the UK, where he covered everything from the emergence of smartwatches to the arrival of self-driving cars. During this time, Spencer has become a seasoned traveller, racking up air miles while travelling around the world reviewing cars, attending product launches, and covering every trade show known to man, from Baselworld and Geneva Motor Show to CES and MWC. While tech remains one of his biggest passions, Spencer also enjoys getting hands-on with the latest luxury watches, trying out new grooming kit, and road-testing all kinds of vehicles, from electric scooters to supercars.