Some auto companies want to put ChatGPT in your car

I was in the midst of mapping out a road trip from Chicago to New York when my planning hit a snag.

My would-be travel companion, a Microsoft-powered AI chatbot, could tell me all about the history of various famous parks, questions about checking or changing the oil, and restaurants near EV charging stations. But it seemed unaware of any hotels between the two major cities, no matter how the question was phrased.

The in-car AI in my CES demo of the technology was the product of a Dutch digital mapping company called TomTom, one of a handful of companies thinking about how large language models (LLMs) might serve as a far-ranging assistant and navigational system for vehicles in the future—both for the road itself and the array of computerized amenities that dashboards have accumulated in recent years.

Volkswagen also announced at this year’s CES that it would integrate ChatGPT into certain models through a partnership with auto software company Cerence. Amazon teamed with BMW to show off LLM-powered Alexa capabilities that serve as a sort of conversational car manual. And TomTom said before the show that it had collaborated with Microsoft to build out its own generative AI system for vehicles.

But this type of AI use is clearly very much in its infancy and isn’t always able to handle complex instructions. Whatever kinks there may be to iron out, Michael Harrell, TomTom’s SVP of engineering maps, told Tech Brew he thinks that generative AI will come to change the way that drivers interact with their cars.

“It is the breakthrough that we’ve been looking [for], that’s been necessary,” Harrell said. “And why I say that is that the hardest thing right now that people have with their car is how they interface with it—it’s still clunky, it’s not easy. It’s easier to use your phone.”

And as far as whether your car might “hallucinate” a roadside diner, Harrell said TomTom has worked to cut down on such fabrications by training the system on the company’s own mapping data.

“We’ve curated our data and trained it in a way that minimizes, significantly reduces the risk of hallucination,” Harrell said. “So it doesn’t eliminate it, but significantly reduces it.”

Dashboard director

That new interface is a response to the “massive transformation” in vehicle infotainment systems, Harrell said. As cars become even more computerized and aspects of driving become more autonomous, entertainment features and apps are growing more important. A survey from Salesforce released at CES found that consumers ranked “connected features” as almost as important as brand when buying a vehicle.

But existing voice platforms don’t always have the range and contextual awareness to manage these tasks, Cerence Chief Revenue Officer Christian Mentz said. He imagines Cerence’s AI system handling queries as far afield as brainstorming a birthday card or telling a kid about dinosaurs.

“The current system always has certain limitations,” Mentz told Tech Brew. “So the customer wants to get the task completed. And what you would like to avoid is dead ends. There’s nothing more frustrating when we want to get something done, and the task doesn’t get completed.”

In a demonstration of Cerence’s tech in a Volkswagen car on the showroom floor, Vania La Rocca, Cerence’s director of product operations, showed how statements like “I ran out of garlic” or “I’m really looking forward to eating some chicken tikka masala tonight” could yield a list of nearby grocery stores or Indian restaurants, respectively, though the system had some trouble generating a recipe based on a list of spoken ingredients.

The system consists of Volkswagen’s Cerence-powered in-car voice platform, IDA, augmented with ChatGPT for questions that the former is unable to answer. But the ChatGPT part isn’t able to, say, answer questions about locations along a particular route, La Rocca said.

LLMs lacking

Not everybody in the automotive software industry is sold on this tech as it exists now. Armin Prommersberger, SVP of automotive product management at Samsung-owned Harman, cited some challenges in-car LLMs face around hosting and connectivity.

“What we are after is context,” Prommersberger said. “If LLMs are an appropriate tool for that, I am totally for them—100%. [But] LLMs have certain boundary conditions that will prevent them [from] getting into mobile targets. Just think about latency. Just think about reception.”

Prommersberger said Harman is more focused at the moment on applying AI to improve its driver monitoring system (DMS)—a collection of sensors and software that seek to detect whether a driver’s full attention is on the road—and other features based around non-generative AI.

“Even though we can already do so much with [the DMS], it is by far not anywhere close to where we can see it can go,” Prommersberger said. “The refinement and further development of third-party orchestration AI, that is a key focus.”