New research delves into how pedestrians can more safely interact with AVs

Eye contact, hand gestures, nods––these are some of the unspoken signals drivers often exchange with pedestrians.

But what if there’s no driver?

That’s the question at the heart of ongoing research at Kean University in New Jersey, where researchers are using virtual reality technology to better understand how pedestrians behave when they cross the street––with the goal of identifying ways for autonomous vehicles to communicate with people and therefore improve safety.

“That’s how in the real world we operate. We don’t just rely on the traffic signals. We also communicate with the driver,” Lakshmi Subramanian, an assistant professor in Kean’s department of computer science and technology and the research’s leader, told Tech Brew. “So I think the research that is happening here in academia will definitely be very useful for manufacturers as well as policymakers.”

The problem: How pedestrians interact with AVs is becoming increasingly relevant as the number of driverless cars on the road increases. Robotaxi companies are scaling up their operations and expanding to new cities. Self-driving trucks are running delivery routes. And as the number of self-driving cars has grown, so too has the number of crashes involving AVs.

There have been high-profile incidents involving driverless vehicles and pedestrians that put the technology under scrutiny from government officials and members of the public, such as an October 2023 incident in which a Cruise robotaxi struck and dragged a pedestrian.

Still, AV sector leaders have made bold promises about the tech being safer than human drivers, and there are many efforts underway to use technology to tackle a traffic safety problem in the US that kills tens of thousands of people every year. Pedestrians and cyclists tend to make up a disproportionate number of road deaths.

“This is an important issue, especially with the pedestrian fatality rate,” Subramanian said. “We have to make sure pedestrians are safe, especially when self-driving cars are deployed on a large scale in the future.”

Possible solutions: Subramanian joined Kean in 2022. Prior to that, in her capacity as a doctoral student at the University of Iowa, she contributed to a growing body of research on AVs and pedestrian safety.

Her research there focused on children, and used an “immersive virtual environment” to assess two ways to improve safety: having the AV signal safe crossing with a green light when it was still driving or to signal when it came to a full stop. The published results indicated it was safer for the children when the vehicle signaled while at a complete stop.

For the research at Kean, Subramanian and her team of student researchers are using “CAVE-VR simulation technology,” which the university secured via the National Science Foundation Major Research Instrumentation Program, according to a blog post.

Participants use VR headsets to navigate various traffic scenarios. They’re asked to cross the road when they feel safe enough to do so.

“It gives us good implications of what happens when pedestrians try to interact with these self-driving cars, and how the visual communication helps to promote safety in road-crossing scenarios,” Subramanian explained.

One of the possible solutions under consideration is to install external displays on AVs that can communicate information to pedestrians. Eventually, this technology could potentially incorporate audio signals.

The research also aims to better understand how people behave when crossing multi-lane streets where AVs are driving, since much of the research to date has focused on single-lane roads.

“Real-world traffic is more complicated,” Subramanian said in a statement, “and so our research has implications for [the] scalability of autonomous vehicles.”