Autonomous Vehicle Testing: From Simulation to Real-World Roads

Before an autonomous vehicle can take the road, it needs to be tested for as many scenarios as possible.

• Simulation software and model-based development tools can create virtual "test-drive" environments.
• HIL, or "hardware-in-the-loop," testing connects a controller's real signals to a test system that simulates reality. A controller like an engine ECU, for example, can, in effect, be tricked into "thinking" it is in the finished car. With HIL, manufacturers are then able to put the vehicle components through a range of possible scenarios, without the costs of real-word drives.
• And, then, of course, there's taking the car out for an actual spin — both in on the open road and in closed courses.

In a live Tech Briefs presentation titled Standardization for Automated-Vehicle Testing and Simulation, a reader had the follow question for two industry experts.

"For AV testing, what are the respective role of simulation, closed course, and public road testing?"

Read the edited responses below from Jamie Smith, Director of Transportation Research and Design at the automated test manufacturer National Instruments, and Jeffrey Wishart, Managing Engineer at the consulting firm Exponent.

Jamie Smith, NI: The biggest shift has been embracing simulation. We've been using simulation in control design for years, and we've come to realize that the only way that we're going to achieve the challenges that the industry faces is by combining HIL testing, road testing, and simulation testing. That's been a big shift in our philosophy, as well as where we're making investments.

Another Reader Question for Jeff and Jamie

A reader asks, "Is the Public Ready for a Level-4 Autonomous Vehicle?"

Jeffrey Wishart, Exponent: I think that Jamie's percentages of mostly simulation, with some closed-course and some on-road, is about right, and where the industry seems to be heading.

I think that the fidelity of the simulation is a key component, and we need to understand how closely the simulation is to the real world, in order to understand its value. If you do a million scenarios and tests but the fidelity is low, what is the value of that? SAE has an On-Road Automated Driving (ORAD) task force that's working on that at the moment.

The closed course is really valuable, because you can validate your simulations, or a subset of your simulation tests. But it's expensive and it's very time consuming.

Public-road testing is quite useful because you will get scenarios that you maybe haven’t seen before or haven't thought of, but it's also takes a lot of time and you do a lot of driving that isn’t all that useful. It might seem very boring most of the time, with only a small percentage [of driving scenarios] that are interesting. And public-road testing is also very expensive to do.

They each have their role. Everyone will do all three, but it’s a matter of understanding the limitations and the advantages for each one.

What do you think about the vehicle testing options? Share your questions and comments below.