Testing for emergent behaviours in complex systems
Before long, perhaps even within a year, we will start to see Connected Autonomous Vehicles (CAVs) on our roads, mingling with regular cars. But that’s just the beginning. One day, there will be CAVs which are not only driverless but will have no driving controls. Before then, we had better know all there is to know about what can go wrong. Nicholas Moylan, one of XPI Simulation’s leading product specialists, looks at the challenges ahead.
Every time technology advances, another layer of automation is added to hide its complexity from humans. It’s the only way to make things usable. To make things safe as well as usable, however, extensive testing is required. But, with complex systems, we quickly reach the limit of what ordinary testing can achieve.
Let’s take a look at autonomous vehicles. The industry is young but already there is a multitude of makes and models. Sensors and components are sourced from various third parties. And the vehicles must be tested on many different types of road, from motorways to town centre traffic systems, in all weathers and all traffic conditions.
Simply testing each system and every component to make sure that they perform to specification isn’t nearly enough. Bringing so much new technology into a system of systems is pretty much guaranteed to create strong emergent behaviours with unpredictable, perhaps fatal, outcomes as different systems interact with each other and the environment.
At the moment, this isn’t a problem because autonomous vehicles require a human driver to supervise them, ready to take back control at any time if something unexpected happens. But the industry wants to move beyond that to an age where autonomous vehicles don’t feature a steering wheel or foot pedals; an age where everyone is a passenger.
According to RAND Corporation, an American non-profit, global policy think tank, in order to demonstrate the same level of safety as human drivers (not, perhaps, what one would call a ‘stretch target’) autonomous vehicles would have to be driven an appalling 8.8 billion miles. What’s more, the testing would have to be representative of the target operating environment. Billions of miles on racetracks will tell us nothing about safety on public roads.
So how do we manage that? How do we make testing faster, easier and less expensive, as well as comprehensive enough to reveal emergent behaviours?. A British organisation called OmniCAV is close to a solution.
OmniCAV, digital twins and simulation
OmniCAV is a UK government-funded cross-industry collaboration to create a real-world testing and certification environment, using sophisticated simulation technology and a digital twin.
The digital twin is an accurate representation of real 32 km road-loop in Oxfordshire, which features urban, peri-urban, and rural roads and a varied range of road layouts, using data from a variety of sources. Here’s how it works.
First, a 3D model of the route was built by Ordnance Survey, the UK national geospatial mapping agency. Dynamic elements such as traffic lights were added, using data provided by Oxfordshire County Council. And GPS signals were replicated for noise and signal degradation, enabling the whole to mirror the form and function of the real world.
To test a CAV along the virtual route, data provided from the vehicle manufacturer is used, together with as much of the actual system software and hardware as possible, for maximum fidelity. During testing, the digital twin is populated with other systems, including environmental and weather effects and other road users, which could interact with the CAV.
Into the future: the Age of Autonomy
OmniCAV has taken a huge leap towards making autonomous vehicles an everyday reality. And it will, I believe, play an important (although not exclusive) role in certifying autonomous vehicles as safe for use on public roads. But there is a bigger picture here.
Technology never gets simpler, only more complex. To ensure safety, we must test beyond the design specs to discover and counter emergent behaviours. That is particularly true for any ‘system of systems’ which has some degree of autonomy: cars, planes, boats, defence systems, robots, anything.
The good news is the technology being developed by OmniCAV’s is eminently transferable. One day it could be the go to solution for reliable, repeatable, cost-efficient testing, and for studying emergent behaviours, across a huge range of industries. Welcome to the Age of Autonomy.
You can download a full version of the paper from the Documents section below.
About Nicolas
Nicholas Moylan is a Product Specialist with XPI Simulation, leading the technical strategy and product line management of the company’s autonomous vehicle and driving simulation business. He has recently led research into intelligence and mission autonomy to improve survivability and situational awareness for Thales UK. He has more than 10 years of experience in airborne and maritime ISR (Intelligence, Surveillance and Reconnaissance) systems and has served in the British Army Reserves since 2015. XPI Simulation is a wholly owned subsidiary of Thales.