Trustworthy AI: a framework and a compass

At first glance, the race for artificial intelligence may seem simply to be a race for performance: for any given task, the aim is to find the most powerful, efficient, energy-conscious and adaptable model, trained on the most up-to-date data, all in the name of efficiency and, ultimately, competitiveness. But the challenge goes far beyond technology and business performance. Developing competitive AI tools is essential, but it is not enough to inspire trust. Making them reliable, transparent, robust and responsible is. In critical sectors, AI will not gain ground solely on the strength of what it can do, but also on the confidence we can place in it. This is a long-term endeavour, bringing together responsibility and innovation.

At Thales, this conviction is not new. The Group has a long track record of designing critical systems across a wide range of sectors, including aerospace, cybersecurity and digital identity, defence and civil security — systems that must remain reliable over time and whose decisions must be fully traceable. As early as 2019, Thales introduced its TRUE AI approach (Transparent, Reliable, Understandable and Ethical AI) to anticipate the changes artificial intelligence would bring, in line with the company’s values. From 2020, the creation of the confiance.ai programme alongside major industrial players including Airbus, Naval Group and Renault followed the same direction, as did the introduction of Thales’s Digital Ethics Charter in 2022. “Trust has been built into everything we do with AI from the outset,” says Kalina Genet, Data and AI consultant at Thales. “Making sure AI systems are reliable and secure, while also considering their human impact and social acceptability… These are not new issues for us.”

This requirement for trust is a prerequisite for for its large-scale deployment, especially in critical systems. In such environments, integrating AI means addressing a wide range of constraints: system complexity, extremely demanding requirements in terms of reliability and availability, to embed AI in components with strict limitations on size, weight and power consumption; connectivity between subsystems that may be intermittent, low-bandwidth or even contested through jamming. 

This is a level of complexity that Thales already manages at industrial scale.

The four pillars of trust

The position paper “Trustworthy AI: The Fundamentals” identifies four inseparable pillars underpinning trust in AI systems. The first, validity, is about ensuring that a system properly fulfills the mission it was designed to perform - completely, without exceeding it - and under the intended conditions. The second, security, refers to its ability to withstand failures, cyberattacks and attempts at manipulation. Transparency and explainability make it possible to understand a result and trace the chain of events that led to it. Finally, accountability means complying with ethical, legal and regulatory frameworks, including data privacy rules and the protection of sensitive information, while identifying, controlling and mitigating risks, including societal and environmental ones. 

On the security front, “Thales brings together expertise in both artificial intelligence and cybersecurity: we develop technologies to protect AI systems against emerging threats, while also harnessing AI to strengthen cybersecurity,” adds David Sadek.

Putting trust into practice in defence systems

One area immediately comes to mind when discussing trustworthy AI: the military domain, which represents a significant part of Thales’ business and where AI-assisted decisions can have immediate consequences for a mission and, ultimately, for human lives. “Target detection using the TALIOS pod (the laser targeting and guidance pod fitted to Rafale fighter aircraft) can lead to critical decisions. We simply cannot afford to not trust the system,” explains Thomas Delavallade, AI Strategic Technical Authority at Thales Digital Services (TSN) and co-author of the position paper. 

Air traffic control: Singapore opts for NexGen

Air traffic control is one of the areas where the benefits of AI are particularly clear… and where the need for trust is equally evident. At any given moment, air traffic controllers must process vast amounts of information — flight paths, changing weather conditions and the state of the airspace — and make decisions within extremely tight timeframes. AI can help them identify potential conflicts and single out, from a constant flow of data, the information that most urgently requires their attention. In doing so, AI can ease their workload. But this assistance is only acceptable if its recommendations are reliable, understandable and traceable. Here too, an error can put human lives at risk.

This requirement lies at the heart of NexGen, Thales’ air traffic management system, recently selected by Singapore. Using artificial intelligence, it optimises aircraft arrival and departure flows, helping to reduce delays, fuel consumption and carbon emissions. Singapore’s decision underlines the point: in such a critical environment, AI can only be adopted if it provides strong guarantees of reliability and safety. In other words, guarantees of trust.

Automation bias: a risk that cannot be overlooked

Another sector in which trust is crucial is the maintenance of nuclear power plants. Thales supports EDF with AI-enhanced tools capable of processing large volumes of data to assist plant maintenance operators. Yet the very effectiveness of these systems can create a risk of its own, as Thomas Delavallade explains: 

“One of the challenges with increasingly capable AI agents is that they can create automation bias. The system works well, performance is good and, over time, people tend to check the machine’s recommendations less and less.”

If the tool eventually makes a mistake or encounters an unusual situation, will the human operator still be able to take back control? Thales has clearly identified this issue and factors it into the design of its systems. More broadly, it raises another fundamental question: how do we preserve human expertise in a world that will increasingly rely on assistance from artificial intelligence?

Building trust is a collective effort

Reliability, security, transparency, compliance, sustainability: trust in AI must be built at every level, from the design of the technology through to its use in operational environments. Far from working on these issues in isolation, Thales compares and develops its approaches alongside other industrial companies, research laboratories and institutions through joint working groups and programmes. The work initiated through confiance.ai is continuing at European level through the European Trustworthy AI Association and the CSIA programme (Trust in AI Systems). Their ambition is to prepare for the next wave of challenges associated with generative AI, cybersecurity and hybrid AI (AI combining for example machine learning with logical reasoning).

This cooperation also extends to shaping future standards. In aviation, Thales works within EUROCAE’s WG-114 alongside Airbus, Collins Aerospace, research laboratories, EASA and the FAA to define an appropriate framework for the use of AI in a sector where certification requirements are particularly stringent. The Group has also contributed to the work of CEN-CENELEC, which develops common technology standards at European level.

In defence, Thales contributes to the Communauté Thématique de Normalisation pour la Défense (CTND), under the aegis of AMIAD and the DGA, with the aim of accelerating the deployment of AI in French military systems. Finally, in the field of frugal AI, the Group is contributing to the drafting of the AFNOR/Ecolab Spec 2314 specification.

At a time when AI is driving profound technological change, no single organisation can build the foundations of trust alone. It is through a collective, evolving approach, grounded in shared values, that we will be able to deploy AI that is ethical and responsible, high-performing yet always ultimately at the service of people.