Frugal AI: the key to embedding artificial intelligence in combat optronics
© Thales © Julien Lutt
Deep learning techniques now meet a clear operational need, and frugal AI – artificial intelligence that achieves strong performance while using minimal resources – is a potential game-changer for armed forces in a battlespace of unprecedented complexity and uncertainty. Building on Thales's know-how in this strategic area, the first deep learning AI of its kind will soon equip the TALIOS optronic pod for combat aircraft, charting new territory in the race for real-time target detection and classification
Embedding AI in the systems deployed by combat aircraft and other high-tech platforms is anything but a passing fad
The use of deep learning techniques in the systems deployed by combat aircraft, drones and loitering munitions is one of the most striking developments of the decade.
It's probably one of the biggest lessons of the war in Ukraine so far. The constant and rising threat of drones, the battle for electromagnetic dominance and the return of near-peer combat have upended the theatre of operations as we knew it. In a modern battlespace that is denser, more fluid and more complex than ever, merely capturing aerial images is no longer enough – new-generation optronic sensors need to be able to analyse and interpret those images in real time. It's the only way to reduce the cognitive burden on the pilot, speed the decision-making process and stay ahead in the OODA loop (Observe, Orient, Decide, Act). In this context, artificial intelligence is the third eye, the bridge to heightened awareness and deeper insight, and the key to operational superiority.
Pioneering deep learning in airborne optronics
The use of deep learning techniques in the systems deployed by combat aircraft, drones and loitering munitions is one of the most striking developments of the decade. And with more than 800 specialists in AI and data analytics, Thales is blazing the trail, filing more patents in this category than any other European company in 2023.
The TALIOS targeting pod in service since 2021 is a fine illustration of Thales's expertise in deep learning technologies, with a brand new deep learning module known as ATD/R (Automatic Target Detection and Recognition) currently undergoing live flight tests. The module builds on almost 10 years of R&D and is due to enter operational service from 2027.
Deep learning has a lot to offer. "As combat aircraft crews approach cognitive saturation, these algorithms automatically detect and classify targets over vast areas of terrain to help pilots understand the operational situation in real time. We have already trained the AI to recognise a broad range of objects in many different strategic configurations," explains a dedicated deep learning engineer from Thales. The AI-augmented TALIOS pod will guarantee continuity in the intelligence pipeline as the aircrew will no longer need to return to base to analyse images with intelligence personnel on the ground.
The challenges of frugal AI
Like any revolutionary concept, embedded deep learning algorithms come with some serious engineering challenges. How can AI achieve the level of performance required, for example, with the limited computing power and heat dissipation available on board an aircraft – especially when targets of interest may be few and far between? The superior resource efficiency of frugal AI holds the answer.
First let's consider the thermal implications of embedded AI. A combat aircraft pod has a very low heat dissipation budget, so it's a real feat of engineering to get algorithms typically designed for powerful GPUs to run on circuit boards with strictly limited physical capabilities. Thales is overcoming this challenge in a number of ways, including model compression, the use of FPGAs, custom compute boards developed by the sovereign supply chain and "mixture of experts" architectures designed to get more capability without proportionally more compute.
The other important roadblock in developing frugal AI for deep learning applications in this context is the scarcity of operational images to learn from. The system needs to immediately recognise a vast range of different objects on the battlefield, so the armed forces must be able to use their own sources to continuously improve the performance of the AI.
Research and development in full swing
Guided by three principles – performance, sovereignty and resource efficiency –Thales's optronics teams are already fully mobilised on the pod of the future. Codenamed T&R (Targeting and Reconnaissance), the new product will combine the capabilities of the TALIOS and Reco-NG pods and incorporate a new set of IA functionalities.
"Research into embedded AI for combat optronics is in full swing at Thales, with a particular focus on frugal AI, hybrid AI and, ultimately, physics-informed AI," notes our deep learning expert. "Today, Thales offers young engineers the ideal environment to develop these new ideas and bring them to maturity."