INTEGRATING MACHINE LEARNING AT THE TACTICAL EDGE

The U.S. Army is planning to leverage Artificial Intelligence/Machine Learning (AI/ML) neural network-based algorithms to improve warfighter situational awareness in tactical vehicles. Integration of these complex algorithms into new and legacy architectures is a nuanced challenge to ensure accurate real-time performance and influence tactical decision-making.

By Jason Miller, U.S. Army Combat Capabilities Development Command C5ISR Center

As Artificial Intelligence/Machine Learning (AI/ML) continues to expand across all societal domains, it is understood that Al/ ML capabilities can be matured with the intention of supporting a multitude of functions, ranging from system diagnostics monitoring for preventative maintenance to autonomous driving to nominating potential threats in complex environments. Currently, the objective of incorporating these capabilities into military platforms is to shorten and improve human decision making by condensing the vast amount of data available within a vehicle to digestible chunks.

While the vision is simple, the practical implications of using AI/ML algorithms to fix all these problems is far from solved. Legacy military platforms have been in service for many years sometimes decades — and include components, sub-systems, and architectures that make straightforward and efficient processing upgrades very difficult. To effectively leverage these emerging AI/ML-based algorithms, the right data (including quality video, position, and time) needs to be available at the right place at the right time. Planning for this type of system is best considered early in the design phase to ensure that the architecture supports adequate data bandwidth and routing, power, and space allocation. An effective Aided Threat Detection and Recognition product, or AITDR, requires integration with the platform sensors and connection to the system controller, which would be expected to host data from other critical subsystems resident on the vehicle.

STANDARD POINT OF EVOLUTION

Standardized interfaces, which define both required and optional parameters to enable the AI/ML capabilities have yet to be codified, making each individual integration effort a unique challenge. For performant AiTDR algorithms, high resolution, low latency, full motion video (FMV) with metadata must be accessible to a capable processor, where more video feeds or higher neural network complexity necessitates larger processing and power demands. While it may be practical to place advanced processing at the edge, as close to the sensor as possible to minimize the data bandwidth requirements on the vehicle network, that placement will limit the ability of other sensor systems to leverage that processor, likely sacrificing the efficiency of central processing solutions. Tactical vehicles that include turrets have additional constraints such as slip-rings that will be constraining components when attempting to pass high bandwidth FMV between a turret and a vehicle hull. Emerging interface standards, such as C5ISR/ EW Modular Open Suite of Standards (CMOSS) and CMOSS Mounted Form Factor (CMFF) aim to assist in defining data availability and form factors, but challenges remain with high dynamic range FMV.

It is crucial to understand the desired end-state of a capability designed to offset or aid in human tasking. Decisions such as, “Who is the end consumer of this data?” “What information needs presented?” or “What accuracy is required?” will shape the design and implementation. Emerging programs can leverage AITDR capabilities for the purpose of autonomous search and track, where the algorithms are tightly coupled to the other platform subsystems and a system controller which choreographs how to scan, what to focus on, whether to enable other components, when and where to output data, and when to aggregate AITDR nominations with disparate data. Once interwoven with the platform control, high levels of automation will ultimately be possible. Legacy platforms with predefined architectures and hardware solutions would be expected to incorporate a less integrated AITDR capability, where a stand-alone processor may be simply added between a sensor video feed and the User Interface to nominate potential targets for real-time manual interrogation. When designed properly, either level of integration is expected to be a significant advantage for future mounted formations.

CENTER TARGETING GREATER SURVIVABILITY

The business of detecting threats is time critical and error-intolerant, as misses or incorrect nominations may impact warfighter survivability. Though there are many challenges and architectural decisions to consider, the U.S. Army Combat Capabilities Development Command’s (DEVCOM) Command, Control, Communications, Computers, Cyber, Intelligence, Surveillance, Reconnaissance (C5ISR) Center located at Aberdeen Proving Ground, MD and Ft. Belvoir, VA, has been maturing and integrating these capabilities into being key concepts demonstrators since 2018. Through iterative design and testing, complex architectures have been built to distribute data from Electro-Optic/Infrared (EO/IR) sensors, gimbals, lasers, position, navigation, and timing (PNT), tactical networks, Graphical User Interfaces (GUI), etc. Through a deep understanding of how these neutral networks function and what data is needed to perform the prescribed task, the C5ISR Center has established a template for Program Managers as well as industry partners to leverage when designing future system architectures or adapting legacy systems. Though the C5ISR Center has been successful in demonstrating a high level of algorithm maturity and system integration into numerous complex Army priority exercises, adherence to Army standards has yet to be enforced. Final integrations of complaint algorithms will require network security controls, conformance with alternate processing and architectures standards, and unknown access to reliable networked information. These complicating factors are tempered slightly because of the requirement that a human must remain in (or at least on) the loop before using lethal methods to engage a target. A reduced probability of detection in an AiTDR system can be offset by having a human make the final call on target/no target.

As it pertains to algorithm performance upgrades, these can be achieved through multiple methods. One method aims to improve the performance of individual neural network components through the addition of new training, which results in an updated “weights” file. Trading this file does not change the complexity of the network, making for a simple — and sometimes incremental — software update. Unfortunately, algorithms are likely to start plateauing in performance, even when additional training data is incorporated, necessitating inclusion of more complex algorithm components into the pipeline to yield more significant performance gains. Without careful up-front planning, inclusion of additional algorithm processes may quickly expand beyond the size, weight, and power budget allotted to the AiTDR processor. If fusion processing across multiple detector networks is required to achieve the desired performance, the solution may not be practically implementable if allocations had not been made for enhanced capabilities. Due to the uncertainties of how AI/ML products will perform over their deployed lifetime, significant platform processing margin and advanced-planned architectures
should be considered early in the design phase to ensure the ability to incorporate future enhancements. Updating AiTDR algorithms may be achievable, but when satisfying specific performance requirements, it will be critical to ensure that the system still functions as expected when changes are made, necessitating a robust verification plan. If the AiTDR algorithm is updated and incorporates new functionality, downstream processing may be impacted, requiring additional coding and qualification of system software and user interfaces in order to leverage the new functions. As AI/ML technologies are transitioned from Research and Development to full-scale deployment, Program Managers are going to be challenged with both the integration of new capabilities as well as performance verifications if it is expected that algorithms are updated at a high tempo.

TAMING WHAT MAKES CAPABILITY TICK

As with all software, test and evaluation of AI/ML systems can be difficult, often requiring specific tools, expertise, and data. An inherent challenge to testing these AiTR systems is to not introduce biases into the evaluation processes and data. Therefore, independent sequestered data evaluation is considered the gold standard for evaluating AITR algorithms and their expected performance in operationally relevant environments.

As part of C5ISR Center’s role as the Army’s AiTR experts, the center has stood up an evaluation lab that serves as the Army’s “honest broker” when evaluating both Army and industry developed algorithms.

Many questions remain with respect to tactically deployed AI/ML, such as:

• What is satisfactory performance and how is compliance of the integrated solution verified?
• What level of controls should be exposed to user to adjust algorithm performance on the fly?
• How often should algorithms be updated in vehicles and what is the limitation of acceptable changes once a compliant solution is deployed?
• Should platforms fighting in the same formation have the same AiTDR capabilities?
• Should AiTDR algorithms be hosted on dedicated Line Replaceable Units or integrated into multi-purpose vehicle video processing electronics?

Though plenty of uncertainties remain, the Army understands one thing completely: Al/ML-based capabilities are here to stay.