AI on the battlefield: the nightmare of poisoned data
AI is also at the heart of military technologies to improve their effectiveness on the battlefield. Its widespread use, however, exposes it to a very particular attack, that is, " poisoning the well" of the data that feeds its intelligence.
“I don't think our data is poisoned now,” US Deputy Defense Secretary Jennifer Swanson stressed at the Potomac Officers Club conference on Wednesday, “but when we fight a near-peer adversary, we're going to have to know exactly what the vectors are. of threat."
Every machine learning algorithm must be trained on data – lots and lots of data. The Pentagon is making a huge effort to collect, collate, curate and clean its data, so that analytical algorithms and fledgling AIs can make sense of it. In particular, the preparation team must eliminate all bad data points, before the algorithm can learn the wrong thing.
The battle of the dais
Commercial chatbots, from 2016's Microsoft Tay to 2023's ChatGPT, have shown how data quality can influence AI development and operation. In the case of AI used in the military sector, the data could be "Poisoned", that is, falsified and distorted, in a deliberate and guided manner by potential adversaries of a country's armed forces, and this technique is precisely defined as "data poisoning".
“Any commercial LLM [Large Language Model] that is out there that learns from the Internet is poisoned today,” Swanson said bluntly. “But I'm honestly more concerned about what you call, you know, 'normal' AI, because these are the algorithms that are actually going to be used by our soldiers to make decisions on the battlefield.”
In the case of the Pentagon it is not a question of training chatbots with data taken from the network. The Army should train it on a reliable and verified military data set within a safe and secure environment. Specifically, he recommended a system at DoD impact level 5 or 6, suitable for sensitive (5) or classified (6) data.
By the summer there should be the first sample of artificial intelligence IL-5 LLM, i.e. based on level 5 data. This can be useful for all types of back-office functions, summarizing reams of information to make more efficient bureaucratic processes. “But our primary concern is the algorithms that will inform decisions on the battlefield.”
Poisoning the AI data that decides activities on the battlefield can instead be a much deeper and more difficult problem to solve.
CJADC2, AI testing and how to defeat poisoned data
Getting the right military-specific training data is especially critical for the Pentagon, which aims to use AI to coordinate future combat operations across land, air, sea, space and cyberspace. The concept is called Combined Joint All-Domain Command and Control (CJADC2), and in February the Pentagon announced that a functioning “ minimum viability capability ” has already been fielded at select headquarters around the world.
Future versions will add targeting data and strike planning, connecting to existing service-wide AI battle command projects: the Air Force's ABMS , the Navy's Project Overmatch and the Army's Project Convergence .
Project Convergence, in turn, will use the technology developed by the newly formed Project Linchpin , which will be the point AI in guiding strategic decisions.
In other words, the Army is trying to apply to machine learning the “agile” feedback loop between development, cybersecurity and current operations (DevSecOps) used by leading software developers to quickly launch new technologies and continuously update them.
The problem is that, in reality, we don't know how to guide these processes and the companies that commercially manage the algorithms have no idea how they work. Normally we are used to seeing the functioning of programs as deterministic, as the fixed response to a certain situation. This does not happen with AI, where the answer is not always predetermined, as demonstrated by OpenAI applications.
There is also a further problem: each AI implementation causes a flow of data which in turn is integrated into the AI database and therefore defines its future processes. All this makes the problem of data poisoning and the purity of the information supplied to AI programs extremely important, essential, in the definition of strategic decisions and in their transmission.
The next few years will see the development of a new battlefield: that of data, in which each side will seek to corrupt and falsify the data base on which the decisions of the others are based.
Thanks to our Telegram channel you can stay updated on the publication of new Economic Scenarios articles.
The article AI on the battlefield: the nightmare of poisoned data comes from Economic Scenarios .
This is a machine translation of a post published on Scenari Economici at the URL https://scenarieconomici.it/ia-sul-campo-di-battaglia-lincubo-dei-dati-avvelenati/ on Mon, 22 Apr 2024 08:00:38 +0000.