Why it doesn’t make sense to ban autonomous weapons
Acceptable use
Just likefacial recognition, which is also under immense scrutiny with increased bans across the U.S., it is not the technology that is the problem — it is its acceptable use. We must define the circumstances where such systems can be used and where they cannot. For example, no modern-day police agency would ever get away with showing a victim a single suspect photograph and asking, “is this the person you saw?” It is similarly unacceptable to use facial recognition to blindly identify potential suspects (not to mentionthe bias of such technologiesacross different ethnicities, which goes well beyond AI training data limitations to the camera sensors themselves).
Another technology that suffered from early misuse is automated license plate readers (ALPRs). ALPRs were not only useful for identifying target vehicles of interest (e.g., expired registrations, suspended drivers, even arrest warrants) but the database of license plates and their geographic locations turned out to be quite useful for locating suspect vehicles following a crime. It was quickly determined that this practice was offside as it violated civil liberties and we now have formal policies in place for data retention and acceptable use.
Both of these AI innovations are examples of incredibly useful but controversial technologies that need to be balanced with well-thought-out acceptable use policies (AUPs) that respect issues ofexplainability, bias, privacy, and civil liberties.
Protection
Unfortunately, defining AUPs may soon be seen as the “easy” part as it only requires us to be more mindful to consider and formalize which circumstances are appropriate and which are not, although we need to move much faster in doing so. The most difficult consideration with the adoption of AI is ensuring that we are protected from the inherent dangers of such systems, which are not yet widely known today —thatAI is hackable.
AI is susceptible to adversarialdata poisoningand model evasion attacks that can be used to influence the behavior of automated decision-making systems. Such attacks cannot be prevented using traditional cybersecurity techniques because the inputs to the AI, both during model training and model deployment time, fall outside the organization’s cybersecurity perimeter. Further, there is a wide gap in the necessary skillsets that are required to protect these systems because cybersecurity anddeep learningare often mutually exclusive niche skills. Deep learning experts typically do not have an eye for how malicious actors think and cybersecurity experts typically do not have the deep knowledge about AI to understand the potential vulnerabilities.
As but one example, consider the task for training an Automated Target Recognition (ATR) system to identify tanks. The first step in this task is to curate thousands of training images to teach the AI what to look for. A malicious actor that understands how AI works canembed hidden imagesthat are nearly invisible to data scientists but completely flip to a new unseen image when resized to the input training dimension during model development. In this case, the above image of a tank can be poisoned to completely flip to a school bus during model training time. The resulting ATR is being trained to recognize both tanks and school buses as threat targets. Remember the difficulty of keeping humans in the loop?
Many will dismiss this example as either unlikely or even impossible but recall that neither the AI experts nor the cybersecurity experts understand the complete problem. Even if data supply chains are secure, breaches andinsider threatshappen daily, and this is just one example of literally an unknown number of possible attack vectors. If we’ve learned anything it’s that all systems are hackable given a motivated malicious actor with enough compute power — and AI was never created with security in mind.
It does not make sense to ban AI weapons systems as they are already here. We cannot police the development, and we cannot guarantee that humans remain in the loop as these are the realities of AI innovation. Instead, we must define when it is acceptable to use such technology and, further, that we take all measurable action to protect such technologies from adversarial attacks that are no doubt being developed by malicious and state actors.
This article was originally published byJames StewartonTechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original articlehere.
