Ethical Challenges of AI-Powered Autonomous Drones in Warfare
As technology increasingly influences modern warfare, a significant unresolved ethical challenge remains: should future AI-powered drones be granted the authority to kill? This question grows more urgent as governments and the defense industry recognize the expanding role of autonomous systems in combat.
Drones are currently deployed in large numbers in conflicts such as the war in Ukraine, and AI assists bombing missions in the Iran conflict. Some observers anticipate that future weapons will require greater operational autonomy, necessitating the integration of a moral framework within these systems.
In 2023, Mustafa Suleyman, CEO of Microsoft’s AI division and co-founder of UK-based DeepMind, addressed the issue of machines making moral decisions. He stated:
“AIs cannot be people – or moral beings.”
David Omand, former head of the UK intelligence agency GCHQ, told he believes AI can establish a “moral” configuration for unmanned weapons. Meanwhile, UK Armed Forces Minister Al Carns recently told the Financial Times there is a need to explore this further. These perspectives highlight the ethical and technological challenges of programming morality into autonomous weapons.
Limitations of AI in Moral Decision-Making
Zee Talat, a machine learning academic at the University of Edinburgh’s School of Informatics, argues that large language models—the foundation of modern generative AI systems like chatbots—are inherently incapable of moral decision-making.
AI systems are trained on extensive datasets to predict the most probable next word or sentence in a sequence. Talat explains this differs fundamentally from human moral and ethical reasoning, even if the AI has processed all known philosophical literature.
“If you have a machine that’s probabilistic by nature it will veer towards the most likely answer in a situation. Do we think that morality follows probabilistic notions?”
Talat recently co-authored a paper asserting that ethical evaluation is an “open-ended, debate-based, sociopolitical process” beyond AI’s capability to replicate.
Andrew Rogoyski of the Institute for People-Centred AI at the University of Surrey notes that AI systems have advanced significantly since ChatGPT’s 2022 debut, yet questions remain about their ability to replicate the nuance of moral judgment.
“Morality is deeply complex, contested, culturally shaped, and something most humans never fully resolve, even for themselves,”
he said.
“Perhaps the real question is whether we understand morality well enough to codify it. Until we do, we cannot expect machines to embody something we ourselves cannot clearly articulate.”
Challenges in Establishing a Universal Moral Code for Autonomous Weapons
The possibility of a universally recognized moral code for autonomous weapons is uncertain. Jessica Dorsey, assistant professor of international law at Utrecht University, highlights the difficulty of determining whose morality an autonomous drone would follow, especially as the United Nations continues to seek global consensus on lethal autonomous weapons systems.
Another critical issue is programming AI to distinguish combatants from civilians. Article 57 of the Geneva Conventions mandates that combatants must take “everything feasible to verify that the objectives to be attacked are neither civilians nor civilian objects … but are military objectives.”
Dorsey warns that if the legal framework is flawed, deploying AI-powered drones on a large scale risks replicating mistakes rapidly and extensively.
“War is filled with so many variables and it is a given that things will go wrong. And when that happens at AI-like speed, it is difficult to unravel,”
she said.
Necessity of Autonomy in Military Competition
Some experts argue that increased autonomy in drones, including embedded rules of engagement and moral considerations, will become essential if other nations continue advancing similar technologies rapidly.
Nicholas Wright, neuroscientist and author of Warhead, a book on the human brain and war, stated:
“For any military to compete effectively against other high-end militaries it is going to need a large amount of systems that will be required to take decisions on their own.”
Despite industry hype, AI-powered drones remain an emerging technology with limited battlefield deployment. Over 100 startups across the US and Europe are developing drones and drone software platforms, ranging from light, low-altitude surveillance drones to heavier armed craft. However, these companies hold sharply divergent views on the future decision-making capabilities of such systems.
Human Judgment and AI-Assisted Weapons
Olaf Hichwa, co-founder of US drone startup Neros, emphasizes that morality is inherently human and that AI-assisted weapons should extend human judgment rather than replace it.
“Morality is the province of human beings, which is why AI-assisted weapons systems need to be built in a way that extends the judgment and decision-making of the operator rather than wholesale replacing it,”
he said.
Hichwa envisions drones enhancing human pilots’ abilities rather than supplanting them.
“A lot of people are looking at autonomy somewhat incorrectly. They’re not looking at it through the eyes of the user. They’re saying, let’s make fully autonomous weapon systems,”
he explained.
“But the reality of warfare is that it’s still very much a human-on-human interaction.”
One application of AI in warfare could be reducing the cognitive load on operators of first-person view (FPV) drones, which are widely used in the Ukraine conflict. These small craft transmit battlefield views to human pilots via video links. In some scenarios, drones might autonomously navigate the “last mile” of a mission.
“One example of autonomy is … right now, pilots require a lot of very precise training to get the drone to travel along the right angle. Some autonomy could be the user or the human sets a general path and the drone follows that path.”
Autonomous Defense Systems and Target Identification
Jon Gruen, CEO of Fortem Technologies, which develops drone defense systems, offers a contrasting perspective. He notes that the US already employs autonomous defense systems to intercept incoming drones and missiles, with AI integrated throughout the process, including target identification and, at times, weapon deployment.
“When you detect something, you then need to identify it. That identification is done autonomously now for the most part.”
He added that the same technology can determine when to fire a weapon and guide it to its target.
Future Scenarios for Autonomous Targeting
Alex Fink, CEO of Swarmer, a US-Ukraine startup developing autonomous drone software, envisions a future where humans may not select individual targets but instead designate a “kill box”—an area where any target is authorized for engagement during a specified time.
“A human certifies this area, there are no friendlies, and I guarantee there will be no friendlies in the next 15 minutes. Any vehicle in this area is pre-approved for this period of time,”
Fink said.
“If a human is able to take that responsibility upon themselves, that they know there will be no friendlies, no bystanders in this area, then that type of mission is possible.”
However, while the technology may exist, a moral and legal consensus on such practices remains lacking.
