Autonomous Weapons Based on Artificial Intelligence and Human Control: Challenges and Solutions in International Humanitarian Law

Document Type : Research Paper

Authors

1 M.A in International Law, Payam Noor Law School, Tehran, Iran;

2 Ph.D. in Islamic Jurisprudence and Assistant Professor, Department of Jurisprudence and Law, Imam Khomeini and Islamic Revolution Research Institute, Tehran, Iran

10.22099/jls.2026.55440.5456

Abstract

Introduction
Following the emergence of artificial intelligence on the global technological stage, its application in military weaponry—which, unlike traditional programming models, can learn how to perform a task through machine learning (i.e., training and post-use feedback)—poses a fundamental challenge to the principle of human control within International Humanitarian Law (IHL). IHL seeks to regulate the conduct of parties to armed conflict and mitigate unnecessary suffering (while acknowledging the military necessity of weakening the adversary). Since the inherent capacity for continuous "learning" from the environment renders the behavior of a machine learning system and its processing of new inputs unpredictable and inscrutable, the very possibility of meaningful human control over such systems, characterized by unpredictable behavior and opaque decision-making processes, is called into serious doubt. An even more profound question arises: are machine learning systems inherently dependent on human control to ensure compliance with IHL in the first place?
Methods
This article first provides a technological overview of autonomous learning systems, explicating their capabilities for continuous learning, analysis, and adaptation based on experience, alongside their opaque decision-making processes and unpredictable outcomes. This analysis establishes how these characteristics complicate the exercise of meaningful human control. Subsequently, by examining a system's capacity to rapidly process vast datasets and employ them in making strategic and precise decisions, the article demonstrates the significant potential of such systems to enhance overall compliance with IHL.
Results and Discussion
Machine learning is now deployed across a broad spectrum of military activities, primarily by reducing the rate of erroneous actions and reactions in complex environments. Relying on deep learning and reinforcement learning, these systems have transcended human limitations in solving complex battlefield problems. Unencumbered by human cognitive constraints and biases, they prove far more efficient at generating innovative solutions. However, this very capability renders machine behavior, both pre- and post-event, unpredictable and incomprehensible. This opacity may impede adherence to IHL's precautionary requirements, including the duty of constant care. Regarding the role of human control in fulfilling this duty, three principal theories are debated: "meaningful human control," "the necessity of human intervention prior to attack decisions," and "the possibility of eliminating prior human intervention."
Under the third theory, given that the decision-making of learning systems in sensitive situations is founded upon big data and processes opaque to human understanding, direct human supervision or intervention is rendered practically meaningless. Crucially, however, even absent human comprehensibility and direct control, such systems might be capable of making targeting decisions that are more compliant with IHL than those made by human-controlled systems. This potential superiority may stem from their ability to process and analyze immensely larger volumes of data with precision and speed exceeding human faculties. Consequently, their use should not be prohibited solely on the grounds of precluding meaningful human supervision, unless conclusive evidence is presented establishing the absolute necessity of such oversight.
Conclusion
The deployment of machine learning-based autonomous systems in combat reveals the imperative to redefine "meaningful human control." The findings indicate that under certain conditions, these systems can operate in accordance with IHL principles without direct human supervision, while in others, human control remains essential to guarantee compliance. The conventional wisdom that "more human control is invariably better" requires reassessment. The focus must shift toward ensuring "acceptable predictability" in system performance, such that it aligns with IHL standards and remains evaluable. This framework clarifies the pathways for system design, testing, and certification, applying human oversight at the most consequential stages.
Ultimately, through responsible design and deployment, learning systems can harness technological capabilities to reduce civilian casualties while upholding core IHL principles, such as military necessity and the mitigation of unnecessary harm. The proposed approach establishes an equilibrium between technological capability and human responsibility, enabling the optimal utilization of novel technologies within the bounds of International Humanitarian Law.


Keywords

Main Subjects

References

References
af Jochnick, C., & Normand, R. (1994). The impact of technology on international humanitarian law. International Review of the Red Cross, 76(835), 1–30. https://doi.org/10.1017/S0020860400089648
Akerson, D. (2013). The illegality of offensive lethal autonomy. In D. Saxon (Ed.), International humanitarian law and the changing technology of war (pp. 65–87). Leiden: Brill Nijhoff.
Alaee, Saber & Hosseinzadeh, Javad (2022). Analysis of the Independence of Black Box Algorithms in Algorithmic Contracts and Its Legal Consequences. Journal of Legal Studies, 14(1), 251-278. DOI: 10.22099/jls.2022.40211.4340 [In Persian]
Article 36. (2013, April). Killer robots: UK government policy on fully autonomous weapons. London: Article 36. (Retrieved 10 Aban 1402) from: http://www.article36.org/wp-content/uploads/2013/04/Policy_Paper1.pdf
Bhuta, N., Beck, S., & Geiss, R. (2016). Present futures: Concluding reflections and open questions on autonomous weapons systems. In N. Bhuta, S. Beck, R. Geiss, H.-Y. Liu, & C. Kreß (Eds.), Autonomous weapons systems: Law, ethics, policy (pp. 347–375). Cambridge: Cambridge University Press.
Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer.
Boulanin, V., & Verbruggen, M. (2017). Mapping the development of autonomy in weapon systems. Stockholm: Stockholm International Peace Research Institute (SIPRI). https://doi.org/10.13140/RG.2.2.22719.41127
Cummings, M. L. (2006). Automation and accountability in decision support system interface design. Journal of Technology Studies, 32(1), 23–31. https://www.jstor.org/stable/43604352
Curtis E. LeMay Center for Doctrine Development and Education. (2019). Volume 1: Basic doctrine. Montgomery: Curtis E. LeMay Center. (Retrieved 10 Aban 1402) from: https://www.doctrine.af.mil/Portals/61/documents/AFDP_3-0/3-0-AFDP-BASIC-DOCTRINE.pdf
de Spiegeleire, S., Maas, M., & Sweijs, T. (2017). Artificial intelligence and the future of defense: Strategic implications for small- and medium-sized force providers. The Hague: Stockholm International Peace Research Institute (SIPRI).
Deeks, A., Lubell, N., & Murray, D. (2019). Machine learning, artificial intelligence, and the use of force by states. Journal of National Security Law & Policy, 10(1), 1–34. https://jnslp.com/2019/12/02/machine-learning-artificial-intelligence-and-the-use-of-force-by-states/
Esmaeili, Morteza (2024). The Interplay of Power and the Responsibility to Protect Doctrine; A Case Study of the 2023 Hamas-Israel Conflict. Journal of Legal Studies, 16(1), 43-74. [In Persian]
Gill, T., Fleck, D., Boothby, W., & Vanheusden, A. (2014). Autonomous weapons and international humanitarian law: Challenges and perspectives. Journal of Conflict and Security Law, 19(1), 15–40. https://doi.org/10.1093/jcsl/kru001
Goose, S. (2014). Autonomous weapons and international humanitarian law: A policy perspective. Geneva: United Nations Institute for Disarmament Research (UNIDIR).
Gretton, A. (2017). Legal and ethical issues in AI and autonomous weapons. London: Routledge.
Grut, C. (2013). The challenge of autonomous lethal robotics to international humanitarian law. Journal of Conflict and Security Law, 18(1), 5–23. https://doi.org/10.1093/jcsl/krs026
Hassabis, D., & Silver, D. (2017, October 18). AlphaGo Zero: Learning from scratch. DeepMind Blog. (Retrieved 10 Aban 1402) from: https://deepmind.com/blog/alphago-zero-learning-scratch/
Henckaerts, J.-M., & Doswald-Beck, L. (2005). Customary international humanitarian law: A contribution to the understanding and respect for the rule of law in armed conflict. International Review of the Red Cross, 87(857), 51–80. https://doi.org/10.1017/S1816383100181130
Host, P. (2016, November 5). Deep learning analytics develops DARPA deep machine learning prototype. Defense Daily. (Retrieved 10 Aban 1402) from: https://www.defensedaily.com/
Human Rights Watch. (2015). Mind the gap: The lack of accountability for killer robots. New York: Human Rights Watch. (Retrieved 10 Aban 1402) from: https://www.hrw.org/report/2015/04/09/mind-gap/lack-accountability-killer-robots
International Committee of the Red Cross (ICRC). (2014). Autonomous weapon systems: Technical, military, legal and humanitarian aspects. Geneva: ICRC. (Retrieved 10 Aban 1402) from: https://www.icrc.org/en/document/report-icrc-meeting-autonomous-weapon-systems-26-28-march-2014
Kania, E. B. (2017, June 8). China's quest for an AI revolution in warfare. The Strategy Bridge. (Retrieved 10 Aban 1402) from: https://thestrategybridge.org/the-bridge/2017/6/8/chinas-quest-for-an-ai-revolution-in-warfare
Keller, J. (2015, July 24). DARPA TRACE program using advanced algorithms. Military Aerospace. (Retrieved 10 Aban 1402) from: https://www.militaryaerospace.com/articles/2015/07/hpec-radar-target-recognition.html
Knight, W. (2017). The dark secret at the heart of AI. MIT Technology Review. (Retrieved 10 Aban 1402) from: https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/
Lant, K. (2017, September 12). China, Russia and the US are in an artificial intelligence arms race. Futurism. (Retrieved 10 Aban 1402) from: https://futurism.com/china-russia-and-the-us-are-in-an-artificial-intelligence-arms-race
Margulies, P. (2017). Making autonomous weapons accountable: Command responsibility for computer-lethal force in armed conflicts. In J. D. Ohlin (Ed.), Research handbook on remote warfare (pp. 405–442). Cheltenham: Edward Elgar Publishing.
Marra, W. C., & McNeil, S. K. (2013). Understanding "the loop": Regulating the next generation of war machines. Harvard Journal of Law & Public Policy, 36(3), 1139–1186. https://heinonline.org/HOL/P?h=hein.journals/hjlpp36&i=1153
Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175–183. https://doi.org/10.1007/s10676-004-3422-1
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 1–21. https://doi.org/10.1177/2053951716679679
Mosier, K. L., Skitka, L. J., Heers, S., & Burdick, M. (2001). Aircrews and automation bias: The advantages of teamwork? The International Journal of Aviation Psychology, 11(1), 1–14. https://doi.org/10.1207/S15327108IJAP1101_1
Neslage, K. (2015). Does "meaningful human control" have potential for the regulation of autonomous weapon systems? National Security and Armed Conflict Law Review, 6, 151–180. https://heinonline.org/HOL/P?h=hein.journals/nsecar6&i=155
Parsa, Nahid (2024). Inefficiency of Existing Insurance Laws in Accidents Caused by the Independence of Artificial Intelligence (A Case Study of Fully Self-Driving Cars). Journal of Legal Studies, 16(4), 75-108. [In Persian]
Pilloud, C., de Preux, J., Swinarski, C., & Zimmermann, B. (Eds.). (1987). Commentary on the Additional Protocols of 8 June 1977 to the Geneva Conventions of 12 August 1949. Geneva: International Committee of the Red Cross. https://doi.org/10.1163/9789004277285
Program on Humanitarian Policy and Conflict Resolution. (2010). Autonomous weapons systems: Legal and ethical issues. Cambridge: Harvard University.
Protective Edge. (2015). Report on military operations in Gaza. New York: United Nations Office for the Coordination of Humanitarian Affairs (OCHA).
Roff, H. M., & Moyes, R. (2016, April). "Meaningful human control, artificial intelligence and autonomous weapons": Briefing paper prepared for the informal meeting of experts on lethal autonomous weapons systems. London: Article 36. (Retrieved 10 Aban 1402) from: http://www.article36.org/wp-content/uploads/2016/04/MHC-AI-and-AWS-FINAL.pdf
Scharre, P. (2016, February). Autonomous weapons and operational risk. Washington, D.C.: Center for a New American Security (CNAS). (Retrieved 10 Aban 1402) from: https://s3.amazonaws.com/files.cnas.org/documents/CNAS_Autonomous-weapons-operational-risk.pdf
Scherer, M. U. (2016). Regulating artificial intelligence systems: Risks, challenges, competencies, and strategies. Harvard Journal of Law & Technology, 29(2), 353–400. https://jolt.law.harvard.edu/assets/articlePDFs/v29/29HarvJLTech353.pdf
Schmitt, M. N. (2013). The use of autonomous weapons systems under international law. Harvard National Security Journal, 4(1), 20–55. https://harvardnsj.org/wp-content/uploads/sites/13/2013/01/Schmitt_Final.pdf
Schmitt, M. N. (2017). Autonomous weapon systems and international humanitarian law: A reconceptualization. Harvard National Security Journal, 8, 1–12. https://harvardnsj.org/wp-content/uploads/sites/13/2017/06/Schmitt-Autonomous-Weapon-Systems-and-IHL.pdf
Schmitt, M. N. (Ed.). (2017). Tallinn manual 2.0 on the international law applicable to cyber operations (2nd ed.). Cambridge: Cambridge University Press.
Schuller, A. L. (2017). At the crossroads of control: The intersection of artificial intelligence in autonomous weapon systems with international humanitarian law. Harvard National Security Journal, 8(2), 379–435. https://harvardnsj.org/wp-content/uploads/sites/13/2017/11/Schuller-Final.pdf
Silver, D., & Hassabis, D. (2017). AlphaGo Zero: Learning from scratch. DeepMind Blog. (Retrieved 10 Aban 1402) from: https://deepmind.com/blog/alphago-zero-learning-scratch/
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D. (2018, December 6). AlphaZero: Shedding new light on chess, shogi, and Go. DeepMind Blog. (Retrieved 10 Aban 1402) from: https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go
Simonite, T. (2017, July 19). AI could revolutionize war as much as nukes. WIRED. (Retrieved 10 Aban 1402) from: https://www.wired.com/story/ai-could-revolutionize-war-as-much-as-nukes/
Smith, B. W. (2016). Lawyers and engineers should speak the same robot language. In R. Calo, A. M. Froomkin, & I. Kerr (Eds.), Robot law (pp. 98–117). Cheltenham: Edward Elgar Publishing.
Stensmo, M., & Sejnowski, T. J. (1996). Neural networks, autonomous agents, and the law of armed conflict. Journal of Law, Information and Science, 6(2), 45-60. https://doi.org/10.5778/JLIS.1996.6.stensmo.45
Switzerland, Permanent Mission of Switzerland to the United Nations Office and other international organizations in Geneva. (2016, March 30). Towards a "compliance based" approach to LAWS. Geneva: Permanent Mission of Switzerland. (Retrieved 10 Aban 1402) from: https://documents.unoda.org/wp-content/uploads/2016/04/20160330_Switzerland.pdf
The Economist. (2018, March 15). America v China—The battle for digital supremacy. The Economist. (Retrieved 10 Aban 1402) from: https://www.economist.com/leaders/2018/03/15/the-battle-for-digital-supremacy
Trapp, K. (2013). Great resources mean great responsibility: A framework of analysis for assessing compliance with API obligations in the information age. In D. Saxon (Ed.), International humanitarian law and the changing technology of war (pp. 159–180). Leiden: Brill Nijhoff.
Turner, J. (2018). Robot rules: Regulating artificial intelligence. London: Palgrave Macmillan.
Wagner, M. (2014). The dehumanization of international humanitarian law: Legal, ethical, and political implications of autonomous weapon systems. Vanderbilt Journal of Transnational Law, 47(5), 1371–1424. https://scholarship.law.vanderbilt.edu/vjtl/vol47/iss5/3 /
 
Treaties & Judicial
Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons Which May Be Deemed to Be Excessively Injurious or to Have Indiscriminate Effects (CCW), Protocol III on Prohibitions or Restrictions on the Use of Incendiary Weapons, October 10, 1980, 1342 U.N.T.S. 171.
Hague Convention (IV) respecting the Laws and Customs of War on Land and its annex: Regulations concerning the Laws and Customs of War on Land, October 18, 1907, 205 Consol. T.S. 277.
International Criminal Tribunal for the Former Yugoslavia. (2000, November 16). Prosecutor v. Delalić et al. (Case No. IT-96-21-T), Judgment.
Legality of the Threat or Use of Nuclear Weapons, Advisory Opinion, 1996 I.C.J. 226 (July 8).
Protocol Additional to the Geneva Conventions of 12 August 1949, and relating to the Protection of Victims of International Armed Conflicts (Additional Protocol I), June 8, 1977, 1125 U.N.T.S. 3.
Rome Statute of the International Criminal Court, July 17, 1998, 2187 U.N.T.S. 90

Files

Share

How to cite

Statistics