US Air Force Chief of AI Testing and Operations Colonel Tucker Hamilton has been embroiled in controversy for claiming that a military drone could decide to kill an operator to accomplish a mission. Last May, he described a scenario where this would be possible during a conference in London, speaking on behalf of his country’s Air Force.
At the meeting, organized by the Royal Airspace Society, Hamilton said that a military operation simulation with drones predicted that the armed vehicle should identify and target a threat — a surface-to-air missile. In this scenario, the operator asked the equipment to destroy the target. In some scenarios, however, the operator asked the drone not to hit the target, but in this case, the machine lost points for not destroying it.
Machine learning, according to the official, made the AI understand that it would be advantageous to eliminate the operator, as it prevented the mission from being accomplished several times. In the simulation, this would have happened, and in response, the drone was taught that it could not kill the operator.
He would then choose to destroy the communication tower, used by the human to prevent the target from being destroyed by giving instructions. A transcript of the entire account was published on the event’s website, which led to controversy among the public and especially the military.
Controversies, retractions and possibilities
Last Friday (2), the US Air Force released a note denying that a simulation of this nature had ever taken place. Hamilton himself communicated to the Royal Airspace Society that he would have expressed himself badly, but still stated that the scenario would be possible: “we have never done this experiment, but we would not have to do it to know that it is a plausible result”. He goes on to say that this is one of the real-world challenges posed by AI, ensuring that the Air Force is committed to its development ethically.
The retraction was not very convincing for several organizations, such as the Russian military channel BMPD, which stated on its Telegram channel: “The Colonel (Hamilton) is not the type of person to tell jokes in a serious defense conference”. He also remembered that he is responsible for one of the most advanced artificial intelligence programs in the US Armed Forces — Venom, which is located at Eglin Air Force Base. In it, F-16 fighters were robotized to function as drones.
And even when they aren’t in remotely operated drones, AIs are on many warplanes around the world, helping pilots make decisions faster. Ethical concerns about drones used to revolve around the lack of legal accountability for operators, who killed targets thousands of miles away, almost unaccountably. Even so, drones remain very important in conflicts such as the recent invasion of Ukraine by Russian forces.
Before he died in 2018, physicist Stephen Hawking already commented on the risks of AI for humanity. At the UN there is a separate panel just to discuss the danger of autonomous weapons, with the comically long name of the Convention on Prohibitions or Restrictions on the Use of Some Conventional Weapons Which May Be Considered Excessively Dangerous or Having Indiscriminate Effects.
The urgency on the subject has grown thanks to the domestic use of AIs, with services such as ChatGPT and Midjourney making images and texts generated at the request of users pop up. Research in the area has been the subject of concerns and warnings, such as the recent document published last Tuesday (30) by more than 350 scientists, including the creator of ChatGPT, asking that those responsible treat AI as a pandemic or a nuclear war , which must have the risks controlled and strongly monitored.
Hamilton himself last year claimed that AI is “not cool to have, it’s not fashion and it is forever changing our society and our military”. In any case, it is yet another warning sign that technology needs supervision and care, otherwise it becomes a risk to humanity — like any tool created by ourselves.
Source: DefenceIQ, Royal Aeronautical SocietyFolha de S. Paulo
