The US military is keeping it secret who won the real-world dogfight between a human pilot and an AI flying an F-16 fighter jet

The tests occurred last September at Edwards Air Force Base in California, where the two jets flew at speeds of up to 1,200 miles per hour and practiced a range of dogfighting scenarios. Footage released by the US Defense Advanced Research Projects Agency showed the jets maneuvering through the skies around one another in a step up from previous testing.

The AI-controlled fighter jet, called the X-62A Variable Stability In-flight Simulator Test Aircraft, is a modified version of an F-16.

Col. James Valpiani, commandant of the US Air Force Test Pilot School, said the AI agents in the aircraft were able to be upgraded or switched out between missions, potentially complicating air warfare for the enemy.

“We were able to generate changes to software overnight,” he explained, adding that “we were able to upload the software changes to the aircraft while it was holding short, ready to take off and even airborne.” He said that “we’re able to transition between multiple versions of the same AI agent airborne, between combat sets.”

The X-62A has taken at least 21 test flights since it was built in December 2022.

Despite the historic nature of this moment, with DARPA calling it a “transformational moment in aerospace history,” the US military isn’t sharing an answer to the big question: who won. Officials were mum on details but emphasized how important the exercise was for future progress.

Although it remains unclear if the man or machine prevailed in this moment, AI’s track record in this area has been stellar. Back in August 2020, AI won a simulated dogfight against a human operator 5-0. It was a sweep, and the human pilot never even scored a hit.

At the time, former US Navy and US Air Force pilots told Business Insider they weren’t surprised by the outcome given that the fight may not have necessarily been realistic.

One argued that the AI algorithm appeared to have had access to information that it wouldn’t have in the real world and was operating in a combat environment it was trained for. In many ways, it was like a video game, not actual air-to-air combat with real aircraft racing into the merge. They said they AI would’ve probably “crashed and burned” in a real fight.

That was years ago, though, and AI has evolved tremendously. Hefron told reporters that they trained this AI model for the real-world fight by conducting “over millions and in some cases, billions” of runs, rewarding specific successful outcomes and reinforcing positive behaviors.

Both Hefron and Valpiani also said that trusting the AI was a key priority for the program, emphasizing a “responsible application of autonomy” and situations where the AI must follow explicit instructions and guidelines.

Dogfights, even in a training situation, can be dangerous. Last September’s test built on a previous experience in which an AI agent piloting the X-62A battled simulated threats.

Hefron said that the simulation and real-world have their unique differences, leading to problems when AI has been trained for specific situations that may not happen.

“That’s something we’ve been able to collect data for, we’ve been able to kind of dissect and understand, and then come up with many new approaches to tackle that problem,” he said.

In 2019, DARPA said that “no AI currently exists” that “can outduel a human strapped into a fighter jet in a high-speed, high-G dogfight,” but warfare is evolving, especially when it comes to manned-unmanned teaming and autonomous systems.

That may or may not still be true — the military won’t say — but the progress in this space is notable.

DARPA said it “seeks to automate air-to-air combat, enabling reaction times at machine speeds and freeing pilots to concentrate on the larger air battle.”