VigilSAR Publishes Public AI Model Leaderboard for Defense-Related Tasks

VigilSAR Defense LLM Benchmark — The public benchmark page — aggregate results public, task set private. Source: vigilsar.com

In a move that could reshape how AI models are evaluated for defense and intelligence applications, VigilSAR has publicly released its latest LLM leaderboard. Unlike typical rankings, this leaderboard focuses specifically on trustworthy AI for intelligence-surveillance-reconnaissance work, emphasizing reasoning, reporting, and restraint rather than general trivia or broad capabilities.

The evaluation involved 14 different models tackling 300 tasks, with scores collected as of July 17, 2026. Importantly, the results are based on an private test set that remains secret to prevent models from training on it, ensuring the integrity of the rankings. The standings are presented as confidence bands rather than exact ranks, with the top model, claude-fable-5, confidently leading at a score of 67.77 and firmly placed in Band A.

A notable new entry is Moonshot’s Kimi K3, which debuted at an impressive 64.65 points, earning it a spot in Band B. Remarkably, Kimi K3 outperformed all GPT and Gemini models on the leaderboard, marking a significant milestone for the Chinese newcomer in this specialized domain. The leaderboard also highlights the performance of the GPT-5.x family, categorized mainly in Bands C and D, and Gemini models in Bands E and F.

VigilSAR public LLM leaderboard — The leaderboard — compare bands, not rank numbers. Source: vigilsar.com/benchmark

The scoring system incorporates confidence intervals, held-out gaps, and a reference row to ensure transparency and reliability. Additionally, the evaluation considers deployment readiness, with at least one model scored as “sovereign-deployable,” reflecting real-world operational constraints. VigilSAR emphasizes that vendor claims are not enough—only measured performance on rigorous tests establishes trustworthiness in sensitive settings.

The purpose of this public ranking is to foster honest comparison and accountability among AI developers working on defense-related tasks. The site openly publishes cost-per-correct-answer metrics and other economic factors to promote transparency. As the AI landscape evolves, VigilSAR’s approach underscores the importance of objective, independent evaluation over marketing claims.

This initiative highlights how the AI community is increasingly focused on safety and reliability in critical applications. With the debut of Kimi K3 and the ongoing transparency efforts, defense agencies and developers alike now have a clearer picture of which models are best suited for sensitive intelligence work. For those interested, you can explore the ongoing rankings and detailed scores at the public leaderboard.