patch&proof.
The briefing ↗
THE INDEPENDENT SECURITY RESEARCH JOURNAL
← Incident desk

Defense / 0006-bf-094 / historical case file

MITRE's ATT&CK evaluations publish results without vendor scores

Since 2018, the paid, opt-in programme has released raw detection data rather than rankings, by design.

Historical analysis. Event dates and source publication dates are recorded with the evidence.
Visual published with the cited source, shown for identification of the record. Image: miro.medium.com ↗Source image · credit and context below
The defensive takeaway

Read a vendor's cited MITRE result against the raw technique-level data, and do not treat absence from a round as proof of a detection gap.

What the document says

MITRE has run ATT&CK-based product evaluations since 2018, when it announced the first round using an APT3 emulation plan, with Carbon Black, CounterTack, CrowdStrike, Cylance, Endgame, Microsoft, RSA and SentinelOne as initial participants. The programme's own FAQ states plainly that MITRE is not going to score, rank or rate vendors. Instead, a red team emulates a named threat actor's documented techniques against each participating product, and MITRE publishes what each product detected, and how, at a technique level. Participation has been paid since that FAQ was written, and results for an initial cohort of vendors are released together for fairness, while later rolling-admission participants have results released as each completes. The current methodology overview, as retrieved 16 September 2026, describes a more structured pipeline than the 2018 launch: a threat-intelligence team scopes a real campaign, a red-development team operationalises it, and a detection-engineering team defines the scoring rubric before execution, with a combined scoring framework introduced for the most recent round.

Confidence and limits

The no-rankings policy and the paid, opt-in participation model both come directly from MITRE's own published FAQ and methodology pages, so they are well established. What is harder to verify from the public pages alone is the technical detail of any single round's results, since the results interface itself did not return readable content to an automated fetch; a reader wanting a specific round's findings should open that round's page directly rather than rely on a vendor's summary of it.

Why it mattered

A no-scores policy was a deliberate choice against the incentive to reduce a technical evaluation to one marketable number. A product's standing depended on reading the underlying detections, not a headline percentage, which raised the analytical bar for a defender comparing tools but also made the results harder for a non-specialist buyer to use unassisted.

Defensive takeaway

When a vendor cites MITRE ATT&CK Evaluations results, ask for the specific techniques it detected and how, not only the vendor's own summary framing of the round.

  • Does the vendor's marketing quote a score that the programme itself says does not exist?
  • Has the vendor participated in the most recent round, or is it citing an older one?
  • If a vendor is absent from a round, does that reflect a declined paid evaluation or a demonstrated gap? The public record does not say which by default.

Reading these evaluations well means treating them as a detailed, technique-level record rather than a competitive leaderboard. The programme was built that way on purpose, and a summary that flattens it back into a ranking is doing something the evaluation itself declined to do.

Evidence ledger

Open the source. Check the claim.

MITRE's own 2018 launch post, FAQ and current methodology page establish the no-rankings policy, the paid participation model and the general evaluation pipeline. The live results interface did not return readable content to this review, so specific round-by-round technical findings are not restated here.

Incident / event
Method analysis
Lead source published
2018-07-06
MITRE's ATT&CK-based Evaluations for Security Vendors are Underway2018-07-06 · project-primary
Announces the 2018 launch of ATT&CK-based evaluations using an APT3 emulation plan and names the first participating vendors.
↗
ATT&CK-based Product Evaluations: Frequently Asked Questions2018-08-14 · project-primary
States MITRE's policy not to score, rank or rate vendors, describes the paid participation model, and explains how initial and rolling-admission results are released.
↗
Methodology Overview - ATT&CK EvaluationsSource date not assigned · project-primary
Describes the current evaluation pipeline and scoring framework, as retrieved 16 September 2026.
↗
Search the evidence
Source image / inspection view

View original source ↗Source credits and usage context remain attached. Check original terms before reusing an image.