What it means
Precision answers: of the positive results a detector produced in a defined evaluation, how many were correct? The denominator is all positive results, not all events observed and not all real attacks. Precision helps explain the investigation burden of a rule or model, but it says nothing by itself about attacks that were missed. Always state what counts as a positive, how outcomes were labeled, and which time period or dataset was used. Unreviewed alerts are unknown outcomes, not automatically false positives.
AN ILLUSTRATIVE SCENARIO
Comparing two versions of an account-theft detector
In a labeled exercise, a detector produces twenty alerts. Twelve identify the target attack behavior and eight match legitimate activity. Precision is twelve divided by twenty, or sixty percent. A new version produces ten alerts, nine of them correct: ninety percent precision. That improvement sounds useful, but the team still checks whether the new version missed more attacks before deciding it is better for the intended use.
Put it to work
- Define the target behavior and the unit being counted, such as alerts or incidents. Avoid mixing several alerts from one incident with counts of unique incidents.
- Label a suitable evaluation set using evidence and consistent rules. Record true positives and false positives, and separate cases whose outcome remains unresolved.
- Calculate TP divided by TP plus FP, then report sample size and context alongside the result. Compare recall and investigation cost before changing a detector’s threshold.
How to check your work
Recompute the result from a small labeled table. Confirm that every item in the denominator is a positive result, and inspect a sample of both correct and incorrect labels. Document any sampling or review bias.
Connect the ideas
- Recall
Detected true events divided by all true events in a defined evaluation set: TP / (TP + FN). Unknown missed events limit production estimates.
- False positive
An alert that did not represent the harmful event the detector was meant to catch.
- Alert
A notification that a rule or model wants a human to look at one or more records.
- Coverage
An explicit statement of which sources, time windows, and rows were actually collected, including truncation.