Alert-Threshold Tuning
Measure continuous-audit alert precision, recall, and false positives.
Description
Alert precision and recall summarize labeled outcomes; tuning requires comparing actual thresholds separately.
Alert precision and recall summarize labeled outcomes; tuning requires comparing actual thresholds separately.1
Measure continuous-audit alert precision, recall, and false positives.
When to use Alert-Threshold Tuning
Use this tool to measure continuous-audit alert precision, recall, and false positives.
Prepare the documented facts, aligned totals or expected/observed evidence labels requested. Choose applicable requirements and thresholds separately before interpreting a planning screen.
How to use Alert-Threshold Tuning and understand its results
- Prepare the documented facts, aligned totals or expected/observed evidence labels requested. Choose applicable requirements and thresholds separately before interpreting a planning screen.
- Enter truePositiveAlerts, falsePositiveAlerts, missedEvents, minimumPrecisionPercent, minimumRecallPercent using the field table and calculation rules, then run the tool.
- Read precisionPercent, recallPercent, falsePositivePercent, meetsTargets. The explanation below defines the units, signs, ordering and any flags or placeholder values.
- If the tool reports an input issue, correct the values and run it again. Apply the stated assumptions when interpreting a successfully returned result.
| Input field | Meaning | Value type; see rules below | Requirement |
|---|---|---|---|
| truePositiveAlerts | True-positive alerts | integer; minimum: 0 | Required |
| falsePositiveAlerts | False-positive alerts | integer; minimum: 0 | Required |
| missedEvents | Known missed events | integer; minimum: 0 | Required |
| minimumPrecisionPercent | Minimum precision (%) | number; minimum: 0; maximum: 100 | Required |
| minimumRecallPercent | Minimum recall (%) | number; minimum: 0; maximum: 100 | Required |
Enter nonnegative integer counts with at least one alert and one known event, minimum percentages 0–100. Return percentages, FP share of alerts and inclusive two-target gate.
precision=TP/(TP+FP); recall=TP/(TP+missed) Units, indexing, boundary conditions and exceptional cases are defined in the calculation rules above.
| Output field | Meaning | Value type |
|---|---|---|
| precisionPercent | Alert precision (%) | number; minimum: 0; maximum: 100 |
| recallPercent | Alert recall (%) | number; minimum: 0; maximum: 100 |
| falsePositivePercent | False-positive share (%) | number; minimum: 0; maximum: 100 |
| meetsTargets | Meets supplied precision and recall targets | boolean |
{
"input": {
"truePositiveAlerts": 80,
"falsePositiveAlerts": 20,
"missedEvents": 20,
"minimumPrecisionPercent": 75,
"minimumRecallPercent": 75
},
"output": {
"precisionPercent": 80,
"recallPercent": 80,
"falsePositivePercent": 20,
"meetsTargets": true
}
}Limitations and assumptions
FP share is not false-positive rate among true negatives. No threshold search, event labeling or system execution is performed.
Numeric inputs, where present, must be finite. Follow the calculation-specific rules for overflow, underflow and exceptional cases. Missing required fields, extra fields, wrong value types and violations of the calculation rules are input errors. A successfully returned false flag is an evaluated outcome; its meaning is explained above.
Alternative or Complementary methods
Use Disparate Impact: Calculate selection-rate disparity against a caller-supplied ratio threshold. For a complementary task, use Inherent Risk Score: Combine inherent-risk factors into a weighted score.
References
-
Alert-Threshold Tuning — tool.io
Similar or alternative tools
- Disparate Impact
Calculate selection-rate disparity against a caller-supplied ratio threshold.
- Inherent Risk Score
Combine inherent-risk factors into a weighted score.