Description
signedGap = confidence-(correct ? 1 : 0); brierLoss = signedGap².
Response proportions, confidence losses and raw coder agreement summarize different aspects of recorded performance. Their denominator and missingness rules matter, and one trial or descriptive rate does not establish calibration, reliability or ability.
signedGap = confidence-(correct ? 1 : 0); brierLoss = signedGap².
When to use this tool
Use Metacognitive Confidence Accuracy One Trial when checking a supplied response, confidence or coder-agreement contribution.
How to use the tool and how it works
- Enter confidence, correct. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read signedGap, brierLoss. Use the output convention below.
Require confidence in [0,1]. signedGap = confidence-(correct ? 1 : 0); brierLoss = signedGap².1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| confidence | Confidence | Required. Finite number; minimum 0; maximum 1. |
| correct | Correct | Required. Boolean. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| signedGap | Signed Gap | Required. Finite number. |
| brierLoss | Brier Loss | Required. Finite number. |
{
"input": {
"confidence": 0.8,
"correct": true
},
"output": {
"signedGap": -0.19999999999999996,
"brierLoss": 0.03999999999999998
}
}Limitations and assumptions
- This is a single binary outcome's calibration loss. It does not establish a learner's metacognitive ability or a stable calibration estimate from one trial.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use sufficient suitable observations and the appropriate statistical method for stable calibration or reliability conclusions.
References
Similar or alternative tools
- Item Guessing Baseline Gap
rawGap = observedSuccess-guessProbability; normalizedGap = rawGap/(1-guessProbability).
- Classroom Observation Coder Agreement Pair
Count compared pairs and exact agreements, then proportion = agreements/compared.