Description
rawGap = observedSuccess-guessProbability; normalizedGap = rawGap/(1-guessProbability).
Response proportions, confidence losses and raw coder agreement summarize different aspects of recorded performance. Their denominator and missingness rules matter, and one trial or descriptive rate does not establish calibration, reliability or ability.
rawGap = observedSuccess-guessProbability; normalizedGap = rawGap/(1-guessProbability).
When to use this tool
Use Item Guessing Baseline Gap when checking a supplied response, confidence or coder-agreement contribution.
How to use the tool and how it works
- Enter observedSuccess, suppliedGuessProbability. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read rawGap, normalizedGap. Use the output convention below.
Require observedSuccess in [0,1] and suppliedGuessProbability in [0,1). rawGap = observedSuccess-guessProbability; normalizedGap = rawGap/(1-guessProbability).1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| observedSuccess | Observed Success | Required. Finite number; minimum 0; maximum 1. |
| suppliedGuessProbability | Supplied Guess Probability Unit: 1. | Required. Finite number; minimum 0; strictly below 1. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| rawGap | Raw Gap | Required. Finite number. |
| normalizedGap | Normalized Gap | Required. Finite number. |
{
"input": {
"observedSuccess": 0.75,
"suppliedGuessProbability": 0.25
},
"output": {
"rawGap": 0.5,
"normalizedGap": 0.6666666666666666
}
}Limitations and assumptions
- The normalized gap may be negative. Guess probability is supplied, not inferred from response behavior; this is not a significance test or validated ability correction.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use sufficient suitable observations and the appropriate statistical method for stable calibration or reliability conclusions.
References
Similar or alternative tools
- Classroom Observation Coder Agreement Pair
Count compared pairs and exact agreements, then proportion = agreements/compared.
- Metacognitive Confidence Accuracy One Trial
signedGap = confidence-(correct ? 1 : 0); brierLoss = signedGap².