Description
Count compared pairs and exact agreements, then proportion = agreements/compared.
Response proportions, confidence losses and raw coder agreement summarize different aspects of recorded performance. Their denominator and missingness rules matter, and one trial or descriptive rate does not establish calibration, reliability or ability.
Count compared pairs and exact agreements, then proportion = agreements/compared.
When to use this tool
Use Classroom Observation Coder Agreement Pair when checking a supplied response, confidence or coder-agreement contribution.
How to use the tool and how it works
- Enter first, second, missingToken. Keep parallel arrays aligned. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read compared, agreements, proportion. Use the output convention below.
Require equally long first and second code arrays and at least one pair where neither code equals missingToken. Count compared pairs and exact agreements, then proportion = agreements/compared.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| first | First | Required. Array of text; maximum UTF-16 length 8192; maximum items 256. |
| second | Second | Required. Array of text; maximum UTF-16 length 8192; maximum items 256. |
| missingToken | Missing Token | Required. Text; maximum UTF-16 length 8192. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| compared | Compared | Required. Safe integer. |
| agreements | Agreements | Required. Safe integer. |
| proportion | Proportion | Required. Finite number. |
{
"input": {
"first": [
"on",
"off",
"?"
],
"second": [
"on",
"on",
"off"
],
"missingToken": "?"
},
"output": {
"compared": 2,
"agreements": 1,
"proportion": 0.5
}
}Limitations and assumptions
- This is raw agreement after pairwise missing exclusion, not chance-corrected kappa, a reliability model or verification of coding quality.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
Alternative or Complementary methods
Use sufficient suitable observations and the appropriate statistical method for stable calibration or reliability conclusions.
References
Similar or alternative tools
- Item Guessing Baseline Gap
rawGap = observedSuccess-guessProbability; normalizedGap = rawGap/(1-guessProbability).
- Metacognitive Confidence Accuracy One Trial
signedGap = confidence-(correct ? 1 : 0); brierLoss = signedGap².