Description
Reject overlap between knownTokens and unknownTokens.
Exact label matching answers whether supplied strings agree under a chosen vocabulary. It preserves a useful distinction between missing, unknown, ambiguous, and unrecognized records.
Reject overlap between knownTokens and unknownTokens.
When to use this tool
Use Historical Roster Unknown Status Token when you need an explicit record-label comparison or deterministic key.
How to use the tool and how it works
- Enter token, unknownTokens, knownTokens. Keep parallel arrays aligned. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read status. Use the output convention below.
Reject overlap between knownTokens and unknownTokens. Classify token by exact membership as unknown, known, or unrecognized.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| token | Token | Required. Text; maximum UTF-16 length 8192. |
| unknownTokens | Unknown Tokens | Required. Array of text; maximum UTF-16 length 8192; maximum items 256. |
| knownTokens | Known Tokens | Required. Array of text; maximum UTF-16 length 8192; maximum items 256. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| status | Status | Required. Text; maximum UTF-16 length 8192. |
{
"input": {
"token": "?",
"unknownTokens": [
"?"
],
"knownTokens": [
"present",
"absent"
]
},
"output": {
"status": "unknown"
}
}Limitations and assumptions
- Unknown and unrecognized are distinct vocabulary states; neither is resolved from external historical evidence.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
Alternative or Complementary methods
Normalize or reconcile identifiers only under a documented policy, then verify the underlying source records separately.
References
Similar or alternative tools
- Evidence Claim Qualifier Token
Reject duplicate suppliedQualifiers, then return exact token membership and its zero-based index, or -1 when absent.
- Product Label Unit Token
Check unitToken by exact membership in allowedUnits.