Low-Cardinality Category Lister

List first-encounter distinct nonempty values for data columns whose observed category count is at most an explicit maximum, excluding an explicit header.

Description

Low-cardinality columns can contain useful category vocabularies.

Low-cardinality columns can contain useful category vocabularies. Counting distinct decoded values avoids confusing repeated observations with separate categories.^[1]

List first-encounter distinct nonempty values for data columns whose observed category count is at most an explicit maximum, excluding an explicit header.

When to use Low-Cardinality Category Lister

Use this for the following task: List first-encounter distinct nonempty values for data columns whose observed category count is at most an explicit maximum, excluding an explicit header.

Prepare the CSV text, decoded cells or byte values requested below. Choose delimiter and header flags explicitly when those fields are present.

How to use Low-Cardinality Category Lister and understand its results

Prepare the CSV text, decoded cells or byte values requested below. Choose delimiter and header flags explicitly when those fields are present.
Enter csv, delimiter, hasHeader, maxCategories using the field table, then run the tool.
Read columns. Interpret flags, units, ordering and placeholder values using the rules below.
Review any reported issue or failed check, correct the supplied values when appropriate, and verify the result in its intended context.

Input field	Meaning	Value type and bounds	Requirement
csv	CSV text	string; maximum text length 100000	Required
delimiter	Delimiter	string; choose ",", ";", "\t"	Required
hasHeader	Has header	boolean	Required
maxCategories	Max categories	integer; minimum 1; maximum 10000	Required

Parse rectangular records with explicit header handling. Exclude exactly empty strings, deduplicate in encounter order and return qualifying zero-based columns with values.^[2]

Parse delimiters only outside quoted fields and decode doubled quotes. A quoted line break stays inside its cell; malformed quoting is returned as an input error.

emit column when 1 ≤ distinct nonempty values ≤ maxCategories

This summarizes the declared operation. Units, indexing, inclusivity and exceptional cases are explained in the surrounding text.

Output field	Meaning	Value type
columns	Columns	list of record: column (integer), values (list of string)

json

{
  "input": {
    "csv": "id,color\n1,red\n2,blue\n3,red",
    "delimiter": ",",
    "hasHeader": true,
    "maxCategories": 2
  },
  "output": {
    "columns": [
      {
        "column": 1,
        "values": [
          "red",
          "blue"
        ]
      }
    ]
  }
}

Limitations and assumptions

Whitespace-only values count as categories. The tool does not infer semantic types or choose a universal category limit.

Where CSV is parsed, the shared parser accepts comma, semicolon or tab separators, doubled quotes and quoted line breaks, with CR/LF/CRLF record endings. Blank records remain records. CSV inputs are bounded to 100000 UTF-16 units and 10000 fields. Malformed requests produce input errors. A successfully evaluated request can still contain false flags, empty result lists or a result outside the supplied threshold; those outcomes have the meanings stated above.

Alternative or Complementary methods

Choose High-Cardinality Column Flagger for this operation: Flag columns whose distinct-to-observed ratio exceeds an explicit threshold, counting exact nonempty data cells and excluding an explicit header; zero observations never flag. For a complementary task, use First-N Distinct Values Per Column for this operation: Sample the first N distinct decoded data values per column in encounter order, counting empty strings as values and reporting the full distinct count.

Further background: Format and language reference: Low-Cardinality Category Lister.

References

Format and language reference: Low-Cardinality Category Lister — RFC Editor
Low-Cardinality Category Lister — tool.io

Similar or alternative tools

High-Cardinality Column Flagger
Flag columns whose distinct-to-observed ratio exceeds an explicit threshold, counting exact nonempty data cells and excluding an explicit header; zero observations never flag.
First-N Distinct Values Per Column
Sample the first N distinct decoded data values per column in encounter order, counting empty strings as values and reporting the full distinct count.