Duplicate Rows Extractor

Extract every CSV data record whose exact decoded row occurs more than once, retaining all members of duplicate groups in original order and preserving an optional header.

Description

Duplicate extraction retains all members of repeated-row groups.

Duplicate extraction retains all members of repeated-row groups. This differs from returning only later copies or deleting duplicates.^[1]

Extract every CSV data record whose exact decoded row occurs more than once, retaining all members of duplicate groups in original order and preserving an optional header.

When to use Duplicate Rows Extractor

Use this for the following task: Extract every CSV data record whose exact decoded row occurs more than once, retaining all members of duplicate groups in original order and preserving an optional header.

Prepare the CSV text, decoded cells or byte values requested below. Choose delimiter and header flags explicitly when those fields are present.

How to use Duplicate Rows Extractor and understand its results

Prepare the CSV text, decoded cells or byte values requested below. Choose delimiter and header flags explicitly when those fields are present.
Enter csv, delimiter, hasHeader using the field table, then run the tool.
Read csv, duplicateRowCount. Interpret flags, units, ordering and placeholder values using the rules below.
Review any reported issue or failed check, correct the supplied values when appropriate, and verify the result in its intended context.

Input field	Meaning	Value type and bounds	Requirement
csv	CSV text	string; maximum text length 100000	Required
delimiter	Delimiter	string; choose ",", ";", "\t"	Required
hasHeader	Has header	boolean	Required

Parse records, exclude an explicit header, count JSON-serialized decoded row keys and retain every row with frequency above one in original order.^[2]

Parse delimiters only outside quoted fields and decode doubled quotes. A quoted line break stays inside its cell; malformed quoting is returned as an input error.

The shared writer doubles interior quotes and quotes cells when needed for delimiters or line breaks, including a single empty-field record. It normally emits LF record endings without a terminal newline; any tool-specific quote-all behavior is stated above.

duplicateRowCount = number of data rows whose exact row key occurs more than once

This summarizes the declared operation. Units, indexing, inclusivity and exceptional cases are explained in the surrounding text.

Output field	Meaning	Value type
csv	CSV text	string
duplicateRowCount	Duplicate row count	integer

json

{
  "input": {
    "csv": "name\nAda\nGrace\nAda",
    "delimiter": ",",
    "hasHeader": true
  },
  "output": {
    "csv": "name\nAda\nAda",
    "duplicateRowCount": 2
  }
}

Limitations and assumptions

Exact cell strings and boundaries determine equality. A group of two contributes two rows, not one redundant copy.

Where CSV is parsed, the shared parser accepts comma, semicolon or tab separators, doubled quotes and quoted line breaks, with CR/LF/CRLF record endings. Blank records remain records. CSV inputs are bounded to 100000 UTF-16 units and 10000 fields. Malformed requests produce input errors. A successfully evaluated request can still contain false flags, empty result lists or a result outside the supplied threshold; those outcomes have the meanings stated above.

Alternative or Complementary methods

Choose Unique Rows Dedupe First Occurrence for this operation: Remove exact duplicate decoded CSV records after an optional header, retaining first occurrences and their original order; cell boundaries remain significant. For a complementary task, use Row Hash Digest Dedupe Key for this operation: Fingerprint one decoded row using legacy MD5 over UTF-8 JSON array text, preserving cell boundaries; confirm row equality before deduplication because MD5 is not collision resistant.

Further background: Format and language reference: Duplicate Rows Extractor.

References

Format and language reference: Duplicate Rows Extractor — RFC Editor
Duplicate Rows Extractor — tool.io

Similar or alternative tools

Unique Rows Dedupe First Occurrence
Remove exact duplicate decoded CSV records after an optional header, retaining first occurrences and their original order; cell boundaries remain significant.
Row Hash Digest Dedupe Key
Fingerprint one decoded row using legacy MD5 over UTF-8 JSON array text, preserving cell boundaries; confirm row equality before deduplication because MD5 is not collision resistant.