Quoted Token List Parser

Parse whitespace-separated double-quoted JSON string tokens only, preserving literal spaces and decoded JSON escapes; no shell execution.

Description

Parse whitespace-separated double-quoted JSON string tokens only, preserving literal spaces and decoded JSON escapes; no shell execution.

A sequence of JSON string literals can represent text tokens containing whitespace. The separators lie outside the quotes.

When to use this tool

Parse whitespace-separated double-quoted JSON string tokens only, preserving literal spaces and decoded JSON escapes; no shell execution.

Prepare the supplied text fields for this operation.
Compare the complete example with the representation used by your own input before carrying the result into another step.

How to use the tool and how it works

Enter text. Use the exact text syntax, units and index conventions explained below.
Run the tool. If input validation fails, correct the supplied fields before interpreting any result.
Read result.tokens. Interpret false, null and empty lists using the distinctions below; they can be valid outputs.

Skip whitespace, scan each double-quoted token while respecting escaped characters and validate it with JSON parsing. Require whitespace or end after every token, retaining decoded strings and accepting an empty list.^[1]

Input field	Meaning and representation	Accepted input
text	text.	Text; at most 8,192 UTF-16 code units, subject to the grammar above

Output field	Interpretation
result	Object containing the named outputs below.
result.tokens	The tokens list in the ordering and representation described above.

json

{
  "input": {
    "text": "\"a b\" \"c\\n\" \"\""
  },
  "output": {
    "result": {
      "tokens": [
        "a b",
        "c\n",
        ""
      ]
    }
  }
}

Limitations and assumptions

Single quotes, unquoted tokens and non-JSON escapes are rejected. No shell expansion, comments or command interpretation occurs.
All declared input fields are required and unexpected top-level fields are rejected. Numbers must be finite; lists are limited to 128 entries and ordinary text fields to 8,192 UTF-16 code units, with tighter limits where stated.
JSON text inputs must parse within bounded nesting and size limits. Numeric outputs outside the safe finite range are rejected; large exact integer results use decimal strings. Indexes and half-open spans are zero-based unless a rank is explicitly described as one-based.

Alternative or Complementary methods

Use a parser for the actual target format when the task requires CSV, shell, URL, calendar or filesystem semantics. These tools implement the specific small grammars stated here, without adding those surrounding conventions.

References

Tool behavior and example: Quoted Token List Parser — tool.io

Similar or alternative tools

Quoted Token List Formatter
Format explicit strings as whitespace-separated JSON quoted tokens; deliberately not a shell quoting or command builder.
Balanced Parenthesis Token Splitter
Split a literal delimiter only outside balanced parentheses; parentheses are retained, quotes and escapes have no special meaning.