Skip to content

Main content

Option Token Source Map

Map UTF-16 name/value source spans for already-tokenized argv long assignments; other arguments remain whole tokens, no shell source reconstruction.

Loading tool…

Description

Map UTF-16 name/value source spans for already-tokenized argv long assignments; other arguments remain whole tokens, no shell source reconstruction.

An argv vector already consists of literal argument strings. Shell quoting and tokenization happen before this representation; joining argv elements with spaces does not reconstruct a safe command string. Option aliases, duplicates, arity and dependencies are application policies that these tools take explicitly.

Map UTF-16 name/value source spans for already-tokenized argv long assignments; other arguments remain whole tokens, no shell source reconstruction.

When to use this tool

Use this tool to map UTF-16 name/value source spans for already-tokenized argv long assignments; other arguments remain whole tokens, no shell source reconstruction.

How to use the tool and how it works

  1. Enter arguments using the input definitions below.
  2. Run the operation. Resolve malformed-input errors before interpreting the result.
  3. Read result.arguments. Interpret signs, units, ordering and null values using the rules below.

For each supplied argv token beginning --, locate its first equals. Return nameSpan before it and valueSpan after it; otherwise nameSpan covers the whole token and valueSpan is null. All spans are relative to that token.1

Input fieldMeaning and unitsAccepted input
argumentsLiteral argv string vector in source order; each element is already one argument.Dense list of at most 128 strings, each at most 8,192 UTF-16 code units.
Output fieldInterpretation
resultObject containing the fields below.
result.argumentsFormatted literal argv vector or token-local source-map records, as stated above.
json
{
  "input": {
    "arguments": [
      "--x=a",
      "file"
    ]
  },
  "output": {
    "result": {
      "arguments": [
        {
          "index": 0,
          "token": "--x=a",
          "nameSpan": [
            0,
            3
          ],
          "valueSpan": [
            4,
            5
          ]
        },
        {
          "index": 1,
          "token": "file",
          "nameSpan": [
            0,
            4
          ],
          "valueSpan": null
        }
      ]
    }
  }
}

Limitations and assumptions

  • This lexical map does not validate option names or reconstruct shell-source offsets, quotes or escapes.
  • Inputs are literal tokens or explicit records. The restricted long-name grammar is [A-Za-z0-9][A-Za-z0-9-]* where required. No universal option syntax, shell semantics or process execution is inferred.
  • Every declared top-level input is required and unknown fields are rejected. Ordinary strings are limited to 8,192 UTF-16 code units and lists to 128 entries, with tighter operation-specific bounds where stated. JSON text has bounded nesting and node counts.
  • Numeric outputs must be finite and within the absolute safe-integer magnitude; exact larger integers use decimal text. Source positions are zero-based UTF-16 offsets and span ends are exclusive unless a different convention is explicitly stated.

Alternative or Complementary methods

Use the application’s option schema to choose arity, aliases and conflict policy. Pass a literal argv vector to the intended process interface; shell command construction requires separate quoting rules.

References

  1. Tool behavior and example: Option Token Source Map — tool.io

Similar or alternative tools

  • Long Option Assignment Joiner

    Format one literal long-option argv token under a restricted ASCII option-name grammar; returns one argument, not shell quoting.

  • Long Option Assignment Splitter

    Split one strict ASCII --long-name[=value] argv token at the first equals; no shell parsing or implicit next-token consumption.