Robots Meta Content Values Lookup

Resolves Google Search robots index and follow directives, including all/none and restrictive conflicts. Reports recognized boolean directives and unhandled tokens; parameterized directives are outside this profile.

Description

Robots metadata communicates indexing and link-following preferences to search crawlers.

Robots metadata communicates indexing and link-following preferences to search crawlers. Conflicting restrictive directives override permissive ones in this Google Search-oriented profile.^[1]

The tool returns indexAllowed, followAllowed, recognized, unhandled. The fields describe the supplied local pattern or effective state, or contain the formatted markup shown below.

When to use Robots Meta Content Values Lookup

Use this for the following task: Resolves Google Search robots index and follow directives, including all/none and restrictive conflicts. Reports recognized boolean directives and unhandled tokens; parameterized directives are outside this profile. Provide the requested local fields extracted from the relevant element or attribute.

How to use Robots Meta Content Values Lookup and understand its results

Prepare the parsed element names, attribute values or direct element-child lists requested below. Keep attribute presence separate from its value.
Enter content according to the field table, then run the tool.
Read indexAllowed, followAllowed, recognized, unhandled. Check the effective state, recognized-token flags or generated markup against the rules below.
Correct any reported local mismatch and check the resulting markup in its intended document and browser context.

Input field	Meaning	Value type and bounds	Requirement
content	Content	string; maximum text length 100000	Required

Supply the content string. Split comma or ASCII-whitespace-separated directives, fold ASCII case and deduplicate recognized and unhandled tokens in first-seen order.

indexAllowed excludes none/noindex; followAllowed excludes none/nofollow

The symbols and boundary conditions refer to the supplied fields and the implementation described above.

Output field	Meaning	Value type
indexAllowed	Index Allowed	boolean
followAllowed	Follow Allowed	boolean
recognized	Recognized	list of string
unhandled	Unhandled	list of string

json

{
  "input": {
    "content": "index, noindex, follow, max-snippet:20"
  },
  "output": {
    "indexAllowed": false,
    "followAllowed": true,
    "recognized": [
      "index",
      "noindex",
      "follow"
    ],
    "unhandled": [
      "max-snippet:20"
    ]
  }
}

Limitations and assumptions

Only the stored Boolean directive profile is handled; parameterized tokens such as max-snippet:20 are returned unhandled. Flags do not guarantee crawling, indexing or actual search-engine behavior.

The tool does not fetch a page, run a browser or validate an entire HTML document. Malformed tool requests produce input errors; valid requests can still report unrecognized values or failed local checks.

Alternative or Complementary methods

Choose nofollow/sponsored/ugc Difference Lookup when you need its operation: Classifies nofollow, sponsored and ugc link annotations according to Google Search guidance. These annotations do not suppress browser navigation. Choose Canonical link rel Check for the complementary operation: Checks the canonical relation and a nonempty parsed href. Relative href values are accepted; this does not establish URL validity or canonical selection by search engines.

Background reference: Robots meta tag, data-nosnippet, and X-Robots-Tag specifications.

References

Robots meta tag, data-nosnippet, and X-Robots-Tag specifications — Google Search Central

Similar or alternative tools

nofollow/sponsored/ugc Difference Lookup
Classifies nofollow, sponsored and ugc link annotations according to Google Search guidance. These annotations do not suppress browser navigation.
Canonical link rel Check
Checks the canonical relation and a nonempty parsed href. Relative href values are accepted; this does not establish URL validity or canonical selection by search engines.