Website Tracking Tools

Spider Simulator

See the text-oriented view a basic search-engine crawler extracts from a page's returned HTML — its meta tags, heading outline, internal and external link counts, and visible text, with no JavaScript executed.

Website Tracking ToolsServer-backed/spider-simulator

Step by step

How to use the Spider Simulator

Enter the page URL you want to inspect into the Page URL field
Click Simulate crawler to fetch the live HTML server-side
Review the crawler-visible title, meta, headings, links, and extracted body text
Check the indexable status and word count to confirm crawlers see real content

Pro tip: If visible text is thin (under 150 words) or noindex appears, crawlers may skip the page despite a rich-looking browser view.

Seeing the page like a crawler

Visitors see your page after the browser has applied styles, images, and scripts. A simple crawler sees something far plainer: the raw HTML stripped down to text and structure. This simulator reproduces that view by fetching the page from our Cloudflare edge and extracting the meta tags, the heading outline, the links, and the visible text from the markup that was returned.

Laying the page out this way exposes a gap many audits miss: content that looks rich to a human but is sparse or disordered once the styling is gone.

What the extraction returns

Crawl signals — the title, meta description, canonical, robots directive, and the HTML lang attribute a crawler reads first, with indexability derived from the robots tag.
Word count — how many words of visible body text were found in the HTML, flagged when it falls below roughly 150 words.
Heading outline — the heading sequence with its levels, up to the first fifteen, which reveals whether the page has a logical, scannable structure.
Link counts — how many links resolve to the same host (internal) versus another host (external), summarised as a single in / out figure.
Visible text preview — the readable body content extracted from the HTML with tags stripped, shown up to roughly the first 900 characters.

The JavaScript blind spot

This is the most important thing to understand about the tool. It does not run JavaScript. If your headings, copy, or links are injected by a framework after the page loads in a browser, they exist only in the rendered DOM, not in the HTML our edge receives, so they will not appear in the extracted view, and the word count will look thin even on a page that feels full.

That is precisely the value of the simulation: when expected content is missing here, it warns you that a basic crawler relying on the raw HTML may miss it too, and that you may be depending on rendering that not every crawler performs.

Reading the results honestly

A thin word count and an empty heading outline on a visually full page is a signal to check whether your content is server-rendered. A healthy extraction, with a sensible heading order and a meaningful word count, means the essentials are present in the HTML itself. Watch the robots line too: if it contains noindex, the page is asking to be kept out of search regardless of how good the content is. Bear in mind this models a basic, text-only crawler; the most capable search engines do render JavaScript, so a gap here is a caution flag rather than proof a page is invisible everywhere. As with the other tools, the fetch uses an identified tool user agent and only public URLs are reachable, so pages that block automated requests may return little to analyse.

Worked examples

Real inputs and their results

A server-rendered article

Input: https://www.iana.org/help/example-domains

Output: Indexable: Yes · Words seen: 176 · Headings: 2 · Links: 24 in / 4 out • Title: "Example Domains" · Meta description: (missing) · Canonical: (none) · Robots: (default — indexable) · HTML lang: (unset) • Outline: H1 "Example Domains", H2 "Further Reading" • Visible text preview opens: "Domains Protocols Numbers About Example Domains As described in RFC 2606 and RFC 6761 , a number of domains such as example.com and example.org are maintained for documentation purposes..."

Measured 2026-07-17. The body copy sits in the HTML, so a text-only crawler reads it without running any JavaScript. The unset HTML lang attribute is the one thing flagged here.

A page with almost no text to read

Input: https://example.com

Output: Indexable: Yes · Words seen: 19 · Headings: 1 · Links: 0 in / 1 out • Title: "Example Domain" · Meta description: (missing) · Canonical: (none) · HTML lang: en • Outline: H1 "Example Domain" • Visible text preview: "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more"

Measured 2026-07-17. Nineteen words trips the thin-content flag. This page really is that sparse rather than JavaScript-rendered — the extraction alone cannot tell those two cases apart, which is why a low count is a prompt to inspect the HTML yourself.

FAQ

Spider Simulator: common questions

Why does the page look empty here but full in my browser?

Your browser ran the JavaScript that builds the page; this tool does not. A near-zero word count and empty outline mean the content is not in the raw HTML our edge received.

Does Google see exactly what this shows?

Not exactly. This models a basic text-only crawler. Google can render JavaScript, so a gap here is a warning to investigate rather than proof the content is hidden from every engine.

How does it count internal versus external links?

It resolves each link in the HTML and compares its host to the page's host: same host counts as internal, a different host counts as external, shown as a single in / out figure.

Why is the word count flagged as thin?

The tool flags visible body text under roughly 150 words because that is often a sign the real content is loaded by JavaScript or the page is genuinely sparse for a crawler to read.

Is the full visible text shown?

No. The preview is capped at roughly the first 900 characters with tags stripped, which is enough to confirm whether real body text exists in the HTML without dumping the whole page.

How is this different from the Website Checker?

The Website Checker summarises core health signals like status and tag lengths; this tool reconstructs the fuller text-and-structure view a crawler extracts, including the heading outline, link split, and a body-text preview.

Site standards

How HighSEOTools handles data and methodology

I document how each check works, where its data comes from, and which parts are only estimates.

Editorial standards About Contact