The Crawl module analyzes many publicly available pages — HTTP statuses, links, redirects, metadata, headings and selected Performance, SEO, Accessibility, Ecology and Security signals. It shows problem scale and recurring errors — not a full enterprise crawler or a guarantee every URL will be found.
A single URL analysis shows what happens on one page. Crawl shows whether a problem is isolated or systemic.
The same empty meta on dozens of pages, a redirect chain in the menu or missing H1 in a blog template — Crawl turns that into a priority map. It does not replace detailed modules or manual post-deploy QA.
Four steps from an address list to deciding which pages to fix first — with no full Googlebot simulation promise.
Publicly available pages found via links, sitemap or your URL list.
HTTP codes, redirects, redirect chains and broken internal links.
Recurring SEO, accessibility and technical issues in templates.
Crawl finds publicly available URLs via internal links, sitemap or your provided list. Pages behind login or without links may not enter the analysis.
HTTP codes, redirects, redirect chains and internal links leading to errors are checked. You see where users or bots hit 404s or unnecessary redirects.
Recurring SEO, accessibility and technical issues are grouped — e.g. the same missing meta in a category template or contrast in a form component across many pages.
A list of URLs with the highest issue count or weight. Homepage, offer and contact form may outrank an old blog post — with no automatic repair.
Live progress
Live crawl progress: how many URLs were discovered, how many analyzed, what sits in the queue. A large site does not vanish into a black box for fifteen minutes — you see the analysis actually working.
/blog/site-performance-auditCrawl scope
You set the scope deliberately: the whole site, a section (e.g. /blog) or a single page — with sources from links, the sitemap and a manual list. Credit cost follows the scope, so you do not pay for a crawl you do not need.
Errors and verification
Every broken URL has a re-verification button: after the fix you check a single link without re-running the whole crawl — and see "Fixed" or "Still broken" right away.
/blog/old-post 404 Still broken/products/archive 301→200 Fixed/docs/api-v1 404 Check againCrawl comparison
Two crawls side by side: new errors, fixed URLs, status changes. After a deploy you know whether fixes worked across the whole site — not just on the page you checked by hand.
No. Crawl analyzes publicly available URLs it can discover via links, sitemap or your list. Pages behind login, forms, dynamic JS or without links may not be included.
No. Crawl in Insight focuses on the technical site picture and recurring issues within product modules. It does not replace enterprise tools or full manual inventory.
The SEO module analyzes one URL in depth. Crawl shows whether the same issue repeats across many pages and how many URLs need attention — a site map, not just one page.
No. Crawl covers technical site basics — statuses, links, metadata, indexability. Rank tracking and SERP visibility are a separate module when available on your plan.
Crawl answers: what does the state of many pages look like after running an analysis. Monitoring watches whether something got worse over time between measurements — alerts and regressions, not a full site map.
One scan shows what is worth fixing. No card, no commitment — results in about 2 minutes.