GOJI
All terms
AI VisibilityReviewed by Mark S.

Crawling and indexing

Crawling is search engines discovering and reading your pages; indexing is them storing those pages so they can appear in results. No crawl, no index; no index, no ranking: everything in SEO sits downstream of these two.

Why it matters

Plenty of "why don't we rank" mysteries are really "we're not indexed" problems: a stray noindex tag, a blocked section, pages the crawler can't reach. These failures are silent: the site looks fine to humans while being partially invisible to engines.

How it works

Crawlers follow links and sitemaps to find URLs, fetch and render the pages, and decide what deserves indexing based on quality, uniqueness and directives you control (robots rules, noindex, canonicals). Search Console shows you exactly what got crawled, indexed, or excluded and why.

What to do about it

Open Google Search Console's page-indexing report. The "excluded" reasons list is your technical SEO to-do list, in priority order, for free.