Digital marketing term
Indexing
The step after crawling when a search engine stores a page and may show it for queries. Being crawled is not enough if the page never enters the index.
Detailed explanation
The path is roughly discover, crawl, render, then index. robots.txt affects crawling, noindex affects indexing, and a canonical says which URL should count. Sitemaps and internal links speed discovery of new or updated pages.
URL Inspection in Search Console separates “crawled, not indexed” from other failures. Thin, duplicate, blocked JavaScript, or weak quality can keep a page out.
On bilingual sites, each language URL must be indexable for hreflang to work. Thin English copies may be noindexed on purpose so the stronger language is not diluted.
Frequently asked questions
- Why isn’t a page indexed?
- noindex, a canonical to another URL, robots blocks, thin content, or simply not crawled yet are the usual causes.
- How long does indexing take?
- With internal links and a sitemap, hours to days. Newer, weaker sites often wait longer.
Related terms
Internal links for the topic cluster — read these concepts together.
- SEOSEO (Search Engine Optimization) is the set of technical and content practices that help a website rank more visibly in organic search results.
- Robots.txtRobots.txt is a simple text file placed in a website's root directory that tells search engine bots which parts of the site they should and shouldn't crawl.
- SitemapAn XML sitemap is a file listing a site's URLs that helps search engines discover and crawl pages more efficiently, especially on large or complex sites.
- Crawl BudgetCrawl Budget refers to the amount of resource search engine bots allocate to crawling a website's pages within a given time period; it especially affects how promptly important pages get indexed on large-scale sites.
- Canonical URLA Canonical URL is an HTML tag that tells search engines which address should be treated as the "primary" version among multiple pages with identical or very similar content.
- Duplicate ContentThe same or near-identical text living at more than one URL. Search engines then have to pick a version, and visibility splits.