Technical SEO Checklist: Indexing, Sitemaps and Canonical URLs

Begin with a page a crawler can reach

Technical SEO helps search engines reach and understand useful content. Start with a page that answers a clear question, then check whether its public URL works without an account. Google lists crawl access, a successful HTTP response and indexable content among its technical requirements. Meeting them makes a page eligible; it does not guarantee inclusion or ranking.

Inspect the response rather than judging from a screenshot. A page can show a useful-looking error while returning 200, or render normally for you because you have a login cookie. Test an article and a deep link from a fresh session, not just the homepage.

Keep crawl rules and indexing directives distinct

Robots.txt tells cooperating crawlers which paths they may request. A noindex directive tells a supported search engine not to include a page in results once it can read the directive. Blocking a page in robots.txt can prevent the crawler from seeing its noindex instruction. Neither mechanism protects confidential information from an ordinary visitor.

Keep drafts and private areas behind real access control. On public pages intended for search, check for an accidental noindex header or metadata tag inherited from staging. If an edge service challenges every bot, verify that legitimate search crawlers can still reach the content. Do not disable security wholesale to fix a crawl problem.

Choose a preferred URL consistently

A canonical URL identifies the preferred version when similar pages have multiple addresses. Check that the page's canonical, internal links, sitemap and redirects agree about HTTPS, hostname and path. A canonical is a signal, not an instruction that guarantees Google's chosen URL.

For example, if an article is available with tracking parameters, keep its canonical on the clean article address. If a page has genuinely moved, use an appropriate redirect and update links rather than pointing every retired URL to the homepage. Check that the canonical destination returns useful content and is itself indexable.

Make the sitemap and navigation useful

Include the canonical public pages you want discovered. Do not add login screens, broken routes or every possible filtered view simply to make the file larger. Google supports several sitemap formats; validate the actual output and ensure listed URLs use the preferred hostname.

Use truthful modification dates. A new deployment does not mean every article changed. Link each guide from a relevant directory or topic page, and add related reading that helps the next step. A sitemap complements those links; it does not replace a navigable site.

Check results in Search Console

After verifying ownership, submit the sitemap and inspect representative URLs. Review crawl failures, excluded pages and the canonical Google selected. Search Console reports are observations, not a command to index a page immediately. A low-traffic site may also take time to accumulate useful performance data.

Keep a small recurring review: inspect one new article, one established article and the main directories after significant changes. Fix misleading titles, broken links and unanswered reader questions as well as technical errors. Discovery works best when a page is both accessible and worth reading; no checklist can manufacture demand or instant rankings.

Record changes and dates so that an indexing problem can be compared with a real release rather than a guess about what changed.