The Technical Layer

This is the shortest chapter in Part III because the technical layer has a specific, bounded job: nothing here wins you answers, but any of it can silently take you out of them. The best content, entity and corroboration work in the world does nothing if the machines can't fetch it. Think of this chapter as the pre-flight checklist.

The fundamentals haven't moved

Because retrieval runs on search indices, everything that mattered for classic technical SEO still matters: pages indexed, an accurate sitemap, sensible canonicals so the engines know which version of a page is the real one, fast load times, and a site that works on a phone. If you have an SEO practitioner, this is their home ground and nothing here replaces it. What follows is the layer answer engines add on top.

Check who you're blocking

Your robots.txt file tells crawlers what they may fetch, and AI crawlers identify themselves there: GPTBot and OAI-SearchBot for OpenAI, PerplexityBot, Google-Extended for Google's AI training, ClaudeBot for Anthropic, and others. Over the past few years a wave of advice told businesses to block them all on principle, and plenty of CDN and security products now block AI crawlers by default without asking. Here's the thing to understand before deciding: blocking the crawlers that feed answers removes you from the answers. For a publisher whose content is the product, blocking might be a defensible trade. For a business whose website exists to win customers, blocking the engines your buyers ask is closing the shop to the street your customers moved to. Decide deliberately, per crawler, and then audit that your CDN, firewall, or security plugin isn't overruling you silently. This misconfiguration is common, invisible from the outside, and fully explains some businesses' absence from entire engines.

Make the content reachable without gymnastics

The fetchers answer engines send out are simpler than a full browser, and some of them execute little or no JavaScript. A site that renders its content only after scripts run can look complete to a human and nearly empty to a machine. The test takes one minute: view your page's source, or fetch it with a tool that doesn't run scripts, and check whether your actual words are in the HTML. If they're not, talk to your developer about server-side rendering. While you're in there: real headings marked up as headings rather than styled text, dates on content that's dated, and text as text rather than baked into images. None of this is exotic. It's the difference between handing the machine a document and handing it a puzzle.

The small additions

Two cheap items round out the layer. An llms.txt file at your site root, a plain-text page describing who you are, what you do, and where your key pages live, written for AI systems the way robots.txt is written for crawlers. It's an emerging convention rather than a guarantee, but it costs an hour and states your canonical facts in the most machine-friendly format that exists. And visible freshness: last-updated dates on your important pages, kept true. Engines prefer sources that look maintained, and a site whose newest content is three years old reads as abandoned even when the business is thriving.

Run this checklist once, properly, then re-check quarterly, because platforms update, plugins change defaults, and blocks reappear. That's Part III complete: liftable content, a coherent entity, corroboration, question-answer pairs with their markup, and a technical layer that lets machines at all of it. What remains is making the whole thing run without you thinking about it every day, which is Part IV: the operation.