Technical GEO | Growth

This is entry 9/20 in Growth’s GEO whitepaper series — Ch.6, Technical GEO. You can find the full table of contents and the complete PDF on the whitepaper page.

Answer-First: Technical GEO consists of three things: adopting llms.txt, applying schema markup across the board, and strategically allowing AI crawlers. According to HtmlRAG research, web pages that preserve HTML structure show meaningfully better RAG performance than plain text — the technical foundations of traditional SEO remain just as valid for AI citation. Without that technical foundation, even the best content never enters an AI’s field of view.

Why technical infrastructure comes first

Many brand marketing leads ask, “Can’t we just improve the content first?” That’s a fair instinct, but order matters. To continue the metaphor from the GEO three-axis strategy: without a road, even the best car never reaches its destination. If an AI crawler can’t properly read your content when it visits your site, every content strategy and every external-mention effort that follows loses much of its effect.

Allen AI and University of Washington’s OpenScholar research (2024) backs this up. Even large models like GPT-4o, when generating answers to scientific questions from their own internal knowledge without structured sources, hallucinated 78–90% of their citations. By contrast, OpenScholar — which retrieves and references real sources using retrieval-augmented generation (RAG) — matched human-expert-level citation accuracy. Making your content “structurally readable” to AI is a precondition for citation.

Pan et al.’s knowledge-graph roadmap research (IEEE TKDE, 2024) similarly concludes that structured data reduces LLM hallucination and strengthens entity recognition. For AI to recognize the information your website provides as a clear entity, that information needs to be structured in a machine-readable form. That’s the essence of the technical axis of GEO strategy — Technical GEO.

Let’s walk through Technical GEO’s three pillars one at a time. You can see where this technical system fits within the whitepaper as a whole on the GEO whitepaper page.

Technical GEO is a system in which llms.txt, schema markup, and AI crawler strategy operate as one body.

llms.txt — a manual for AI, about your website

llms.txt is a standardized website-description file for AI agents and LLMs that emerged in the second half of 2024. If robots.txt is a file that tells search engine crawlers “where you’re allowed to crawl,” llms.txt is a file that systematically summarizes for AI “what this website is, what information it offers, and where the core content lives.”

llms.txt is a summary guide that helps AI quickly grasp a site’s identity and core content.

Why does this matter? An AI crawler doesn’t read every one of the thousands of pages on a website it visits. Within a limited context window, it needs to quickly grasp the most essential information. llms.txt plays exactly that “summary guide” role — like handing a new employee a company introduction packet. Adoption is moving fast, too. When SE Ranking analyzed roughly 300,000 domains in late 2025, about 10% already had llms.txt in place, with tech companies like Anthropic, Stripe, and Cloudflare leading adoption.

llms.txt is typically maintained in two versions. The summary version (llms.txt) concisely lays out the site’s identity, core services, and links to its main content. The full version (llms-full.txt) is an expanded edition that describes every service and piece of content in detail. AI agents usually read the summary version first and consult the full version only when they need deeper information.

Here’s a sample llms.txt for a hypothetical B2B marketing agency.

# Growth Marketing Agency

> Growth Marketing Agency is a Seoul, South Korea-based
> B2B digital marketing agency specializing in
> SEO/GEO strategy consulting and content marketing.

## Core Services

- [GEO Consulting](/service/geo-consulting): AI search visibility audits and strategy
- [Content Marketing](/service/content-marketing): B2B expert content planning and production
- [Technical SEO](/service/technical-seo): Website structure optimization and schema implementation

## Expert Content

- [The Complete GEO Guide](/geo-guide): A comprehensive GEO strategy whitepaper (2026)
- [Marketing Insights Blog](/insight): Weekly industry analysis and case studies
- [Case Studies](/case): Client campaign results and ROI analysis

## Contact

- Email: contact@example.com
- Phone: +82-2-1234-5678

Let’s walk through what each element does in this example. The # heading on the first line is the organization’s official name, and it becomes the base unit AI uses to identify the entity. The > blockquote summarizes the organization’s identity in one to three sentences, and it’s what AI uses to immediately grasp “what this site is.” The ## sections group core services, content, and contact info into categories, and the - [link name](URL): description format lets AI understand each page’s purpose in natural language. llms.txt is written in Markdown, because AI models parse Markdown best.

Schema markup — structuring your brand with JSON-LD

Schema markup is a technique for converting a web page’s information into machine-readable structured data. In particular, the JSON-LD (JavaScript Object Notation for Linked Data) format has become the standard, and both Google and the major AI engines recommend it.

JSON-LD schema lets AI read brand information as structured fact rather than something it has to infer.

Why has JSON-LD become more important in the AI era? In traditional SEO, schema markup was treated as little more than “a bonus for earning rich results.” But in the age of AI search, its role has fundamentally changed. When AI reads the text on a web page, it has to infer whether “this text is a company name, a service description, or an address.” JSON-LD skips that inference step entirely — it explicitly states, “this is an Organization, its name is X, and its address is Y.”

BrightEdge, tracking AI Overview (AIO) citations for 16 months, found that more than half (about 54%) of AIO-cited pages overlap with pages that already rank in traditional organic search. That means the technical foundations of traditional SEO — structured data, fast load times, mobile optimization — remain just as valid for AI citation. Technical GEO’s approach isn’t to discard your existing SEO assets; it’s to add an AI-friendly layer on top of them.

Here’s a sample Organization schema to apply to a company website.

{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Growth Marketing Agency",
  "url": "https://www.example.com",
  "logo": "https://www.example.com/logo.png",
  "description": "A B2B digital marketing agency specializing in GEO, SEO, and content marketing.",
  "foundingDate": "2018",
  "address": {
    "@type": "PostalAddress",
    "addressLocality": "Seoul",
    "addressCountry": "KR"
  },
  "sameAs": [
    "https://www.linkedin.com/company/example",
    "https://en.wikipedia.org/wiki/Example"
  ]
}

Let’s look at what role each field plays for AI. @type: Organization declares to AI that this data isn’t plain text — it’s a single organizational entity. name is the entity’s official name, and it’s the anchor AI uses to link the same name found in other sources back to this same entity. description is how AI figures out “what this organization actually does,” raising the odds it gets matched as an answer source for related questions. sameAs is one of the most important fields — it links this entity to authoritative external sources like LinkedIn and Wikipedia, giving AI a trust signal during cross-verification: “this organization is also listed on Wikipedia, and it also has an official LinkedIn page.”

Beyond Organization schema, it’s a good idea to apply Article, FAQPage, and HowTo schema to content pages, and Service and Product schema to service pages. As the knowledge-graph-based RAG research covered in our Entity Authority article demonstrates, graph-structured knowledge representation substantially boosts AI’s comprehension. Schema markup is the practical, hands-on implementation of that structuring.

AI crawler strategy — “allow” should be the default

The most common mistake in AI crawler strategy is a reflexive “let’s just block it for now” approach. That’s a holdover from the defensive mindset of the SEO era. In the GEO era, “allow” needs to be the default, and “block” the exception.

In the GEO era, the default crawler policy is to keep public content open and block only areas that genuinely need protection.

The first step is understanding the major AI crawlers and what they do. GPTBot is OpenAI’s crawler, used to collect training data for GPT-series generative AI models (appearing in ChatGPT’s search results is handled by a separate crawler, OAI-SearchBot). ClaudeBot is Anthropic’s crawler, used to collect public data to improve Claude models, while search citations are handled by Claude-SearchBot. PerplexityBot is a crawler used to surface and link to your website in Perplexity’s search results, and it isn’t used for model training. GoogleOther is a general-purpose crawler Google’s product teams use to collect public content for various purposes, including internal R&D; whether your content can be used to train Gemini models is controlled separately via the Google-Extended token. Bytespider is ByteDance’s (TikTok’s parent company) AI training crawler, and CCBot is the crawler for the Common Crawl project, which underpins the pretraining datasets of most major LLMs.

What happens if you block these crawlers? Your content drops out of AI training data and gets excluded from real-time search as well. When AI receives a related question, it cites a competitor’s content instead of yours. It’s the equivalent of blockading your own road.

Naver’s case is a cautionary tale worth noting. Naver Blog’s robots.txt declares that it “strictly prohibits bot access for AI training and retrieval-augmented generation (RAG) purposes,” explicitly blocking GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, CCBot, and others (as confirmed in June 2026). As a result, the vast trove of Korean-language content on Naver Blog and Naver Knowledge iN is effectively excluded from ChatGPT’s and Perplexity’s answers. For Naver, this was presumably a strategic move to protect its own AI service (AI Briefing) — but if an ordinary company copies this blocking strategy, it’s effectively writing itself out of the age of AI search.

Setting robots.txt to allow AI crawlers is simple. Doing nothing already defaults to allow, but it’s better to state that intent explicitly. That said, areas that genuinely need protection — paid content, customer data pages — should be selectively blocked, while marketing content and public pages must stay open. The criteria for deciding what to block or allow by crawler, and the practical steps for configuring robots.txt, are covered in hands-on detail in our separate AI crawler robots.txt guide.

[Checklist] The 20-item Technical GEO Audit

We’ve organized a checklist for auditing your Technical GEO setup into 4 categories and 20 items. We recommend your IT/development team and SEO team review it together.

The Technical GEO Audit splits four areas into five items each, letting you quickly check your technical foundation.

A. llms.txt and AI accessibility (5 items)

Does an llms.txt file exist at the root directory (example.com/llms.txt)?
Do you also provide llms-full.txt (the detailed version)?
Does llms.txt include the organization’s name, core services, and links to flagship content?
Does llms.txt’s content match your actual site structure and services, and was it updated within the last 3 months?
Does llms.txt include contact information and links to authoritative external sources (Wikipedia, LinkedIn, etc.)?

B. Schema markup and structured data (5 items)

Is Organization schema implemented on the main page in JSON-LD format?
Is Article schema applied to all blog/content pages?
Is FAQPage schema applied to FAQ content, and HowTo schema applied to how-to content?
Is the sameAs field linked to your official social media profiles and authoritative external sources?
Does Google’s Rich Results Test show zero structured-data errors?

C. AI crawler settings (5 items)

Are GPTBot, ClaudeBot, and PerplexityBot left unblocked in robots.txt?
Are GoogleOther and Google-Extended left unblocked?
Is CCBot (Common Crawl) allowed, at minimum, on your major public pages?
Are you selectively blocking only paid content and customer data pages?
Does the X-Robots-Tag HTTP header have no noindex directive targeting AI crawlers?

D. Website technical foundation (5 items)

Do all pages meet Google’s “Good” threshold on Core Web Vitals (LCP, CLS, INP)?
Is mobile-friendly responsive design applied?
Is your XML sitemap up to date and does it include all public pages?
Is HTTPS applied across every page with no mixed-content warnings?
Are page titles (H1) and meta descriptions written to match question-style search intent?

Who owns this: the role of the IT/development team

Technical GEO runs as a collaboration where the IT/development team leads execution and the SEO team defines requirements. Writing llms.txt, implementing schema markup, and configuring crawler settings are all areas that require technical execution. That said, strategic calls — like “what information goes into llms.txt” or “which schema to apply first” — need to be made together with the brand marketing and content teams. See the RACI matrix in our GEO organizational design article for how Technical GEO responsibilities split across departments.

Checking schema markup runs from declaring the organizational entity through content-type-specific structuring to verifying zero errors.

Technical GEO only works when the IT/development team’s implementation and the SEO/brand/content teams’ strategic judgment move together.

Auditing llms.txt means checking not just whether the file exists, but also site structure and links to authoritative external sources.

Key Takeaway

Technical GEO is the technical precondition for every GEO strategy. Build it before content or off-site mention efforts.
llms.txt is the “company introduction” you hand to AI. Adoption is spreading fast right now — write yours before your competitors do.
JSON-LD schema markup lets AI recognize your information as fact, not something it has to infer.
For AI crawlers, “allow is the default, block is the exception.” Blocking everything the way Naver does writes you out of AI search.
Use the 20-item checklist to immediately audit the current state of your technical infrastructure.

If you’re curious how your brand currently shows up in AI answers, reach out for an AI answer-share audit. You can also request the complete GEO whitepaper PDF.

Frequently Asked Questions (FAQ)

How is llms.txt different from robots.txt?

Where robots.txt tells crawlers “what scope you’re allowed to collect,” llms.txt is a Markdown guide that summarizes for AI “what this site is and where its core content lives.” The two files aren’t substitutes for each other — they’re complementary.

What schema markup should you apply first?

Organization schema (JSON-LD) on your main page is the starting point. After linking authoritative external sources like Wikipedia and LinkedIn via the sameAs attribute, extend out to Article, FAQPage, and HowTo schema on content pages, and Service and Product schema on service pages.

What happens if you block AI crawlers?

Your content gets excluded from both AI training data and real-time search, and AI cites a competitor’s content instead when answering related questions. Unless you have a strategic reason for a blanket block like Naver’s, the general principle is to selectively block only pages that need protection while leaving marketing content open.

Who should own Technical GEO?

The standard model is a collaboration where the IT/development team leads implementation and the SEO team defines requirements. Strategic calls — like what information to put in llms.txt or which schema to prioritize — need input from the brand marketing and content teams as well.

GEO whitepaper series: ← Previous chapter · Full table of contents · Next chapter →