Skip to content
Tityra
Login
Start for free
HomeAboutPricingWho It's For
  • Publishers & Media
  • Ecommerce & Retail
  • SaaS & Technology
  • Agencies & Partners
Blog
info@tityra.com
XLinkedIn
  1. Home
  2. /
  3. Blog
  4. /
  5. Content access

Why can't AI agents read
your website?

A page can look complete in a browser while an agent receives a challenge, an empty shell or content without its context. Compare the initial response and the rendered page to find where the problem starts.

3 min read

Doğu Abaris · 29 October 2026

Read the article
Get your free audit

UNDERSTAND AGENT ACCESS

Learn what agents receive from your website.

FOLLOW THE EVIDENCE

See how observations become useful findings.

PUT GUIDANCE INTO PRACTICE

Find focused changes your team can investigate.

Read the article

In this article

One URL can return different viewsFirst, separate access policy from delivery failureCheck whether JavaScript carries the answerReadable text still needs a structureRetest the same path
How we audit
Table of Contents
One URL can return different viewsFirst, separate access policy from delivery failureCheck whether JavaScript carries the answerReadable text still needs a structureRetest the same path
How we audit

Check the response an agent actually receives before changing the words on the page.

Copy linkOne URL can return different views

Your browser combines the server response, JavaScript, network requests and stored preferences into a page. An agent may use a plain HTTP fetch, a search result or a browser with different capabilities. The view you recognise is only one of those paths.

Start with a small, authorised sample that includes the homepage, a service or product page, and a page with important supporting information. Record the URL, time, status, response headers and retrieval method. Compare the initial HTML with the rendered page under the same conditions, then check whether any differences affect the task.

Copy linkFirst, separate access policy from delivery failure

Read the applicable robots.txt rules and examine the actual response. A crawl rule tells cooperating crawlers what they may request. A CDN challenge or 403 response controls what the client receives. Check both when a resource you mean to publish is hard to retrieve.

Record who needs access, which resource was blocked and what the client received. Check CDN rules, bot protection and server logs with the website owner. Adjust the rules for the public resources you want people to find and keep protections in place for private areas.

Illustrative access finding

Observation
A public service URL returns a browser challenge instead of the service description to the tested client.
Next step
Review the edge rule for that route, preserve protections on private paths, and repeat the same request after the change.

Copy linkCheck whether JavaScript carries the answer

A successful HTTP response can still contain little useful content. If the initial HTML only has a loading container, a client that doesn't run JavaScript will miss the service description. A client with a browser can still run into failed requests, consent screens or a page that hasn't finished loading.

Compare the content a reader needs in each view. Is the service explanation present? Can a reader identify prices, conditions and the next step? Where appropriate, render essential public content on the server or generate it at build time. The initial document can carry the main answer while JavaScript handles the interaction.

Illustrative content finding

Observation
The rendered page shows the service description, but the initial HTML contains only an application shell.
Next step
Include the essential description and its heading in the initial HTML, then verify it without JavaScript and in the normal browser flow.

Copy linkReadable text still needs a structure

Once content is available, examine how its meaning is expressed. A visual heading made from a generic container may not communicate the same outline as a heading element. A product comparison can lose its meaning when cells have no clear relationship to row and column headings.

Use descriptive headings, appropriate landmarks and semantic tables where they fit the content. Use structured data to clarify facts the page already provides. Record missing content separately from unclear structure so the team can fix each cause.

Collect evidence for an Agent Experience audit

Copy linkRetest the same path

Keep the observation, expected behaviour, affected resources and verification steps together. After a change, repeat the recorded retrieval method and check the page in a browser. Record which templates and clients the retest covered.

The examples use fictional pages to show how to investigate an access or delivery problem. In Tityra, the report records what the audit observed and the conditions it covered. Individually requested reports receive staff quality review before release. Included weekly AX Audit reports are delivered automatically after the required checks.

Copy linkSources and further reading

  1. AgentReady essentials
  2. Google: JavaScript rendering and crawling
Doğu Abaris

About the author

Doğu Abaris

Doğu Abaris is a co-founder of Tityra and a software and data engineer. An open-source contributor to Wikimedia and MediaWiki projects since 2011, he writes about agent access, semantic structure and evidence-led website audits.

More about Doğu

See what your website delivers.

Choose an agreed scope and get findings your team can act on.

Get your free audit
Share this article

Share this article

Keep exploring

Audit practice

Where an AX Audit can save your team time

An AI assistant can miss a detail that looks obvious on your website. An AX Audit helps you trace gaps in access, structure and context to the pages, templates and publishing processes your team can fix.

Read article

Audit practice

How to audit a website for Agent Experience

An AX Audit checks whether a client can retrieve the information needed for a task. Use the recorded observations to specify a change your team can verify.

Agent Experience

SEO vs AEO vs GEO vs AX

SEO, AEO and GEO focus on how people find your content in search and AI answers. Agent Experience (AX) looks at what an agent can retrieve, understand and use for a task.

Turn findings into
clear next steps.

Get your free AX Audit
BlogAudit methodologyDocsCLIContact

Digital Domain Technologies Ltd, trading as CogNovaMX

33 Merchant Way
York, United Kingdom, YO23 3TS
info@tityra.com
HomeAbout AXPricing
Who It's For
  • Publishers & Media
  • Ecommerce & Retail
  • SaaS & Technology
  • Agencies & Partners

© 2026 Digital Domain Technologies Ltd, trading as CogNovaMX. Company 03651124.

PrivacyTerms and ConditionsCookie informationCookie preferences

Request an AX Audit

Review the access, structure and context of your website. Reports are reviewed by a member of our team before delivery. Track your request and access your report in Tityra.

Selected package
£100
per 100 pages, excluding VAT. Max 500 pages
How are pages counted??Based on the pages included in your website scope, regardless of report length.

Automated analysis, reviewed by a member of our team before delivery.

  • Automated content analysis
  • Documented findings
  • Practical recommendations

What happens next

  1. 01
    Share your website

    Tell us which website and pages you would like reviewed.

  2. 02
    Confirm the scope

    We review your request and confirm the scope before work begins.

  3. 03
    Receive your report

    We'll email you when your report is ready.

Your audit details

Share your contact details and the website you want audited.

We value your privacy

By clicking “Accept all cookies”, you agree to the storing of optional analytics cookies on your device to help us understand site usage and improve our website. You may click the customise button to choose which optional cookies you would like to allow.

Strictly necessary (2)

We save your cookie choice in this browser for six months, so you do not have to choose on every visit. This storage is always on.

Show detailsHide details for strictly necessary
  • Name
    tityra_analytics_consent_v1
    Provider
    Tityra (tityra.com)
    Type
    HTML local storage
    Maximum storage duration
    Persistent

    Remembers whether you accepted or rejected analytics. Your choice is valid for 180 days (six months), after which we ask again. The stored record remains until it is replaced or you clear your browser storage.

  • Name
    tityra_analytics_consent_v1_refusal
    Provider
    Tityra (tityra.com)
    Type
    HTML session storage
    Maximum storage duration
    Browser tab session

    Keeps analytics off if your browser cannot save a rejection in local storage. This fallback is only used when needed and is cleared when a subsequent choice is successfully saved.

Google Analytics helps us understand visits, interest in audit packages and form progress. It only loads after you accept. Advertising features are disabled.

Show detailsHide details for analytics
  • Name
    _ga
    Provider
    Google Analytics (tityra.com)
    Type
    HTTP cookie
    Maximum storage duration
    Up to 180 days (six months)

    Distinguishes visitors to help us understand how the website is used. Set only after you accept analytics; advertising features are disabled.

  • Name
    _ga_*
    Provider
    Google Analytics (tityra.com)
    Type
    HTTP cookie
    Maximum storage duration
    Up to 180 days (six months)

    Maintains analytics session information, such as visits and interactions with the website. Set only after you accept analytics. The asterisk represents the identifier in the cookie name.

You control whether optional analytics runs on this website. We remember your choice for six months, and you can change it at any time using Cookie preferences in the footer.

Rejecting or withdrawing consent stops analytics and removes the Google Analytics cookies this website can access.

The details list the name, provider, purpose, type and storage duration of each entry. Contact details and text entered into our forms are not included in analytics.

Counts describe the entries listed, including the fallback and cookie name pattern, rather than cookies currently stored on your device.

Read our cookie information