Systems
Security
Data
Python

PLEXData Engineering for people and agents that act on systems

Permissions, Limits, Evidence & eXecution.

No. 012 Latest feature

Wednesday, 30 September 2026

plexdata.online

012

Published
Author
Dorian Sotpyrc
Reading
6 minutes · Concept
Status
Framework · not yet tested across agents

Concept · Agents and the web

Agent Surfing: Build a Website an AI Agent Can Actually Navigate

More of your visitors are now agents sent by people, and they arrive without the context a person picks up in seconds. Making a site readable for them starts with one plain file and a few old habits done properly.

References

Minimal animated illustration. A small robot agent rides a surfboard over rolling waves. Ahead of it, a coral flag reading AGENT.md flies from a marker float, with a dotted line of sight from the agent to the flag. Buoys labelled evidence, reports and opinion float past on the waves. Along the bottom, five stages light up in turn: discover, orient, navigate, verify, use.

Agent Surfing

Give an agent readable water and a map, and it can pick its own line.

In this article · 6 sections
  1. 01Being found is the easy part
  2. 02Reading the water
  3. 03Leave a map at the door
  4. 04Make the ground match
  5. 05Don’t write to the agent
  6. 06Try it on your own site

Most of the care that goes into a website assumes a person on the other end. Someone who glances at the navigation, reads a headline, scrolls a little and knows within seconds whether they are in the right place. The agent that person sends gets none of that for free. It comes in through whatever link it found, often deep inside the site, carrying a broad instruction and no feel for the place.

That visitor is now common enough to design for. I have started calling the design problem Agent Surfing: making a site easy for an agent to discover, get its bearings in, move around, check, and bring something useful back from.

§ 01Being found is the easy part

SEO still matters. It is how a machine finds your page at all. But it deals with the moment before arrival, and the questions an agent has afterwards are different. What is this site? Is this page the real version or a copy? Who wrote it, and when? Is it a measurement or somebody’s view? Where would I go to check it?

A person answers most of those without noticing. An agent has to work them out from markup, link text and whatever the page says about itself. Every guess costs it steps, and some guesses end with it quoting the wrong page back to the person who sent it.

So the line I keep coming back to is this: SEO helps a machine find your page; Agent Surfing helps an agent understand where it has landed and where it should explore next.

What I am not claiming matters as much. Agent products differ, and they change month to month. I have not measured how any one of them moves through a site. This is about what any careful agent needs, whichever product it is.

§ 02Reading the water

The surfing picture is deliberate. A good surfer does not fight the sea. They read it, pick a line and commit, and they can only do that because the water gives them something to read. A website is either readable water or chop.

When I break down what an agent has to do on a site it has never seen, I get five moves.

Agent Surfing: five moves
MoveWhat the agent needs to knowWhat the site can give it
DiscoverIs there a map, and where is it?A downloadable AGENT.md at a stable, linked address
OrientWhat is this site, and what does it hold?A plain description, the main sections, who runs it
NavigateWhere next?Stable URLs, semantic HTML, link text that says where it goes
VerifyCan I trust this, and where did it come from?Named authors, dates, canonical sources, evidence kept apart from opinion
UseCan I take this back to my user?Downloads it can open, and statements it can cite cleanly

Most sites manage some of these by accident. The work is doing all five on purpose, and the first is the cheapest to fix.

§ 03Leave a map at the door

Start with one file. A short, plain AGENT.md at a stable address, linked from the footer so it can be reached from any page. Think of it as the note you would leave for a capable stranger who has to find their way around without you: what the site is and who runs it, where the authoritative material lives, how the sections relate, and where to look for the questions people usually bring.

AGENT.mdListing 1 · Illustrative example
# Example Field Notes<!-- Illustrative example. Names and paths are placeholders. -->What this is: an independent publication on home-battery systems.Run by: Jane Example. Contact: /aboutLast updated: 2026-09-30 ## Authoritative sources/data/     original measurements, CSV, with method notes/reports/  our reporting, dated, each linking its data/opinion/  our views, labelled as opinion ## How the sections relateOpinion cites reports. Reports cite data. Start at data to verify. ## To investigate furtherPricing question: /reports/tag/pricing, then /data/prices.csvWho wrote this: /about and the byline on each page
17 lines · plain MarkdownIt describes and points. It does not command.

Keep it short enough to read in one pass, and keep it true. A map of last year’s site is worse than no map, because the agent has no reason to doubt it. A community proposal called llms.txt is aimed at a similar problem.5 The name matters less than having one clear file and linking to it.

PLEX keeps its own at plexdata.online/AGENT.md. Every article here also has an Agent MD button that exports the piece as clean Markdown, with its title, author, date and sources, and each of those exports points back to the site map. An agent that starts from a single article can still find its way to the rest of the site.

§ 04Make the ground match the map

A map only helps if the ground matches it, and most of the ground is ordinary web practice that has quietly become more important.

Addresses come first. An agent that saves a link, or cites one to its user, is trusting that address to mean the same thing next week. I am learning this on this site. It is part-way through a migration, and one of the old article addresses still brings in steady traffic. If that address simply broke, every link, bookmark and index entry pointing at it would break with it. Move a page if you have to, but redirect it.

Then structure. Real headings, lists, tables and links carry meaning without anyone seeing the page.6 A layout built from anonymous boxes looks fine to a person and says nothing to a machine. Titles, descriptions and structured data do the same job one level up: they tell an agent what a page is before it has read it.3

Then provenance. Put a name and a date on every page that makes a claim, and say when it changed. Where the same material lives at several addresses, declare which one is canonical, so the agent is not left choosing between near-copies.4 If the evidence is a dataset, publish the dataset with a name and a description, not just a picture of the chart.

The habit I care about most is keeping evidence, reporting and opinion visibly apart. PLEX labels each piece as news, concept, opinion or review, and inside many articles it separates what is confirmed from what a vendor claims and what is still open. That labelling was written for human readers. It turns out to be exactly what an agent needs to report a finding honestly, as a measurement or as somebody’s view.

The older plumbing still counts. A robots.txt file and a sitemap are still how many crawlers find their way in at all.12

§ 05Don’t write to the agent

There are two tempting wrong turns.

The first is a second website for machines: a stripped-down copy that drifts away from the real one until you are maintaining two sources of truth and hoping they agree. Make the one site legible instead.

The second is filling pages with instructions aimed at AI systems. Hidden or pushy text telling an agent what to do looks exactly like the prompt-injection attacks agent builders defend against, and a well-built agent should ignore it. It also runs against the argument in Stop Trusting the Agent: an agent’s authority should come from its user and its permissions, not from whatever it happened to read along the way. An AGENT.md should describe and point. It should never give orders.

§ 06Try it on your own site

You do not need a lab for a first look. Give an agent you already use a broad job about your own site, something like “find out what this site says about pricing and come back with sources”. Then read what it did, not only what it said. Did it find the map, or wander? Did it describe the site the way you would? Did it keep what you measured apart from what you think? Would you be happy to see its links and dates quoted back to you?

Each fumble points at something specific to fix. I have not run this across agent products for this piece, so there are no results here, only what I would look for.

If I had one afternoon, I would write the AGENT.md, put a name and a date on every page missing them, and make sure nobody, human or agent, has to guess whether a page is evidence or opinion. None of that is new work. The visitor is new.

§References

  1. 1IETF — RFC 9309: Robots Exclusion Protocolrfc-editor.org
  2. 2sitemaps.org — Sitemaps protocolsitemaps.org
  3. 3schema.org — structured data vocabularyschema.org
  4. 4IETF — RFC 6596: The Canonical Link Relationrfc-editor.org
  5. 5llms.txt — a community proposal (check current status before citing)llmstxt.org
  6. 6WHATWG — HTML Living Standardhtml.spec.whatwg.org