On March 20, 2026, Google added a new entry to its official list of web fetchers. It was not a crawler, nor a training bot. It was an agent. The user agent string, Google-Agent, represents a fundamental shift in how the search giant interacts with the open web, introducing a visitor that behaves more like a human proxy than an indexing machine. This is the digital identity behind AI systems running on Google’s infrastructure, the component that visits websites when a person asks an assistant to research a product, compare prices, or fill out a form. The first product to deploy this technology is Project Mariner, Google’s experimental AI browsing tool. The distinction between this new entity and Googlebot is critical. Googlebot crawls the web continuously to index pages for search, operating on its own schedule. Google-Agent, conversely, only appears when a human explicitly requests it. That single difference rewrites the rules of web governance.<\/p>\n
Google classifies Google-Agent as a user-triggered fetcher. This category, which includes tools like Google Read Aloud, NotebookLM, and Feedfetcher, shares a defining property: a human initiated the request. Google’s position on this class of fetcher is clear and consequential. They generally ignore robots.txt rules because the fetch was requested by a person. The logic stems from a straightforward analogy. If you type a URL directly into Chrome, the browser fetches the page regardless of what robots.txt says. In Google’s view, Google-Agent operates on the same principle. It is the user’s proxy, not an autonomous crawler. This represents a meaningful departure from how OpenAI and Anthropic handle similar traffic. ChatGPT<\/a>aa-User and Claude-User both function as user-triggered fetchers, but they respect robots.txt directives. If you block ChatGPT-User in your robots.txt file, ChatGPT will not fetch your page when a user asks it to browse. Google made a different call, creating a significant gap for website owners who relied on robots.txt as a universal access control mechanism. To restrict access from Google-Agent, you now need server-side authentication or access controls, the same tools you would use to block a human visitor.<\/p>\n A more significant, and largely overlooked, development is buried in a single line of Google’s documentation. Google-Agent is experimenting with the web-bot-auth<\/a> protocol, using the identity The Cryptographic Passport for the Agent Era<\/h2>\n
https:\/\/agent.bot.goog<\/code>codecodecodecode. Web Bot Auth<\/a> is an IETF draft standard that functions as a digital passport for bots. Each agent holds a private key, publishes its public key in a directory, and cryptographically signs every HTTP request it makes. The website receiving the request can then verify the signature and know, with cryptographic certainty, that the visitor is exactly who it claims to be. User agent strings can be spoofed by anyone with a simple script. Web Bot Auth cannot be faked. Google adopting this protocol, even experimentally, signals the direction of agent identity for the entire industry. Major infrastructure providers like Akamai, Cloudflare, and Amazon, through its AgentCore Browser, already support the standard. Google’s involvement brings the critical mass needed for widespread adoption.<\/p>\nWhy Cryptographic Identity Matters Now<\/h2>\n