Home / Blog / 40-domain study: 80% of Shopify stores expose /API/ucp/MCP,…
Tech News

40-domain study: 80% of Shopify stores expose /API/ucp/MCP, 0% /agents.md

A 40-domain benchmark reveals 80 percent of probed Shopify stores expose active agent commerce endpoints while zero publish custom agents.md policy guardrails.

By Dillip Chowdary • Nov 17, 2026 • Source: verisource.dev

40-domain study: 80% of Shopify stores expose /API/ucp/MCP, 0% /agents.md
Illustration · Pexels

At a glance

Study date
November 16, 2026
Sample size
40 public domains across 4 industry verticals
Shopify UCP active
80.0% (8 of 10 probed D2C stores)
Custom agents.md
5.0% across 40 domains (vercel.com, linear.app)
Silent WAF blocks
5.0% (2 of 40 domains return HTTP 403 to GPTBot)
Homepage FAQPage schema
7.5% (3 of 40 domains include Q&A pairs)

Verisource released empirical telemetry examining protocol adoption for artificial intelligence search and autonomous agents across forty public web domains on November 16, 2026. The research evaluated live HTTP responses across enterprise software, retail commerce, digital marketing agencies, and multi-location healthcare providers to measure protocol visibility. According to verisource.dev's report, 82.5 percent of tested sites deploy structured data on their homepages, while only 5.0 percent provide custom operational guardrails for autonomous crawlers.

This analysis details the technical findings across all forty domains, tracking machine-readable directories, edge web application firewall rules, and structured schema declarations. Technical leaders, web architects, and e-commerce operators can use these benchmarks to identify configuration gaps that cause major search models and agents to hallucinate pricing, inventory, and service terms. The breakdown highlights where public implementations succeed and where missing metadata breaks automated execution.

01The eighty-point Shopify agent truth gap

Eight of the ten probed direct-to-consumer brands expose active commerce endpoints at slash api slash ucp slash mcp, representing an 80.0 percent exposure rate. Despite exposing these live transaction paths, zero of the ten merchants published a customized agents markdown file to define operational constraints. Only Chubbies provided a custom curated llms text file, leaving nine storefronts without bespoke machine-readable guidance.

Six out of ten commerce stores rely entirely on unedited stock Shopify platform boilerplate originating from shop dot app skill markdown. This default scaffolding lacks explicit merchant business rules, including free shipping minimum purchase thresholds, standard return windows, and promotional bundle exclusions. Without explicit policy definitions, automated shopping agents interacting with stores like Brooklinen, Allbirds, and Gymshark must infer commercial terms from generic templates.

Two merchants, Cuts Clothing and Caraway Home, omitted both endpoints entirely, while Ruggable returned a single page application soft error. True Classic Tees and Beardbrand exposed the active commerce interface alongside stock documentation, creating an 80-point gap between endpoint access and policy governance. This architectural imbalance allows autonomous clients to attempt catalogue discovery without bounded parameters for transactions.

02Silent edge blocks behind open robots files

None of the forty tested domains disallowed GPTBot or PerplexityBot inside their root robots text files, creating the appearance of open access. However, web application firewalls deployed at the content delivery network edge actively blocked crawlers on 5.0 percent of the monitored properties. Probes carrying authentic crawler user agent headers encountered immediate HTTP 403 Forbidden responses rather than permitted entry.

The silent blocks appeared at Ridge, which blocked both GPTBot and PerplexityBot, and Western Dental, which rejected GPTBot. Because edge firewall rules intercept incoming traffic before server routing takes place, administrative teams inspecting standard robots exclusion files fail to detect crawler exclusion. Both brands remain completely invisible to conversational indexing pipelines while maintaining zero explicit disallow directives within their robots policies.

Note: Live probe testing confirmed that the active commerce endpoint was unavailable on Ridge due to dual edge firewall blocks, while nine other commerce domains kept edge routing open. Web application firewalls configured with automated threat rules frequently flag autonomous search engine user agents without alerting domain administrators to the resulting indexation blackout.

03Uncurated boilerplate and single page application errors

Advertisement

Tech Pulse Daily

Get tomorrow's pulse first

Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.

Industry CohortSampleCustom /llms.txtAuto BoilerplateCustom /agents.mdBase JSON-LDFAQPage SchemaWAF HTTP 403
B2B SaaS & Developer Tools1010/10 (100%)0/10 (0%)2/10 (20%)9/10 (90%)0/10 (0%)0/10 (0%)
Shopify & D2C Commerce101/10 (10%)6/10 (60%)0/10 (0%)7/10 (70%)0/10 (0%)1/10 (10%)
SEO, Content & Digital Agencies104/10 (40%)1/10 (10%)0/10 (0%)8/10 (80%)2/10 (20%)0/10 (0%)
Multi-Location Healthcare & Services105/10 (50%)0/10 (0%)0/10 (0%)9/10 (90%)1/10 (10%)1/10 (10%)
All Cohorts Combined4020/40 (50.0%)7/40 (17.5%)2/40 (5.0%)33/40 (82.5%)3/40 (7.5%)2/40 (5.0%)

Thirty of the forty public domains returned HTTP 200 status codes upon receiving GET requests for the llms text path. Deep payload inspection revealed that 33.3 percent of those successful responses failed to deliver functional documentation. Three domains delivered HTML single page application soft errors, while seven domains served automated boilerplate exports generated by Rank Math or Shopify platforms.

Developer-focused software providers achieved complete compliance, with all ten domains publishing hand-curated documentation indexes. In contrast, only 40.0 percent of digital marketing agencies and 10.0 percent of commerce brands provided customized files. KlientBoost, Ruggable, and Aspen Dental returned HTML landing pages under text content headers, forcing retrieval bots to parse client-side application scripts instead of structured markdown text.

04Base organization entities and rare answer schema

Structured data adoption remains widespread across root domains, with 33 of the 40 organizations rendering at least one linked data script element. Enterprise brands consistently define foundational corporate entities, social media profiles, and technical logos using standardized syntax. However, this implementation rarely extends into conversational question answering formats that assist artificial intelligence search models in extracting direct answers.

Only three domains, accounting for 7.5 percent of the tested cohort, published validated FAQPage structured schemas on their primary landing pages. Single Grain, Victorious, and Ask Tia successfully embedded targeted question and answer pairs within their homepages. The remaining 37 domains omit question-first schema, forcing systems like ChatGPT Search, Gemini, and Perplexity to source pricing tiers and commercial terms from third-party aggregators.

Healthcare and multi-location service providers demonstrate high baseline schema usage, reaching 90.0 percent adoption across nine domains, but Hello Tend rendered 36 separate nodes without a single FAQ structure. While One Medical and Animalz omitted schema entirely, most evaluated sites rely on basic organization nodes. This leaves complex service territories, commercial service level agreements, and cancellation rules completely unrepresented in linked data.

05Five probe checks across developer infrastructure

  1. Robots parsing Evaluating root directives for GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, and Google-Extended crawlers.
  2. Edge verification Transmitting root HTTP requests using explicit GPTBot and PerplexityBot headers to identify firewall rejections.
  3. Payload inspection Analyzing responses on text paths to identify single page application soft errors and unedited template boilerplate.
  4. Schema extraction Reviewing linked data JSON blocks across landing pages to detect answer-first question and answer entities.
  5. Protocol handshake Executing remote procedure call tool listing requests against commerce endpoints to verify active agent tooling.
Protocol / Schema SignalDevTools Cohort (N = 10)Commercial Impact
/llms.txt (HTTP 200)90.0% (9 / 10)High documentation-link adoption across modern DevTools
Custom /agents.md20.0% (2 / 10)80% leave pricing tiers & rate limits unguarded against AI hallucination
Full 3-Pillar JSON-LD10.0% (1 / 10)90% omit FAQPage or Offer pricing nodes on root pages

Verisource validated these protocol checks across an addendum cohort of ten developer infrastructure providers, including Inngest, WorkOS, Stytch, Turso, and Trigger dot dev. While 90.0 percent of these developer platforms published valid llms text documentation, only Tinybird and Infisical provided custom agents markdown files. The remaining 80.0 percent leave API rate limits, computational quotas, and enterprise pricing boundaries vulnerable to automated model hallucination.

Structured schema implementation remained equally restricted across the specialized infrastructure cohort, with only Upstash delivering all three core structured data pillars. Platforms including Knock, Highlight, and Cal dot com maintain extensive technical documentation but neglect machine-readable pricing and policy declarations. These empirical measurements prove that modern software platforms prioritize human-readable documentation directories over structured machine constraints for autonomous agents.

Frequently asked questions

What percentage of websites publish custom llms.txt and agents.md files?

Across 40 public domains probed in November 2026, 50.0 percent publish a custom llms.txt file and only 5.0 percent publish a custom agents.md file. Another 17.5 percent serve auto-generated platform boilerplate, and 7.5 percent return HTML single page application soft errors.

How many Shopify stores expose active commerce protocol endpoints?

Eight out of ten probed Shopify direct-to-consumer brands (80.0 percent) expose active commerce endpoints at slash api slash ucp slash mcp. However, zero of those stores publish a customized agents.md file, and six serve unedited platform boilerplate lacking return policies and shipping thresholds.

Can web application firewalls block AI crawlers when robots.txt is open?

Yes, 5.0 percent of probed domains (Ridge and Western Dental) return HTTP 403 Forbidden to GPTBot or PerplexityBot at the content delivery network edge. Neither domain disallows these crawlers inside their root robots.txt files.

How common is FAQPage structured data on homepages?

While 82.5 percent of probed domains deploy base JSON-LD organization schema on their homepages, only 7.5 percent (Single Grain, Victorious, and Ask Tia) include FAQPage structured data. This forces AI search engines to extract pricing tiers and service terms from third-party directories.

Sources

Dillip Chowdary

Author

Dillip Chowdary

Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.

Related on Tech Bytes

Advertisement

5-min tech signal

Weekday briefing for engineers who skip the noise.

No spam · Unsubscribe anytime

Advertisement

✈️ CareerPilot

Your AI job-search copilot

Match your resume against live Ashby, Greenhouse & Lever openings — fit scores, job-specific resume optimization and email alerts.

Find matching jobs →

Free Tools

Browse all tools →