跳到正文
Hacker News (official RSS)(verisource.dev)·· 6 天前品質評分47

40 個網站調查:80% 的 Shopify 商店開放 UCP,卻沒有自訂的 /agents.md

40-domain study: 80% of Shopify stores expose /API/ucp/MCP, 0% /agents.md

重點導讀

有些網站已能讓 AI 搜尋工具讀取資料,但缺少網站自己的價格、退貨和服務規則,AI 可能只能從其他地方猜答案。

  • 這項調查檢查了 40 個網站,發現 10 家 Shopify 商店中有 8 家開放 /api/ucp/mcp,卻沒有一家自訂 /agents.md。
  • 其中 6 家使用 Shopify 提供的範本,裡面沒有商店自己的免運門檻和退貨期限。調查也發現,40 個網站都沒有在 /robots.txt 禁止 GPTBot 或 PerplexityBot,但有 2 個網站仍在網路防護系統擋下這些爬蟲。這表示網站看起來允許 AI 讀取,不代表 AI 一定能順利取得資料。
  • 網站也需要清楚寫出自己的重要資訊。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

正文

Custom /llms.txt

50.0%

20/40 curated · 7/40 boilerplate · 3/40 soft-404

Custom /agents.md

5.0%

2/40 custom (vercel.com, linear.app) · 5/40 stock Shopify

Homepage FAQPage Schema

7.5%

3/40 include FAQPage vs. 33/40 (82.5%) with base JSON-LD

Silent Edge WAF 403 Blocks

5.0%

2/40 return HTTP 403 to GPTBot despite 0/40 blocking in /robots.txt

Cross-Vertical Readiness Matrix (N = 40 Live-Probed Public Domains)Probed 2026-11-16 · 6–8 live HTTP requests

Industry CohortSampleCustom /llms.txtAuto BoilerplateCustom /agents.mdBase JSON-LDFAQPage SchemaWAF HTTP 403
B2B SaaS & Developer Tools1010/10 (100%)0/10 (0%)2/10 (20%)9/10 (90%)0/10 (0%)0/10 (0%)
Shopify & D2C Commerce101/10 (10%)6/10 (60%)0/10 (0%)7/10 (70%)0/10 (0%)1/10 (10%)
SEO, Content & Digital Agencies104/10 (40%)1/10 (10%)0/10 (0%)8/10 (80%)2/10 (20%)0/10 (0%)
Multi-Location Healthcare, Legal & Home Services105/10 (50%)0/10 (0%)0/10 (0%)9/10 (90%)1/10 (10%)1/10 (10%)
All Cohorts Combined4020/40 (50.0%)7/40 (17.5%)2/40 (5.0%)33/40 (82.5%)3/40 (7.5%)2/40 (5.0%)

KEY ARCHITECTURAL TAKEAWAYS

Four protocol gaps separating cited brands from invisible domains

80% UCP ACTIVE VS. 0% CUSTOM /AGENTS.MD

The 80-Point Shopify Agent Truth Gap

Eight of ten probed D2C commerce brands (80.0%) expose active Shopify /api/ucp/mcp commerce endpoints, yet 0 of 10 (0.0%) have customized /agents.md and only 1 of 10 (10.0%) serves a custom /llms.txt. Six of ten serve unedited stock Shopify Shop-skill boilerplate lacking free-shipping thresholds, return windows, and bundle exclusions.

0% ROBOTS.TXT BLOCK · 5.0% WAF HTTP 403

Open /robots.txt Files Mask Silent CDN 403 Blocks

Zero of the 40 probed domains (0.0%) disallow GPTBot or PerplexityBot at root inside /robots.txt, yet 2 of 40 (5.0% — ridge.com and westerndental.com) return HTTP 403 Forbidden when probed with real GPTBot or PerplexityBot User-Agent headers at the CDN edge.

50.0% CURATED · 17.5% BOILERPLATE · 7.5% SOFT-404

One-Third of HTTP 200 /llms.txt Responses Are Uncurated or HTML Soft-404s

Thirty domains returned HTTP 200 on GET /llms.txt, but Content-Type and body inspection revealed 3 HTML SPA soft-404 pages (7.5% of the cohort) and 7 auto-generated Shopify or Rank Math boilerplate dumps (17.5%). Developer SaaS leads genuine curation at 100% (10/10), compared with 40% of SEO agencies and 10% of D2C brands.

82.5% BASE JSON-LD · ONLY 7.5% FAQPAGE

Base Organization Schema Is Common; Answer-First FAQPage Schema Is Rare

While 33 of 40 domains (82.5%) render at least one application/ld+json block on their homepage, only 3 of 40 (7.5%) include FAQPage structured Q&A pairs—leaving ChatGPT Search, Gemini, and Perplexity to infer pricing tiers, service areas, and SLAs from third-party directories.

DOMAIN-LEVEL HTTP DATASET · REPRODUCIBLE TELEMETRY

Filter all 40 probed domains by industry vertical

SHOWING 40 OF 40 DOMAINS

DomainVertical/llms.txt/agents.mdHomepage JSON-LDGPTBot / PerplexityBotShopify UCPLive Check
stripe.comB2B SaaSCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
vercel.comB2B SaaSCUSTOM CURATEDCUSTOM ENRICHED3 nodesHTTP 200 OPEN—Re-Probe →
supabase.comB2B SaaSCUSTOM CURATEDMISSING4 nodesHTTP 200 OPEN—Re-Probe →
resend.comB2B SaaSCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
linear.appB2B SaaSCUSTOM CURATEDCUSTOM ENRICHED0 nodesHTTP 200 OPEN—Re-Probe →
cloudflare.comB2B SaaSCUSTOM CURATEDMISSING3 nodesHTTP 200 OPEN—Re-Probe →
posthog.comB2B SaaSCUSTOM CURATEDMISSING2 nodesHTTP 200 OPEN—Re-Probe →
clerk.comB2B SaaSCUSTOM CURATEDMISSING2 nodesHTTP 200 OPEN—Re-Probe →
dub.coB2B SaaSCUSTOM CURATEDMISSING2 nodesHTTP 200 OPEN—Re-Probe →
cal.comB2B SaaSCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
allbirds.comShopify D2CAUTO BOILERPLATESTOCK SHOPIFY0 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
gymshark.comShopify D2CAUTO BOILERPLATESTOCK SHOPIFY0 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
brooklinen.comShopify D2CAUTO BOILERPLATESTOCK SHOPIFY4 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
chubbies.comShopify D2CCUSTOM CURATEDMISSING1 nodeHTTP 200 OPENUCP ACTIVERe-Probe →
ridge.comShopify D2CAUTO BOILERPLATEMISSING0 nodesBOTH WAF 403UNAVAILABLERe-Probe →
cutsclothing.comShopify D2CMISSINGMISSING2 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
ruggable.comShopify D2CHTML SOFT-404MISSING1 nodeHTTP 200 OPENUNAVAILABLERe-Probe →
trueclassictees.comShopify D2CAUTO BOILERPLATESTOCK SHOPIFY3 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
carawayhome.comShopify D2CMISSINGMISSING2 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
beardbrand.comShopify D2CAUTO BOILERPLATESTOCK SHOPIFY2 nodesHTTP 200 OPENUCP ACTIVERe-Probe →
siegemedia.comSEO AgencyMISSINGMISSING2 nodesHTTP 200 OPEN—Re-Probe →
animalz.coSEO AgencyMISSINGMISSING0 nodesHTTP 200 OPEN—Re-Probe →
omniscientdigital.comSEO AgencyMISSINGMISSING1 nodeHTTP 200 OPEN—Re-Probe →
singlegrain.comSEO AgencyCUSTOM CURATEDMISSING3 nodes · FAQPageHTTP 200 OPEN—Re-Probe →
directiveconsulting.comSEO AgencyCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
victorious.comSEO AgencyMISSINGMISSING2 nodes · FAQPageHTTP 200 OPEN—Re-Probe →
webmechanix.comSEO AgencyAUTO BOILERPLATEMISSING5 nodesHTTP 200 OPEN—Re-Probe →
klientboost.comSEO AgencyHTML SOFT-404MISSING0 nodesHTTP 200 OPEN—Re-Probe →
powerdigitalmarketing.comSEO AgencyCUSTOM CURATEDMISSING2 nodesHTTP 200 OPEN—Re-Probe →
webfx.comSEO AgencyCUSTOM CURATEDMISSING3 nodesHTTP 200 OPEN—Re-Probe →
aspendental.comMulti-LocationHTML SOFT-404MISSING2 nodesHTTP 200 OPEN—Re-Probe →
hellotend.comMulti-LocationMISSINGMISSING36 nodesHTTP 200 OPEN—Re-Probe →
asktia.comMulti-LocationMISSINGMISSING1 node · FAQPageHTTP 200 OPEN—Re-Probe →
onemedical.comMulti-LocationMISSINGMISSING0 nodesHTTP 200 OPEN—Re-Probe →
forthepeople.comMulti-LocationCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
rotorooter.comMulti-LocationCUSTOM CURATEDMISSING3 nodesHTTP 200 OPEN—Re-Probe →
servpro.comMulti-LocationCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
mrrooter.comMulti-LocationCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
aireserv.comMulti-LocationCUSTOM CURATEDMISSING1 nodeHTTP 200 OPEN—Re-Probe →
westerndental.comMulti-LocationMISSINGMISSING2 nodesGPTBOT WAF 403—Re-Probe →

METHODOLOGY & REPRODUCIBILITY SPECIFICATION

How every domain was probed and classified

Each of the 40 public domains was probed directly over HTTPS using the same deterministic parser powering Verisource's production scanner: (1) GET /robots.txt parsed for GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, and Google-Extended directives; (2) homepage requests sent with explicit GPTBot/1.2 and PerplexityBot/1.0 User-Agent headers to catch silent CDN HTTP 403 / 429 blocks; (3) GET /llms.txt and GET /agents.md inspected for text/html SPA soft-404s and stock Shopify (shop.app/SKILL.md) or SEO plugin boilerplate; (4) homepage application/ld+json script nodes parsed for FAQPage entities; and (5) JSON-RPC tools/list handshakes tested against /api/ucp/mcp and /api/mcp on D2C commerce stores.

NOVEMBER 2026 ADDENDUM · DEVELOPER INFRASTRUCTURE COHORT (N = 10)

90% of DevTools ship /llms.txt. Only 20% ship /agents.md.

In a follow-up live HTTP probe across 10 developer infrastructure and B2B SaaS domains (trigger.dev, inngest.com, turso.tech, tinybird.co, knock.app, workos.com, stytch.com, upstash.com, highlight.io, and infisical.com), 9 of 10 (90.0%) served a valid /llms.txt directory, yet only 2 of 10 (20.0% — tinybird.co and infisical.com) published a custom /agents.md pricing and limits guardrail file, and only 1 of 10 (10.0% — upstash.com) shipped all 3 JSON-LD pillars.

Protocol / Schema SignalDevTools Cohort (N = 10)Commercial Impact
/llms.txt (HTTP 200)90.0% (9 / 10)High documentation-link adoption across modern DevTools
Custom /agents.md20.0% (2 / 10)80% leave pricing tiers & rate limits unguarded against AI hallucination
Full 3-Pillar JSON-LD10.0% (1 / 10)90% omit FAQPage or Offer pricing nodes on root pages

BENCHMARK FAQ

Common questions about the 2026 AI Search Readiness dataset

What percentage of websites have /llms.txt and /agents.md in November 2026?

In our live HTTP readiness probe across 40 public domains spanning B2B SaaS, Shopify D2C, SEO agencies, and multi-location services, 50.0% (20/40) serve a custom-curated /llms.txt file, another 17.5% (7/40) expose auto-generated Shopify or SEO-plugin boilerplate, 7.5% (3/40) return HTML soft-404s with HTTP 200 status, and only 5.0% (2/40 — vercel.com and linear.app) publish a custom /agents.md specification.

Why do Shopify stores show 80% /api/ucp/mcp adoption but 0% custom /agents.md adoption?

Shopify rolled out platform-level /api/ucp/mcp endpoints and stock /agents.md files pointing to https://shop.app/SKILL.md across modern storefronts, which explains why 8 of 10 probed D2C stores respond on /api/ucp/mcp. However, stock Shopify files omit brand-specific free-shipping thresholds, return windows, and bundle exclusions unless the merchant overrides them via templates/agents.md.liquid or /pages/ai-facts.

Can a domain block GPTBot or PerplexityBot even when /robots.txt allows them?

Yes. Zero of the 40 probed domains disallowed GPTBot or PerplexityBot at root in /robots.txt, yet 5.0% (2/40 — ridge.com and westerndental.com) returned HTTP 403 Forbidden at the CDN/WAF layer when requested with GPTBot or PerplexityBot User-Agent headers. Testing both /robots.txt and simulated crawler User-Agent HTTP status codes is required.

BENCHMARK YOUR OWN DOMAIN IN 60 SECONDS

See how your website stacks up against the 40-domain 2026 readiness cohort.

$0 TODAY · 7-DAY FREE TRIAL

來源: Hacker News (official RSS) (verisource.dev)
開啟原文

讀者討論

理性交流 · 即時討論