Service eligibility and regional restrictions
PuppyIP serves only compliant overseas businesses and their authorized personnel. Proxy services are not available in mainland China. The service may only be used for lawful business activities outside mainland China. Use of this service within mainland China is prohibited.
Hosting a proxy IP or server overseas does not change these restrictions. The service must not be provided to end users in mainland China through relaying, forwarding, sharing or resale. Before use, read the Terms of Service.
Key Takeaways
- On August 28, 2026, Shopify announced that merchants can authorize custom crawlers, scripts and auditing tools to access Online Store through Web Bot Auth HTTP signatures.
- Every request must include Signature-Input, Signature and Signature-Agent together. Changing User-Agent alone does not create verified traffic.
- Merchant-generated signatures bind to one connected domain and last at most three months. They cannot be renewed; create a new signature and update the tool.
- Signatures support online-storefront analysis, not checkout access. Ordinary search-engine and large-language-model indexing does not require merchant signatures.
- A 403, 429 or challenge page alone does not prove an IP issue. Compare small batches without a signature, with the current signature and with a new signature; stop and preserve evidence if failure persists.
A scan worked yesterday but now stops at 403 or 429
A team uses Screaming Frog, custom scripts or third-party auditing tools to inspect product pages. The homepage opens intermittently, but bulk requests get 403, 429, CAPTCHA or local-rate-limit responses. Similar symptoms can come from unsigned traffic, malformed signatures, domain mismatch, expired signatures or entry into checkout outside signature scope. Costs include incomplete audits, false monitoring alarms and wasted crawl windows. The wrong instinct is repeatedly changing User-Agent, increasing concurrency or rotating IPs.
First stop expanding the scan and save one failed URL, time, response status and the headers the tool actually sent. Confirm Signature-Input, Signature and Signature-Agent are all present, then check signature name, domain and expiry in Shopify Admin under Online Store > Preferences > Crawler access.
The new entry point establishes identity, not unlimited access
Shopify’s August 28 update exposes Web Bot Auth to merchants: owned crawlers, scripts and tools can use HTTP signatures to prove requests come from authorized sources. The developer update also says unsigned bots and agents face stricter Storefront and online-store rate limits. Valid signatures support a higher access tier, not removal of every rate limit.
Older approaches often guessed platform trust through User-Agent, source IP or reduced request rates. The new approach adds verifiable identity. It does not change robots rules, app permissions, Storefront API tokens or checkout security, nor guarantee that every third-party tool supports all three headers.
Determine whether Web Bot Auth is needed
Applicable tasks include SEO and accessibility audits, automated testing, data analysis and other merchant-authorized automated access to owned storefronts. Merchants can generate signatures for connected domains in Crawler access and configure the three values in tools supporting custom headers.
Ordinary search engines and large language models do not require merchants to generate signatures for indexing. Signatures also do not allow checkout access. If the failed URL reaches checkout or requires login, return to the appropriate Checkout, app or account-permission flow rather than treating the signature as a security-bypass credential.
Six checks from admin to the first request
First, create a clearly named signature under Online Store > Preferences > Crawler access. Second, select only a target domain already connected to the store. Third, choose a necessary lifetime no longer than three months. Fourth, copy Signature-Input, Signature and Signature-Agent completely. Fifth, confirm all three headers accompany every page and resource request. Sixth, crawl one public product page and one collection page with low concurrency and verify responses before expanding.
Do not place signatures in public repositories, screenshots, support-ticket bodies or browser frontend code. Record name, owner, bound domain, creation and expiry. Create a new signature before expiry, validate in small batches, then let the old one expire after the new configuration works. Official guidance says existing signatures cannot have their expiry changed or be renewed.
Separate 403s, 429s and challenges into four layers
First inspect requests: are all three headers present, and have proxies, redirects or tool settings escaped their values? Second inspect signatures: expiry, binding to the current domain and accidental crawling of the other side of myshopify.com versus the custom domain. Third inspect scope: does failure occur only at checkout, login or other extra-security pages? Fourth inspect rate and network: reduce concurrency and compare direct, unsigned and validly signed requests to the same public URL.
A 429 alone proves neither signature acceptance nor an IP ban. If a new signature, correct domain and low-rate small batch still produce the same challenge, stop rotating egress and repeating requests. Preserve request ID, response headers, failed samples and tool version for Shopify or tool support.
Rollback and stop conditions
Keep old scanner configuration before updating and limit the first batch to public pages that do not write data. Stop immediately for increased 4xx errors, unexpectedly reduced crawl counts, signature leakage or redirects to the wrong domain. Remove risky tool configuration and generate a new signature in admin; do not keep using exposed values.
If ordinary unsigned browsing also fails, use the proxy connection troubleshooting checklist to distinguish DNS, TLS, authentication and target-site responses. For stable operation of compliant auditing tools, visit the PuppyIP website for network information. Networking can improve only the connection path, not replace valid signatures, platform permissions or reasonable rates.
Common misconceptions: a signature is not a universal allowlist key
The first mistake is adding only Signature and omitting the other two headers. The second is reusing one merchant signature across domains when it is bound to one domain. The third is changing the date after expiry instead of creating a new signature as required. The fourth is assuming signatures allow checkout or authenticated pages, which have additional security boundaries.
The fifth is attributing every 403 or 429 to proxy quality. Signature format, domain, expiry and rate can produce similar symptoms. Check requests and signatures first, then scope and rate, and finally networking. Do not hide failed identity verification behind IP changes.
Sources
Frequently Asked Questions
When did Shopify open Web Bot Auth to merchants?
The Shopify Changelog announced on August 28, 2026 that merchants could create HTTP signatures authorizing custom crawlers and tools to access Online Store.
Which headers does Web Bot Auth require?
Merchant-generated configuration contains Signature-Input, Signature and Signature-Agent. All three values should accompany every target request.
How long can a signature be used?
Shopify Help Center says at most three months. Expired signatures cannot be renewed or have their expiry modified; create a new one and update the tool.
Why does 429 remain after adding a signature?
Check that all three headers are complete, the signature matches the current domain and is unexpired, and the tool has not entered checkout. Valid signatures establish identity and a higher access tier, not unlimited requests.
Must I create signatures for search engines crawling the store?
No. Official Shopify guidance says search engines and large language models can index stores normally without merchant signatures.
Should I change IPs first for 403 or 429?
No. Validate headers, signature, domain, expiry and page scope first. Investigate networking separately only with independent evidence such as DNS, TLS or connection timeouts.