Start with the disapproved ad’s status detail and expanded URL. Reproduce the exact request path, then compare edge and origin evidence around the reported failure. A browser loading the final URL today tells you how that particular request behaved. The tracking chain, mobile path, region and security rules can produce a different response for AdsBot.
Copy the exact URL and status from the affected ad
Record the status detail, affected platform, time and URL for the ad or asset that failed. Compare the final URL, any mobile override and the expanded URL shown by Google Ads. Tracking templates and parameters can introduce a different host or path. A valid homepage is a separate test from the URL in the disapproval.
Check the suspected URL with the local landing URL checker below for parsing and parameter issues. Then inspect actual redirects and responses from a clean browser or terminal. Use the redirect diagnostic if a hop rewrites the destination or drops click parameters.
Pasted URLs stay in this browser. No URL visits, uploads or saved inputs.
Paste complete http:// or https:// URLs, including their query strings. For a comparison, use URLs from the same test journey. Remove personal information first.
Checks the URLs you paste. Use Google Ads’ URL test and browser diagnostics to verify live redirects, tag loading and consent.
Google Ads destination error details and expanded URLs.
Compare a clean visit with a user-agent reproduction
For a public HTTPS destination you control, substitute its URL and run the following GET. Inspect every response block in headers.txt and the final body in response.html. This command stops if a hop switches away from HTTPS. In Chrome DevTools, test with a clean session, then repeat with the documented mobile AdsBot string for a mobile error. Keep the request time and compare from the reported region when possible.
url="https://example.com/offer?utm_source=google"
curl -sS -L --max-redirs 8 --max-time 30 \
--proto "=https" --proto-redir "=https" \
-A "AdsBot-Google (+http://www.google.com/adsbot.html)" \
-D headers.txt -o response.html \
-w 'HTTP %{http_code} | %{num_redirects} redirects | %{time_total}s\n' "$url"Changing the user-agent string tests how the site responds to that label from your address. To identify real crawler requests, verify the client address independently. Check the body for a challenge, sign-in or error document even when the last status is 200. Cloudflare Challenge Pages expose cf-mitigated: challenge.
Google’s AdsBot web user agents and Cloudflare challenge response header.
Trace the edge decision before debugging the application
- Filter CDN or firewall logs by host, path, UTC window and desktop or mobile AdsBot label. Capture the response status, action, matched rule, cache state and request ID. An origin access log alone can miss a request answered at the edge.
- Take the visitor address from a trusted edge field or a proxy header that only your trusted proxy can set. Compare it with the published Google special-crawler ranges. For a single request, Google also documents verifying the reverse DNS name and resolving it forward to the original address. Save the snapshot date when reviewing historical logs.
- Join the edge record to origin status and latency by request ID where available. With Cloudflare, use the Ray ID alongside the timestamp and path. If you have neither record, check log retention and sampling, then investigate DNS, TLS, network paths or a failure earlier in the tracking chain.
Google’s crawler verification procedure and Cloudflare Ray ID reference.
Run a local log review with synthetic records
Download the Python log reviewer and run python3 adsbot-log-review.py --demo. It reads local data only. The sample uses documentation ranges and returns four records for /offer, leaving out the app crawler label and an unrelated path.
| Request | Label | Range match | Status | Edge action | Next check |
|---|---|---|---|---|---|
| example-a | Desktop | Yes | 200 | None | Confirm expected page body |
| example-b | Mobile | Yes | 200 | Challenge | Trace the matched edge rule |
| example-c | Desktop | Yes | 503 | None | Compare edge and origin status |
| example-d | Desktop | No | 200 | None | Verify address source and range date |
For your own review, export one hostname and a narrow time window. Use the exact CSV header below, with a trusted client IP, the recorded status and edge action, and an exact path. Keep different URL variants separate when a query parameter changes the response. The script preserves status and action so you can compare them to the raw event and the matching origin record.
timestamp,client_ip,user_agent,path,status,edge_action,request_id
python3 adsbot-log-review.py --log crawl.csv --prefixes special-crawlers.json --path /offerFix the layer with evidence, then recheck both paths
- Correct a broken tracking hop, wrong mobile path or inconsistent DNS/TLS configuration where you reproduced it.
- For an edge block, identify the rule and restrict any exception to the verified crawler traffic and appropriate host and paths. A user-agent-only exception is easy to spoof.
- Review any explicit
AdsBot-GoogleandAdsBot-Google-Mobilegroups inrobots.txt. Google documents that both web crawlers ignore the global*group. Keep a functioning, equivalent destination available to people and crawlers.
Download the crawl review worksheet to record the evidence and repeat the checks before requesting a policy review. Use the missing-arrival diagnostic if your concern is paid visitor requests rather than an ad review. If you are also investigating suspicious visits, start an AdProtektor trial to review visitor evidence and configure protection for the site.
Google Ads accessibility and review guidance.
Preview the crawl review worksheet
GOOGLE ADS DESTINATION CRAWL REVIEW
Ad or asset and exact status detail:
First and most recent UTC timestamps:
Reported device or region:
Final URL, mobile override and expanded URL (redact sensitive values):
Tracking-template and redirect hop with the first divergence:
Clean browser result:
Desktop and mobile AdsBot user-agent reproduction:
Terminal HTTP status, body and response headers:
DNS/TLS, IPv4/IPv6 and regional evidence:
Edge log source and time window:
Trusted client address field and proxy trust boundary:
AdsBot label, request ID, status and edge action:
Special-crawler range snapshot date or reverse and forward DNS:
Matching origin request ID, status, latency and cache state:
Evidence gaps (sampling, retention, traffic that never reached origin):
One scoped fix and rollback owner:
Repeat tests for visitor and crawler paths:
Google Ads review and subsequent status: