Paste your URLs below, one per line. This is the most reliable input method with zero CORS restrictions.
Direct in-browser crawling relies on standard JavaScript fetch() requests and is subject to the browser's Same-Origin Policy.
This mode only works if the target website explicitly enables cross-origin access (via Access-Control-Allow-Origin: *).
Because most websites block cross-origin browser requests by default, fetch errors are expected for non-CORS websites.
For complete reliability on any website, please use the Manual Entry or Bulk Import tabs.
Upload or paste an existing sitemap.xml or sitemap_index.xml to load, modify, and re-export your URLs and metadata.
| :: | Status | URL (loc) | Lastmod | Changefreq | Priority | Media | Actions |
|---|
Sitemap Protocol 0.9 & Browser Architecture Guide
Everything you need to know about XML sitemaps, protocol constraints, browser limitations, and search engine best practices.
- Maximum 50,000 URLs per Sitemap: A single
sitemap.xmlfile must contain no more than 50,000<url>entries. If your site has more pages, you must split them into multiple sitemaps and reference them inside asitemap_index.xmlfile. - Maximum 50MB Uncompressed File Size: Sitemaps cannot exceed 50 megabytes when uncompressed.
- UTF-8 Character Encoding: All XML sitemaps must be encoded in UTF-8. Special XML characters (
&,<,>,",') must be escaped as entities. - Canonical, Fully-Qualified Absolute URLs: All
<loc>tags must include the protocol (e.g.https://) and full hostname. Relative URLs are invalid.
Unlike backend crawlers (like Googlebot or desktop software), web applications running in a browser are bound by the Same-Origin Policy.
When a web app attempts to fetch() a third-party webpage, the browser blocks the response unless the server explicitly responds with the HTTP header:
Access-Control-Allow-Origin: *
Because standard production websites do not serve HTML with permissive CORS headers, automated client-side crawling will fail with a network error. This is why our Manual Entry and Bulk CSV/TXT upload modes are the primary, 100% reliable workflow — all processing happens instantly in your browser with zero data leaving your computer.
- <lastmod>: Google actively uses accurate
<lastmod>dates to prioritize re-crawling updated pages. Ensure this date is only updated when meaningful content changes occur. - <priority> and <changefreq>: While part of the protocol, Googlebot generally disregards
<priority>and<changefreq>in favor of its own machine learning crawl budget algorithms. However, other search engines (like Bing and Yandex) still utilize these hints. - Discovery via robots.txt: Adding
Sitemap: https://yourdomain.com/sitemap.xmlto yourrobots.txtallows any compliant search bot to automatically discover your sitemap index.
Google supports extended XML schemas that associate media items with their parent page:
- Image Sitemaps: Use
<image:image>with<image:loc>,<image:title>, and<image:caption>to help Google Images discover images rendered via JavaScript or CDNs. - Video Sitemaps: Use
<video:video>with required<video:thumbnail_loc>,<video:title>,<video:description>, and<video:content_loc>to enable rich video search snippets.