Screaming Frog SEO Spider

Crawl websites to identify SEO issues and opportunities.

Screaming Frog SEO Spider is a powerful website crawling tool designed for SEO professionals and site owners. This fast and efficient software scans websites of all sizes, providing real-time data analysis for informed SEO decisions. It identifies broken links, server errors, and audits redirects, including chains and loops.

The tool extracts data using XPath, CSS Path, or regex, and crawls sites with a breadth-first algorithm similar to Googlebot. It can render web pages using headless Chromium for content discovery. SEO Spider offers features like broken link checking, duplicate content detection, and XML sitemap generation.

SEO Spider performs comprehensive site audits, analyzing page titles, meta descriptions, robots.txt, and security issues. It reports on client errors, URL problems, and measures response time and crawl depth for performance insights. The tool integrates with Google Analytics, Search Console, and PageSpeed Insights, allowing for data export and visualization of site architecture.

Used by major brands and agencies, SEO Spider offers both free and paid versions. The Lite version allows crawling of up to 500 URLs, while the full license removes URL limits and provides access to advanced features like JavaScript rendering.