If you've tried your hand at search engine optimization, you may be familiar with the importance of having keyword-optimized content and quality backlinks to your website. In addition to content and link-building, the technical aspect of your website is also important and should not be overlooked. That's where Screaming Frog's SEO Spider Tool comes in. Screaming Frog is a powerful tool that is meant to simulate the Googlebot and crawl your site the same way Google does. To get started, enter the URL of the site you wish to crawl in the Screaming Frog software. Screaming Frog will immediately begin to crawl your site and present the data it finds in real time. While the software provides an in-depth overview of your site, we'll focus on five SEO issues you can fix with Screaming Frog.
Identify Broken Links
Having broken links on your website can not only affect user experience; it can also affect whether Google decides to show your site in the search results. If your website has dozens of pages with 404 errors, Google might favor showing sites that provide a better user experience in their SERPs. Luckily, finding and updating these broken links is easy with Screaming Frog!
Once your site is crawled, go to the Internal tab and sort the results by Status Code. This will group your site's pages by the response code the crawler received. Pages with a 301 code have been permanently redirected to another page, while pages with a 404 code will result in a "Page Not Found" error. For pages with a 404 code, click on the Inlinks tab in the bottom left of the Screaming Frog window. This tab will list the pages that are linking to the 404 page. With this information, you can then decide whether you want to manually update the broken links on each of those pages or create a 301 redirect from the 404 page to a new page of your choice.
Analyze Page Titles & Meta Descriptions
Screaming Frog is a powerful tool when it comes to analyzing your site's metadata. Visit the Page Titles tab and sort by the Indexability column to see page titles for all of your site's indexed pages at a glance. Below the tab or on the right of the Screaming Frog window, you can filter the pages to see which URLs have a page title that is missing, a duplicate of another page, or over/under the suggested character or pixel limit. Visit the Meta Description tab in Screaming Frog, and you'll be able to find similar information about your pages' meta descriptions.
Pages that are missing a title or meta description or have the same metadata as another page should be updated to include the unique keywords you want that page to rank for. Screaming Frog's Export tool can be used to export this metadata information into a .csv file or an Excel spreadsheet. For more control over your data, you can also visit the Internal tab, use the HTML filter, and sort by the Indexability column. Drag and drop the Address, Title 1, and Meta Description 1 columns so they're adjacent to each other, then copy and paste the data from these three columns into a spreadsheet where you can work on your metadata until it's ready to update on your website.
Identify Pages with Thin Content
In addition to identifying broken links and analyzing metadata, Screaming Frog can also identify pages that have thin content. Under the Internal tab, use the HTML filter and sort by the Word Count column. Since this column often loads in the far right of Screaming Frog, feel free to drag it next to the Address column, so you can quickly identify the pages on your site that have a low word count. With this information, you can decide which pages could use more content, which in turn may help improve your chances of appearing in the search results. You can also identify which pages might have too much content and consider dividing that into two pages, if it makes sense for user experience.
Verify Indexability
When you conduct a site search on Google (e.g. site:google.com), Google will show you the pages your website has in its index. Screaming Frog takes that a step further by also showing you the pages that are not indexed. From the Internal tab, sort your HTML pages by the Indexability column. As the name suggests, indexable pages can appear in Google's SERPs and non-indexable pages cannot. With this information, you can verify whether the correct pages are indexed and whether there are any non-indexable pages that should be indexed. This is incredibly useful to quickly identify whether a landing page might have accidentally been set to 'no-index' or whether an important section of your site may be blocked by your robots.txt file.
Improve On-Site Optimization
On top of crawling your site's metadata, Screaming Frog also crawls your site's content and its headings. Under the H1 tab, you can filter out which pages are missing an H1 heading tag or which pages may have more than once occurrence of an H1. As a general rule of thumb, pages should have a single H1 tag that contains the keyword phrase the page is trying to rank for. Under the H2 tab, you'll find similar information for your H2 headings. Unlike H1s, however, H2 heading tags can occur more than once on a site and should be used to make your content more scannable and user-friendly.
Summary
Screaming Frog is an incredibly powerful crawling tool that can be used to quickly audit a website. With this spider tool, you can see a wide range of information about your website, such as meta data, response codes, response time, internal/external links, canonical URLs, page directives, the presence of structured data, and much more. In this blog, we've covered five SEO issues you can quickly identify and fix with the use of Screaming Frog. For an in-depth technical audit of your website and professional SEO services, contact Geek Powered Studios today!
