![]() This will exclude the menu from being included in the duplicate content analysis algorithm. While this isn’t much of an issue, in this case, to help focus on the main body text of the page its class name ‘mobile-menu_dropdown’ can be input into the ‘Exclude Classes’ box. ![]() You can choose to ‘include’ or ‘exclude’ HTML tags, classes and IDs in the analysis.įor example, the Screaming Frog website has a mobile menu outside the nav element, which is included within the content analysis by default. However, not every website is built using these HTML5 elements, so you’re able to refine the content area used for the analysis if required. The SEO Spider will automatically exclude both the nav and footer elements to focus on main body content. For a new crawl, we recommend using the default set-up and refining it later when the content used in the analysis can be seen, and considered. You’re able to configure the content used for near-duplicate analysis. If you’re interested in finding crawl budget issues, then untick the ‘Only Check Indexable Pages For Duplicates’ option, as this can help find areas of potential crawl waste.Ģ) Adjust ‘Content Area’ For Analysis Via ‘Config > Content > Area’ This means if you have two URLs that are the same, but one is canonicalised to the other (and therefore ‘non-indexable’), this won’t be reported – unless this option is disabled. The SEO Spider will also only check ‘Indexable’ pages for duplicates (for both exact and near-duplicates). The SEO Spider will identify near duplicates with a 90% similarity match, which can be adjusted to find content with a lower similarity threshold. However, to identify ‘Near Duplicates’ the configuration must be enabled, which allows it to store the content of each page. If you’re a free user, then skip to number 3 in the guide.ġ) Enable ‘Near Duplicates’ Via ‘Config > Content > Duplicates’īy default the SEO Spider will automatically identify exact duplicate pages. The first 2 steps are only available with a licence. To get started, download the SEO Spider which is free for crawling up to 500 URLs. ![]() Watch our video, or continue to read our guide below. This tutorial walks you through how you can use the Screaming Frog SEO Spider to find both exact duplicate content, and near-duplicate content where some text matches between pages on a website.ĭuplicate content identified by any tool, including the SEO Spider needs to be reviewed in context. You can limit crawl budget waste and consolidate indexing and link signals to help in ranking. Preventing duplicate content puts you in control over what’s indexed and ranked – rather than leaving it to the search engines. However, at scale it can be more problematic. It’s worth remembering that duplicate and similar content is a natural part of the web, which often isn’t a problem for search engines who will, by design, canonicalise URLs and filter them where appropriate. While a ‘duplicate content penalty’ is a myth in SEO, very similar content can cause crawling inefficiencies, dilute PageRank, and be a sign of content that could be consolidated, removed or improved. How To Find Duplicate Content Duplicate content should be minimised across a website, as it can make it difficult for search engines to decide which version to rank for a query.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |