Nov
I’m sure right now, here you have a very big question about the phrase “Duplicate Content!?”. It is really a problem that not only people who specialize in Content, Marketing, SEOer, … are interested. But it is a topic of everyone, all professions if you own or work on the website.
Have you ever wondered when someone mentioned what Duplicate Content is, like:
- Duplicate Content will 100% harm the website, may be penalized by Google.
- Or, Duplicate Content really does not adversely affect the website, no need to pay too much attention.
However, this is really your question about Duplicate Content:
- What is Duplicate Content?
- What is Duplicate Content really about content?
- If Duplicate Content adversely affects the website, how to handle it?
- More specifically, how to detect duplicate content and how to Check Duplicate Content on the website?
I used to be like you and understand what you need. That is why I am sending you the following instructions. Here’s the answer to all your questions about Duplicate Content and more.
Let’s find out now!
What is Duplicate Content?
In a narrow sense, Duplicate Content (DC) is content on one or more different websites that are similar or completely similar. In a broader sense, it is content that brings little or no value to the visitor. Therefore, pages with little or no useful content are also considered Duplicate Content.
Why is Duplicate Content Bad for SEO?
Duplicate Content can be bad for SEO for two reasons below:
- When there are multiple versions of content, it is difficult for search engines to determine which version to index. As well as showing which version in the results page. This degrades the performance of all versions of the content because they are competing with each other.
- Search engines will have a hard time consolidating link metrics. Examples include relevance, influence, and credibility for the content. Especially when other websites link to multiple versions of that content.
Does Google Penalize Duplicate Content?
Duplicate Content will adversely affect the SEO performance of your website. But it will not cause your website to be penalized by Google if you do not intentionally copy the content of other websites.
If you use a few techniques in your website but don’t intentionally trick Google, there’s no need to worry about getting penalized by Google.
If you have copied a large amount of other website content. Then you are definitely standing on a thin line. Because Google once spoke about the DC issue as follows:
“Duplicate content on a website is not a basis for imposing a penalty. Google only penalizes websites that use DC to deceive and manipulate search engine results.
If your website is having problems with DC and you are not following Google’s recommendations. Then we will choose the best version of the content to show in the search results.”
10+ Common Causes of Duplicate Content and Solutions for them
There are many causes of Technical SEO – Duplicate Content errors. However, I have put together 15 common causes of this problem and ways to solve them.
1. Faceted/Filtered Navigation
Faceted Navigation – Also known as multi-dimensional navigation. A place where users can filter and sort items on the page. E-commerce websites often use it a lot.
This type of navigation attaches parameters to the end of the URL. For example:
Navigation with an end-of-URL parameter
Because there are often many combinations of filters. Multidirectional navigation leads to Duplicate or near-duplicate Content.
Let’s look at the two examples below to better understand this reason:
- bbclothing.co.uk/en-gb/clothing/shirts.html?new_style=Checked
- bbclothing.co.uk/en-gb/clothing/shirts.html?Size=S&new_style=Checked
These URLs are unique, but the content is almost identical.
Also, the order of the parameters usually doesn’t matter. For example, you can access the same page using either of the following URLs:
- bbclothing.co.uk/en-gb/clothing/shirts.html?new_style=Checked&Size=XL
- bbclothing.co.uk/en-gb/clothing/shirts.html?Size=XL&new_style=Checked
How to fix:
Faceted navigation is a very complex problem. If you suspect this is the cause of Duplicate Content. Decide what pages you want Google to index. Then increase the number of useful pages indexed and remove the unnecessary ones.
2. Tracking Parameters
Parameterized URLs are also used for tracking. For example, we can use UMT parameters to track visits in the Google Analytics newsletter campaign section:
Example: example.com/page?utm_source=newsletter
How to fix:
Normalize your parameterized URLs to get SEO-friendly versions without tracking parameters.
3. Session IDs
Session IDs store information about visitors to the website. They usually append a long string to the URL like this:
Example: example.com?sessionId=jow8082345hnfn9234
How to fix:
Normalize URLs to get SEO-friendly versions.
4. HTTPS with HTTP and non-www with www
You can access the website in one of the four variations below:
- https://www.example.com (HTTPS, www)
- https://example.com (HTTPS, non-www)
- http://www.example.com (HTTP, www)
- http://example.com (HTTP, non-www)
The HTTPS version is the first two URLs. Whether you use the www or non-www version, you can still access the website.
However, if you do not configure the server correctly. Then your website will be accessible with these variations. This is really bad and can lead to Duplicate Content issues.
How to fix:
Use redirects to ensure your website can only be accessed by a single version.
5. URLs are case sensitive
Case-sensitive URLs mean that the three URLs below are all different:
- example.com/page
- example.com/PAGE
- example.com/pAgE
How to fix:
Be consistent with internal links (i.e. don’t internally link to multiple URL versions). If this doesn’t solve what is Duplicate Content anymore, then canonicalization or redirection can be tried.
6. Following slash vs non-following slash
Google doesn’t consider URLs with or without trailing slashes. That means Google considers the following 2 URLs as one:
- example.com/page/
- example.com/page
If your content can be accessed using both URLs, it will result in a Duplicate Content error. To Check DC to see if this is the problem, try using both URLs with and without trailing slashes.
For example: If you try to load your post with a URL without a slash, it will redirect to a URL with a slash.
How to fix:
Unwanted version redirect. Example: Without a trailing slash, to the desired version (e.g. followed by a slash). You should also make sure your internal links are consistent at all times. Choose a single version and use it across all URLs.
7. Print-friendly URLs
The print-friendly URL has the same content as the original. And it’s simply another URL.
- example.com/page
- example.com/print/page
How to fix:
Normalize friendly versions to original versions.
8. Mobile-friendly URLs
Similar to print-friendly URLs, mobile-friendly URLs are also duplicates.
For example:
- example.com/page
- m.example.com/page
How to fix:
Standardize the mobile-friendly version to the original version. Use rel=”alternate” to remind Google that the mobile-friendly URL is an alternative to the desktop version of the content.
9. AMP URLs
Accelerated mobile pages (AMP) are also duplicates.
For example:
- example.com/page
- example.com/amp/page
How to fix:
Normalize the AMP version to the non-AMP version. Use rel=”amphtml” to tell Google that AMP URLs are an alternative to non-AMP content.
If you only have AMP content, use a self-referencing canonical tag.
10. Tag and Category Pages
Most CMSs (Content Management Systems) create pages with dedicated tags when you use tags.
Example: If you have an article about Organic Whey Protein. And if you’re using both the terms “protein powder” and “whey” as tags, you’ll end up with two pages of tags like this:
- https://www.calton Nutrition.com/tag/whey/
- https://www.calton Nutrition.com/tag/protein-powder/
This doesn’t always lead to Duplicate Content but it does sometimes cause this problem.
In the case below, there is only one page on the website with two tags – so each page is identical.
How to fix:
There are 2 workarounds as follows:
- Do not use cards. Because they have little or no value at all.
- Do not index pages with multiple tags. This won’t solve the crawl budget issue because Google will still take the time to crawl these pages.
Note that category pages can lead to the same problem as pages with multiple tags. Eg:
- https://www.xs-stock.co.uk/adidas/
- https://www.xs-stock.co.uk/brands/Chelsea-FC.html
Both of these sites are almost identical as there are no products listed in either category. Therefore, all we see is a ready-made copy of the template.
How to fix:
Use a moderate number of categories on your website or even not index your category pages.
11. Attached image URL
Many CMSs create pages specifically for image attachments. These pages usually display nothing but images and some sample copy.
Because this copy is the same across all auto-generated pages, it results in Duplicate Content.
How to fix:
Disable image-specific pages in the CMS. In WordPress, you can do this using a plugin like Yoast.
12. Comments are paginated
WordPress and other CMSs allow paginated comments. This also leads to Duplicate Content because it creates multiple versions of the same URL.
For example:
- example.com/post/
- example.com/post/comment-page‑2
- example.com/post/comment-page‑3
How to fix:
Turn off comment pagination or de-index your paginated pages using a plugin like Yoast.
13. Localization
If you distribute the same content to multiple people in multiple countries but use the same language (e.g. English) it will also lead to Duplicate Content issues.
For example, you can design different versions of your website for users in the US, UK, and Australia. Each version for each of these countries will be nearly identical and differ only in a few small points.
For example, using the word “dollar” in content for Americans and “pounds” in content for British.
However, according to John Mueller, the translated content is not DC.
How to fix:
Use the Hreflang tag to inform search engines about the relationship between variants.
14. Search results page
Many websites have search boxes. Using these boxes typically produces a parameterized search URL.
Example: example.com?q=search-term
How to fix:
Use the Meta Robot tag to remove search pages from the Google index list or block access to pages containing search results in the robots.txt file. Limit internal linking to pages that contain search results on your website.
15. Staging Environment
The Staging environment is an identical or near-duplicate version of the website used for testing purposes.
Example: Imagine that you want to install a new Plugin or change some code on your website. Of course, you don’t want to show them on your website because every day, thousands of visitors visit it.
Therefore, test the changes in the staging environment first. However, the Staging environment affects SEO as Google still indexes them and leads to Duplicate Content problems.
How to fix:
To protect the environment Staging. You should use HTTP authentication, whitelisting IP addresses, or VPN access. If it’s still indexed then use the auto-index stop command to remove it.
How to Check Duplicate Content on Websites
Duplicate Content is content that appears across multiple online locations. This means different websites. If you post your content in multiple places, it will lead to Duplicate Content.
If you copy other people’s content and post it on your website. Or even if they post your content on their website, it is considered Duplicate Content.
So how to Check Duplicate Content to see if your content has DC errors?
Use Google to Check Duplicate Content
A quick way to check if a page is considered duplicate content is to copy the first 10 words of a sentence and enclose them in quotation marks. Then put them on Google. This is how Google recommends to Check Duplicate Content.
If you only test Duplicate Content internally on your website, you will not achieve any useful results.
If other websites perform well similar to yours. Google will evaluate which page is the original page and render it first. If your website is not displayed first, then there is a Duplicate Content problem.
Free Tools to Support Check Duplicate Content Online
Before you publish an article, you should check your content with plagiarism checking tools. Here are free tools you can use to Check Duplicate Content Online.
- Copyscape – With this Check Duplicate Content Online tool, it only takes you a few seconds to check if the content is duplicated with what has been posted. This comparison tool will highlight duplicate content, as well as specify the percentage of duplicate content.
- Plagspotter – This tool can identify pages with duplicate content on the website. This is a great tool to identify which website has stolen content from your website. It also allows to automatically monitor its URLs on a weekly basis to identify Duplicate Content.
- Duplichecker – This tool quickly checks the uniqueness of the content you plan to post on your website. Registered users of this tool can perform up to 50 searches per day.
- Siteliner – This tool makes it possible to check your entire website once a month for Duplicate Content errors. It can also check for broken links. And identify the most prominent pages for the search engines.
- Smallseotools – There are many SEO tools, including a plagiarism checker that helps you identify identical pieces of content.
Conclusion
Hopefully, through this article, you have clearly understood what Duplicate Content is and its harmful effects to your website. Therefore, before publishing any article, you should use the Check Duplicate Content Online tools to ensure your content is unique.
Just follow this guide and get serious about managing Duplicate Content. You will improve your rankings and avoid unnecessary errors on the website.
Good luck!
