Pages Blocked from Search Engines
When optimizing your website for search engines, it's essential to know which pages are accessible and which are not. Ubersuggest's Site Audit tool provides a report "blocked from appearing in search engines."
What Does It Mean When Pages Are Blocked From Appearing in Search Engines?
When a page is "blocked from appearing in search engines," search engines like Google, Bing, and others are instructed not to index that page. As a result, the page will not appear in search engine results, even if someone searches for its content.
Is It Normal to Have Pages Blocked?
It is a common misconception that all pages being blocked are a bad thing. There are several reasons why you might want to block certain pages:
- Privacy: Pages containing private information or for internal use only.
- Duplicate Content: To avoid SEO penalties, pages with duplicate content might be blocked.
- Utility Pages: Pages like "Thank You" after form submission or internal search results.
- Under Construction: Pages that are still in development or not ready for public viewing.
How to Unblock Pages from Appearing in Search Engines?
If you have identified pages that should not be blocked, follow these steps:
- Check the Robots.txt File: This file provides directives to search engines about which pages to crawl or not. Ensure that the page is not disallowed here. For more details, refer to Google’s guidelines here.
- Meta Tags: Ensure that the page does not have a <meta name="robots" content="noindex"> tag. If it does, remove or modify it. To learn more about meta tags, check this article from Neil’s blog.
- Re-submit to Search Engines: Once changes are made, you can request search engines to re-crawl your website. To find out how to ask Google to recrawl your pages, check this support article.
Why Does Ubersuggest Label This as a Critical Issue?
Ubersuggest flags it as a critical issue because, in many cases, essential pages that should be indexed are accidentally blocked. It is crucial to review this report to ensure that only the intended pages are blocked. With that said, we advise you to go through the list and make sure all blocked pages/URLs are there because you want them to be there.
Pro Tip
While it is normal to have certain pages blocked from search engines, it is essential to ensure that these are the right pages. Regularly reviewing the "blocked from appearing in search engines" report in Ubersuggest will help maintain optimal website SEO.