Skip to main content

Site Audit Errors Explained: My Site Crawl Has Stopped

Admin
  • Updated

The Site Audit for Your Website has Stopped

Ubersuggest's Site Audit tool is a powerful feature designed to help website owners identify and rectify SEO issues. However, there are instances where Ubersuggest may not be able to crawl your website. This article aims to guide beginner SEOs on understanding and resolving issues that might prevent Ubersuggest from crawling your website.

 

What is "The Site Audit for Your Website has Stopped"?

This error message indicates that Ubersuggest was unable to complete the site audit for your website. This could be due to a variety of reasons, such as website accessibility issues, robots.txt restrictions, or improper redirects.

Why is this Error Bad for SEO?

If Ubersuggest cannot crawl your website, it means search engine crawlers might also face similar issues. This can negatively impact your site's SEO as search engines will not be able to index your content effectively.

How to Identify the Issue?

Here are some common reasons and how to troubleshoot them:

  • Website Accessibility: Make sure your website is accessible. If it is not, Ubersuggest will not be able to crawl it.

  • Robots Tag: Check if your homepage is set to 'noindex'. If so, change the tag to allow indexing.

  • Robots.txt: Ensure that your robots.txt file allows robots to crawl your website.

  • Redirects: Verify if your website has enabled proper 301 redirects.

How to Fix the Issue?

  • For Website Accessibility: Ensure your website is up and running.
  • For Robots Tag: Change the meta tag to <meta name="robots" content="index">.
  • For Robots.txt: Add a line "Allow: /" under the section "User-agent: *", following Google's guidelines. 
  • For Redirects: Make sure your website redirects properly from http:// to https://.

Locating the Issue on Your Website

You can use the browser's "Inspect Element" tool to check meta tags and robots.txt. For redirects, you can manually enter your website URL in an incognito window to see if it redirects properly. To check your robots.txt file, open it in browser example.com/robots.txt instead of "example" put your website address.

Pro Tip and Further Reading

If you have verified the first four steps and still face issues, your website might have a security system that blocks our robots. You may need to add Ubersuggest's IPs as an exception to your security system.

For further reading, you may want to explore these articles: