Site Audit is one of the most complex features in Ubersuggest. Unfortunately, it does not have the option to exclude a subdomain from scanning.
If a user does not want to see SEO issues related to the subdomain in the audit report, they can disallow the pages inside robots.txt.
How to Disallow Pages on robots.txt?
To disallow a specific page from being indexed by search engine crawlers through the robots.txt file, users can use the "Disallow" directive. Here's how to do it:
Open or create the robots.txt file for the website. This file should be located in the root directory (e.g., www.example.com/robots.txt).
Add a "Disallow" directive followed by the URL path of the page or directory to block from indexing. For example, if a user wants to block a page named "example-page.html", they will add the following line:
Save the robots.txt file and upload it to the root directory of the website.
However, it is important to highlight that the pages will still be detected as blocked pages instead since the Disallow is a directive and not a command.
What Is a robots.txt Directive and Where to Find It?
A robots.txt directive is a set of instructions contained within the robots.txt file, which serves as a protocol for controlling the behavior of web crawlers and search engine bots that access a website's content.
This file, typically in the root directory of a website, communicates guidelines to these bots regarding which pages or directories they can crawl and index, and which ones they should refrain from accessing. By utilizing directives such as "Allow" or "Disallow", website owners can effectively manage crawler access to specific areas of their site, thereby influencing its visibility and presence within search engine results. The robots.txt directive is a fundamental tool for webmasters seeking to optimize their website's indexing process and maintain control over how search engines interpret and display their content.
For a better context of the Site Audit feature, please refer to the following articles:
Please reach out to us at support@ubersuggest.com for any doubts, we love hearing from our community!