Can I Choose Which Page to Crawl?

Can Specific Pages Be Selected for Crawling? 

It is possible to audit specific pages but not to select which pages to crawl. Ubersuggest does not allow users to select specific URLs for crawling, as it conducts a comprehensive analysis of the entire website. 

The tool does not follow a specific crawl order and may analyze different pages each time an audit is performed. Ubersuggest's crawlers are designed to respect sitemaps and discover URLs on pages. The crawling process begins with the homepage and proceeds to other pages from there. 

The sitemap is integral to this process, as the crawlers will look for it in its default location (example.com/sitemap.xml) or the location specified in the robots.txt file (example.com/robots.txt). 

How to Start Crawling? 

Navigate to the Site Audit Page 

1. Access the Site Audit page. 
2. In the left-hand menu, select the project to initiate the website audit. 

3. Add the website and start crawling. 
4. To crawl the specific website, the users must add the URL of the page that should be crawled: 

More details can be found here: Running a Site Audit With Ubersuggest.

How Many Pages Are Possible to Crawl? 

The number of pages Ubersuggest can crawl depends on the website's size and the selected plan's limits. Higher-tier plans allow for more pages to be crawled, with a maximum of 10,000 pages. 

For further assistance, please contact support at [e-mail].