How do I know the number of pages on a website?

It is not uncommon for a website owner or SEO specialist to be faced with the question of how many pages a website has, and sometimes we want to know the number of pages on a competitor’s website. And we are not talking about the number of pages indexed by a particular search engine, which can be checked by typing “site:[url of your site]” in Google. We are talking about how to check the total actual number of pages on a website, when you need to take into account even those pages that are blocked in robots.txt or simply not indexed for some reason.

In Google search, when checking the pages in the index, it is better to scroll to the last page, as the last page will show the exact value, not the approximate one.

Tools for determining the number of pages on a website

• ScreamingFrog

This tool is extremely useful. It collects absolutely all data about the site, including the number of pages (even pages blocked by the robots.txt file), server and link errors, redirects, external and internal links and their status, problems with page titles, metadata, page response time, image alt text, duplicate pages, etc. For each operation, the service generates a report that can be downloaded in the following formats: CSV, XLS, XLSX, GSHEET.

The free version crawls up to 500 URLs and offers 7 functions (keyword analysis, search for broken links, duplicates, and errors). If you have more links on your website, you need to upgrade to the premium version and then you will be able to use absolutely all the functions of the program (there are 29 of them today). To do this, you will need to download and install this tool.

• www.xml-sitemaps.com

In general, this tool is designed to create sitemaps. It creates free and fast sitemaps for resources with the number of pages less than 500, to increase the number of pages you need to buy a subscription. During the generation of the map it bypasses absolutely all your site, as a result you get the exact number of pages on your resource. The advantage of this tool is that it does not need to be downloaded and installed on your computer.

• Beam Us Up

A free service that scans sites for errors, such as: broken links, as well as missing page titles, duplicate content and identify other problems. In short, it does all the same things as ScreamingFrog, but with less quality, but completely free of charge.

To summarize

In this article, we have indicated only the most popular and available tools for identifying the exact number of pages on the site. On the vastness of the Internet you can find dozens of different paid and free crawlers that will help you cope with the task at hand. But it is our experience that the most accurate, convenient and reliable tool is ScreamingFrog. With it, you can always quickly and efficiently unload the site and find all the flaws – from duplicate titles to missing meta descriptions on individual pages.

Serhii Ivanchenko

CEO

Author's page