World's first web census

Abstract

Purpose

To measure the exact size of the world wide web (i.e. a census). The measure used is the number of publicly accessible web servers on port 80.

Design/methodology/approach

Every IP address on the internet is queried for the presence of a web server.

Findings

The census found 18,560,257 web servers.

Research limitations/implications

Any web servers hidden behind a firewall, or that did not respond within a reasonable amount of time (20 seconds) were not counted by the census.

Practical implications

Whenever a server is found, we download and store a copy of its homepage. The resulting database of homepages is a historical snapshot of the web which will be mined for information in the future.

Originality/value

Past web surveys performed by various research groups were only estimates of the size of the web. This is the first time its size has been exactly measured.

Citation

Benoit, D. and Trudel, A. (2007), "World's first web census", International Journal of Web Information Systems, Vol. 3 No. 4, pp. 378-389. https://doi.org/10.1108/17440080710848143