negative zero

Stop Conflating "Deep Web" and "Dark Web"

2021 April 26

[darknets] [info] [rant] [tech] [tor]

"Deep web" and "dark web" mean different things, and yet, people often use one to mean the other. First, let's define them clearly.

The deep web (as opposed to the surface web) is the subset of the World Wide Web which is not indexed by standard search engines. You likely visit several "deep web" pages every day: webmail, social media, maybe paywalled articles from an online publisher. Anything that requires you to sign in to view it is part of the deep web. This very website (the clearnet version that can be accessed with a standard browser like Firefox, Safari, or Chrome without any configuration) probably counts as part of the deep web because I've politely requested search engines not to crawl it.

A dark web is the set of web content available on a darknet (as opposed to clearnet) - an overlay network on the Internet which requires special software, protocols, etc. (such as Tor). A dark web is, then, the subset of the deep web which is only accessible over a darknet.

(I personally think it's useful to differentiate web content on different darknets as in the above paragraph. For example "Tor's dark web" would be the set of .onion websites, and "I2P's dark web" the set of .i2p websites. The general convention is to refer to the dark web as the sum of all web content on all darknets.)

Why does this distinction matter to me so much? Because from my observation, a lot of people who talk about Tor start with defining the deep web. They say that the deep web is the set of content not indexed by Google (usually alongside a scary-looking graphic). They say that Google only indexes about 10% of the web (or 3% or whatever), and the rest is the deep web. (They usually use that one graphic of an iceberg here.)

Then they start talking about the dark web (by which, of course, they always mean .onion sites on Tor specifically) and all the scary, evil things which can be found there. (I have opinions about that as well, but that's a different post.) But they almost never make it clear that this dark web is a different thing from the deep web they were just discussing. This leaves the viewer to draw the conclusion that there's about nine times as much content on Tor (that scary, evil network for hitmen and child pornographers) as there is on the regular internet they use.

I think that without making this distinction clear, this is irresponsible reporting. It's not that Tor houses 90% of the websites out there; it's that a lot of things on the web require login forms or the like. There are a lot of reasons something might be part of the deep web, and it's usually not because it's only accessible through Tor.

I've also encountered various people who post on Tor forums or operate onion sites, who refer to onionland itself as "the deep web". It seems to be a common misconception that "deep web" and "dark web" are synonymous. Please don't contribute to this misconception.