During research for my recent post about Dark Nets, I came across the term Deep Web and wondered what that meant.
The Deep Web is the part of the Internet that is not indexed or catalogued by search engines. In other words, search 'bots for services like Bing and Google and Duckduckgo haven't crawled through these areas and noted their addresses and locations, so they don't show up in search engine results.
A picture of an iceberg is often used to illustrate this idea, with the smaller, above-water section representing the Internet we all use, and the larger, below-water section as the Deep Web. It's a useful image, but I don't know if it's accurate because no one seems to know how big the Deep Web is. It may be bigger or smaller than the regular Internet.
What does the Deep Web contain? While the articles I found all agreed that being non-indexed defines Deep Web content, they did not agree on what that content is.
For instance, one article explained that much of the content is dynamic--pages that are constantly updated and changed like airline flight information or breaking news stories--while much of the regular Internet is static. An example of this is a database where you can only retrieve information with a query.
There are also fee-based sites that you must pay for before being able to access. The article mentions subscription services used by libraries, but I would guess a lot of pay sites are for pornography. The article also points out the dynamic content of social networking sites like Facebook and Twitter, but users must only be friends or followers to access them.
This all sounds fairly benign. In contrast, another article I found describes a far different Deep Web in which anonymity is king, and criminal transactions for drugs, credit card numbers and child pornography thrive. The article has a screen shot from a popular drug sale site that lists all sorts of illegal drugs for sale in varying amounts from anonymous vendors. The author also wisely warns people that if they go wandering around the Deep Web they may find sickening child pornography.
Thus the Deep Web may be anything from a business database to an illegal file sharing site. While I normally supply links to my sources, in this case I'm not because some of them provide instructions that will only tempt people to explore. The regular Internet is vast, with many resources available to you, so you need not view the Deep Web as something you're missing out on. And if what I read about much of the content is true, you're better off avoiding it altogether.
(The pic of a sewer is from: benmayfield.wordpress.)