Monday, April 2, 2012

What is the Deep Web?

During research for my recent post about Dark Nets, I came across the term Deep Web and wondered what that meant. 

The Deep Web is the part of the Internet that is not indexed or catalogued by search engines.  In other words, search 'bots for services like Bing and Google and Duckduckgo haven't crawled through these areas and noted their addresses and locations, so they don't show up in search engine results.

A picture of an iceberg is often used to illustrate this idea, with the smaller, above-water section representing the Internet we all use, and the larger, below-water section as the Deep Web.  It's a useful image, but I don't know if it's accurate because no one seems to know how big the Deep Web is.  It may be bigger or smaller than the regular Internet.

What does the Deep Web contain? While the articles I found all agreed that being non-indexed defines Deep Web content, they did not agree on what that content is.

For instance, one article explained that much of the content is dynamic--pages that are constantly updated and changed like airline flight information or breaking news stories--while much of the regular Internet is static.  An example of this is a database where you can only retrieve information with a query. 

There are also fee-based sites that you must pay for before being able to access.  The article mentions subscription services used by libraries, but I would guess a lot of pay sites are for pornography.  The article also points out the dynamic content of social networking sites like Facebook and Twitter, but users must only be friends or followers to access them.

This all sounds fairly benign.  In contrast, another article I found describes a far different Deep Web in which anonymity is king, and criminal transactions for drugs, credit card numbers and child pornography thrive.  The article has a screen shot from a popular drug sale site that lists all sorts of illegal drugs for sale in varying amounts from anonymous vendors.  The author also wisely warns people that if they go wandering around the Deep Web they may find sickening child pornography.

Thus the Deep Web may be anything from a business database to an illegal file sharing site.  While I normally supply links to my sources, in this case I'm not because some of them provide instructions that will only tempt people to explore.  The regular Internet is vast, with many resources available to you, so you need not view the Deep Web as something you're missing out on.  And if what I read about much of the content is true, you're better off avoiding it altogether.

(The pic of a sewer is from:  benmayfield.wordpress.)


  1. Thanks for the information. I was reading a Wired magazine article recently and it mentioned how the National Security Agency is stepping up its monitoring of the Deep Web and I really had no clue as to what it meant; now I do.

  2. I think many governments and their agencies would be willing to overlook minor drugs sales and credit card fraud on the Deep Web, but the evil presence of child pornography will always draw their attention. And it should because every picture means that somewhere in the world a child has been abused.