8.6 The Deep Web

The Surface Web is also referred to as the 'Visible Web' - its content can be found using search engines such as Google, Yahoo and Bing, and is under continuos surveillance; accounting for up to 4% of all WWW content. The Deep web, however, is also referred to as the 'Invisible Web' - its content cannot be catalogued by search engines and is therefore difficult to keep under surveillance; accounting for up to 96% of WWW material
By T (Own work) – Deep web diagram – CC BY-SA 3.0

There are billions of web pages connected through the Internet. However, estimates are that traditional search engines on the web index just a small fraction of those pages. The rest has come to be known as the “deep web.” This “deep web ” consists of material in databases embedded in websites, Flash animations, information that is password protected or behind registration screens that cannot be “spidered,”, and “dynamic” content of various sorts such as airline flight schedules and news pages that are updated minute-by-minute.

Notably, this information is not necessarily completely “buried” and inaccessible to you as a user. Rather, much of this information is that which cannot be accessed through broad search engines such as Google or Yahoo!. It is important for you to know, then, that the material accessible to you through a web search site is significantly limited in scope – there may well be valuable information out there that you will have to dig a little more deeply to find.

If you need to do a thorough search on a topic, particularly one for which important information would be found inside a database, a directory, or in an Excel spreadsheet, you will have to look into the hidden nooks and crannies of the web to find what you need.

Some of the search engines that claimed to provide results from the deep web are no longer around. The experts now suggest that you identify some of the specialized search engines in the specific field or area of interest and use those to locate materials that the huge general search engines can’t reach. Examples include Economagic for data files, charts and Excel files for a vast array of economic data or Guidestar for information from and about the hundreds of thousands of non-profit organizations in the U.S.

License

Icon for the Creative Commons Attribution 4.0 International License

Information Strategies for Communicators Copyright © 2015 by Kathleen A. Hansen and Nora Paul is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.