Deep Web, Dark Web, Dark Net a Taxonomy of “Hidden” Internet
Total Page：16
File Type：pdf, Size：1020Kb
ADVANCE PUBLICATION Annals of Business Administrative Science https://doi.org/10.7880/abas.0200908a Received: September 8, 2020; accepted: October 16, 2020 Published in advance on J-STAGE: December 5, 2020 Deep Web, Dark Web, Dark Net: A Taxonomy of “Hidden” Internet Masayuki HATTAa) Abstract: Recently, online black markets and anonymous virtual currencies have become the subject of academic research, and we have gained a certain degree of knowledge about the dark web. However, as terms such as deep web and dark net, which have different meanings, have been used in similar contexts, discussions related to the dark web tend to be confusing. Therefore, in this paper, we discuss the differences between these concepts in an easy-to-understand manner, including their historical circumstances, and explain the technology known as onion routing used on the dark web. Keywords: anonymity, deep web, dark web, dark net, privacy a) Faculty of Economics and Management, Surugadai University. 698 Azu, Hanno, Saitama, Japan. [email protected] A version of this paper was presented at the ABAS Conference 2020 Summer (Hatta, 2020b). © 2020 Masayuki Hatta. This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. 1 Hatta Introduction In recent years, the term “dark web” has become popular. The dark web, i.e., a World Wide Web wherein your anonymity is guaranteed and one that cannot be accessed without using special software, was, until recently, of interest to only a few curious people. However, in 2011, the world’s largest online black market, Silk Road (Bolton, 2017), was established on the dark web; with the presence of virtual currencies, which incorporate the anonymity provided on the dark web (Todorof, 2019), it has become a topic of economic and business research. Words similar to “dark web” (such as “deep web” and “dark net”) are used in the same context, but they are completely different technical concepts; this leads to confusion. Deep Web Historically, among the three terms (“dark web,” “deep web,” and “dark net”), the term “deep web” was the first to emerge. Computer technician and entrepreneur, Michael K. Bergman, first used it in his white paper “The deep web: Surfacing hidden value” (Bergman, 2001). Bergman likened web searches to the fishing industry and stated that legacy search engines were nothing more than fishing nets being dragged along the surface of the sea, even though there is a lot of important information deep in the sea, where the nets do not reach. Therefore, he stated that, moving forward, it was important to reach deep areas as well. This was the advent of the deep web. Bergman stated that the deep web was 400–550 times larger than the normal web, and that the information found in the deep web was 1,000–2,000 times the quality of the normal web. The problem is that even now this is used in the context of the dark web. What Bergman (2001) first raised as detailed examples of the “deep” web were the National Oceanic and Atmospheric Administration (NOAA) and 2 Deep web, dark web, dark net United States Patent and Trademark Office (USPTO) data, JSTOR and Elsevier fee-based academic literature search services, and the eBay and Amazon electronic commerce sites; these are still referred to as the “deep web” today. In short, Bergman referred to the following as the deep web: (a) Special databases that could only be accessed within an organization (b) Sites with paywalls wherein content can only be partly seen or not seen at all without registration (c) Sites in which content is dynamically generated each time they are accessed (d) Pages that cannot be accessed without using that site’s search system (e) Electronic email and chat logs That is to say, it refers to a Web that normal search engines, such as Google, cannot edit or index. Incidentally, according to Bergman, in 1994, there were already people using the “invisible web,” in the sense that it could not be searched by a search engine. However, Bergman asserted that the deep web was just deep, and not “invisible,” i.e., it could be searched with innovations. The start-up that he was managing at that time was selling this very technology. Furthermore, following this, Google formed a separate agreement with the owners of databases and started the Google Books project with university libraries, becoming involved in “deep” field indexing; thus, in 20 years, the deep web—in the sense that Bergman used it—is considered to have shrunk considerably. In this manner, originally the “deep” in deep web was simply somewhere that was deep and difficult to web-crawl, and did not contain nuances of good or evil. Despite this, “deep” is a powerful word and, as will be described later, this has led the way in 3 Hatta entrenching the image associated with the dark web as something thick and murky. Dark Net The term “dark net” became popular at virtually the same time as the term “dark web” did. There is a hypothesis that this has been used since the 1970s and although even today, in concrete terms, an IP address that is not allocated to a host computer is referred to as the dark net, the trigger for it being used as a general term as it is now was in 2002 (published in 2003), when a paper was written by four engineers including Peter Biddle (he was working at Microsoft at that time), who called the dark net as the future of content distribution (Biddle, England, Peinado, & Willman, 2003). Sweeping the world at that time was the P2P file sharing service software Napster service (started in 1999) and Gnutella (released in 2000). Operation of File Rogue started at around the same time in Japan. There were fears of copyright infringement, and in the paper written as part of the research on Digital Rights Management (DRM) and copy protection (Biddle et al., 2003), the term “dark net” was clearly being used in the negative meaning of illegal activity. Biddle et al. (2003) broadly defined dark net as “a collection of networks and technologies used to share digital content” (Biddle et al., 2003, p. 155). Based on this, it can be summarized as follows. (1) This started with the manual carrying of physical media such as CDs, DVDs, and more recently, USB memory—the so-called “Sneakernet.” (2) With the spread of the Internet, files such as music files began to be stored on one server, giving birth to the “central server” model. However, if the central server were destroyed, that would be the end. 4 Deep web, dark web, dark net (3) Files or parts of files were shared on multiple servers using Napster or Gnutella and by the shared servers (peer) communicating together—a Peer to Peer (P2P) model (meaning that if only one point of the network was destroyed, the network as a whole would survive) appeared. This P2P model was realized on the existing physical network, using technology known as an overlay network that utilizes non-standard applications and protocols. Additionally, Biddle et al. (2003) noted that as Napster had a central server for searching, it could be controlled using that. Moreover, although Gnutella was completely distributed, the individual peers were not anonymous and you could learn their IP addresses, so it was possible to track them and hold them legally responsible. In this way, measures could be taken in regard to the P2P file sharing at the time, but it was predicted that a new dark net, where these weaknesses were overcome, would emerge. Biddle et al. (2003) considered that, even if protected, it could be widely diffused via the dark net, and that the dark net would continue to evolve. They reached the conclusion that DRM was fundamentally meaningless, and that to eradicate pirated versions, official versions also needed to have a reasonable price and be convenient for customers, as well as compete on the same ground. This pronouncement put the jobs of Biddle et al. at risk (Lee, 2017). However, considering that attempts at measures against piracy through copyright enforcement have continually failed, and that currently piracy is being put to the sword by the emergence of superior platforms, such as Netflix and Spotify, such pronouncements have proven to be correct. 5 Hatta Yet Another Dark Net: F2F Possibly due to the fact that dark net is an attractive name, around the same time as Biddle et al. (2003), the term “dark net” began to be used as a general term for a slightly different technology. This is called Friend-to-Friend (F2F), 1 and as this was implemented as Darknet mode by Freenet, which is one of the main types of dark web software (to be described later), this also became known as Darknet. In this sense, Darknet, or F2F, is a type of P2P network, and the user only directly connects with acquaintances (in many cases, they have met in real life and built up trust via a non-online route). A password or digital signature is used for authentication. The basic concept behind F2F is that a P2P overlay network is constructed over the existing relationships of trust between users. This is a method in Figure 1. Topology of Darknet. A participant with malicious intent (e.g., a red one) cannot easily understand the entire network. Source: the author. 1 The term F2Fitself was invented in the year 2000 (Bricklin, 2000). 6 Deep web, dark web, dark net Figure 2.