Why and How You Should Include the Dark Web in Your Research

Share This Post

The dark web is an area of the internet that’s not accessible using regular commercial browsers. It is most notoriously known for the illegal activity it facilitates. However, for online investigators, the dark web can be a tremendous resource for gathering information, researching their adversaries and following up on threat indicators. 

The key to accessing the dark web safely is for the analysts to become familiar with the dangers that might be lurking there and learn how to protect their organizations and missions from potential harm.

What is the dark web?

The dark web allows users to have encrypted, private access to information, websites and marketplaces. Its contents cannot be found by search engines, and it requires specific software to access. Though the various darknets that comprise the dark web employ different operational methods, the peer-to-peer sharing model allows for decentralization and anonymity among users and site owners. 

Perhaps best known for its association with criminal activity, platforms such as The Silk Road and AlphaBay have become infamous for their use of the dark web to power marketplaces of illegal goods like drugs, weapons and more. But there are less nefarious reasons to access the encrypted dark web: in many countries it allows dissidents to subvert authoritarian regimes and provides a free and open internet model that can evade censorship and provide privacy. 

The sites that make up the dark web are similar in content and style to the surface web, but the traffic is routed and shared differently, making it more difficult to shut down sites or find the original sources of content. For investigators, it can hold crucial information that would be otherwise inaccessible. To acquire these datasets, it is important to understand each area of the web, the different clients available to use them and what precautions should be taken before diving in.

Surface web vs. deep web vs. dark web

The internet most of us use daily is known as the “open web” or “surface web.” It is the traditional format of the web, composed of open pages easily accessed by search engines on any browser.

The “deep web” is the next layer of internet information. These are sites that require login or subscription services, such as academic journals, court record databases or even streaming sites like Netflix. The deep web has some barriers to accessibility while being adjacent to the surface web and is typically accessed via the same browsers.

The “dark web” is the area of the internet that can only be accessed by using special software. There are different variations available, from the most well-known, such as The Onion Router (Tor) to the lesser used, such as Freenet. Of all internet traffic, the dark web only composes a very small amount, but the dark web’s murky depths can contain vital clues to aid online investigators in their research.

Read more: Surface vs Dark Web: not as black and white as it may seem

Understanding the dark web networks 

To access the dark web, investigators need special software or clients. Each version of the dark web provides its own dataset, encryption services and risks from attempting to access it.

Tor, The Onion Router: The most widely used darknet service is Tor, or The Onion Router. It was developed by the U.S. Naval Research Laboratory and was initially designed to provide layers of encryption (hence the reference to onions) in order to anonymize communication between intelligence professionals. By diverting traffic through multiple nodes on its way to the client, the originator of files and sites can remain hidden, making them difficult to trace. The multi-layered encryption gives anonymity to both its users and service providers. Many sites are given a random URL which ends in .onion. However, like any browser, there are still ways to track activity – in Tor, the biggest weakness is the point where information travels between the exit node and the destination site – this unencrypted area presents a vulnerability to users.

ZeroNet: Lesser known darknets include ZeroNet, a peer-to-peer-based web hosting model that doesn’t use IP addresses or domains for websites. Sites are not hosted via a typical service and can only be accessed using a public key. It makes it easy to create and share sites, and almost impossible to shut them down. To access ZeroNet, researchers can use a regular browser with the application running in the background. The content is made available via BitTorrent, which shares bits of information across many peers. By distributing the information among many hosts, it makes it nearly impossible to track down or scrub all the pieces of content from the web. Unlike Tor, ZeroNet is not anonymous.

I2P, Invisible Internet Project: Another network is I2P, or the “Invisible Internet Project”. I2P focuses heavily on encrypting communication between users; and unlike Tor, it encrypts via a peer-to-peer model instead of a single thread. Access to I2P is gained via browser and an application in the background. It provides untraceable communication by establishing one-way tunnels through peers. Each client becomes a node in the tunnel, with tunnels expiring after 10 minutes. The system is referred to as “garlic routing.” One-way messages and their delivery instructions are encrypted for recipients.

Freenet: Freenet is another peer-to-peer network for sharing decentralized data. It is used in two forms – the “opennet” allows connection to any user, while the “darknet” connects only to friends. The ability to access only known contacts provides a higher degree of trust. Access is created through a backend web application and requires a key. While Freenet was originally used to circumvent censorship laws, it is now popular among cyber criminals to offload stolen data and malicious content. The traffic is routed via the closest nodes in the open net. In the darknet, routes are set up manually and only trusted parties know the sender’s node’s IP address. The inconvenience of the darknet infrastructure is outweighed by the security it provides. In this system, information stays available after the publisher has disconnected.

Each dark web service has different advantages and may be utilized for different reasons. To recap:

  • Typical uses for Tor include being used by bad actors to host websites and forums, as well as dark marketplaces to sell malware and contraband; non-malicious actors also use it to spread information in censored countries
  • Less popular though still noteworthy, ZeroNet is most often used to host websites that cannot be taken down and are viewable offline; it is most infamous for being utilized by terrorists like ISIS
  • I2P, on the other hand, is mostly used for encrypted communication; it is an alternative to Tor because it is less monitored
  • Freenet is often used as a delivery mechanism for selling breached data and malware

Benefits and risks of accessing the dark web for online investigations

All of these darknets can benefit researchers depending on their investigative targets: they can help evaluate leads, corroborate or disprove information and track data leaks. They can also provide context of how criminal marketplaces are operating and what tactics are being used to commit hacks and fraud. 

But there are risks to analysts and, by extension, their organization that need to be understood.

Many organizations may have reasonable doubts about allowing their investigators to access the dark web. While going on the dark web is not illegal per se, it is important to take steps to mitigate any risks or potential threats, especially when interacting with sites that harbor illegal activity.

Companies need to develop procedures for any employees who may be utilizing the dark web for their research. Since dark web marketplaces or forums are often monitored by law enforcement, it can be difficult to distinguish between criminal actors and good faith investigators. A good plan and solid record keeping can guide investigators while venturing into the dark web.

Most importantly, analysts need to be aware that dark web sites can often contain malware, which, if they are not careful, can infect their machines and their organization’s networks. Even the software that they need to download to use the dark web services can contain a malicious payload. What’s more, while they are browsing the darknet forums and marketplaces watching certain bad actors, their adversaries could be watching them – and if the investigator’s activity is attributed to their organization, their mission could be jeopardized, sending the perpetrators into hiding. While the dark web does provide some anonymity, there are still risks of investigators encountering malicious content and attribution. 

Read more: Why you should be on the dark web

How to access the dark web safely

Purpose-built, cloud-based solutions, like Silo for Research, provide online investigators with a safe and truly anonymous way to access the dark web, while working in sync with their companies’ IT policies and security measures. Silo offers the best protection for the analysts, and makes their job easier with a full array of tools, including managed attribution (manipulating device and connection attributes to control how they appear online), automated collections, logging web activity for compliance and audit and easy evidence sharing. 

This article was written by Authentic8. To learn more about our secure online research solution, visit our website. Or tune into NeedleStack, the podcast for online researchers. Stories of the Dark Web Diaries: dark web stories with Jack Ryder is one of our listeners favorites. 

More Articles