What is the dark web?

used with permission from IBM Big Data & Analytics Hub
by Tyler Carbone, Chief Operating Officer, Terbium Labs

What is the dark web? Finding consensus on the answer to that question is surprisingly challenging. Typically, studies discussing cyber threat and the dark web focus on sites hosted on hidden services within the Tor network. These sites—accessed by URLs ending in “onion”—are only accessible through Tor, an Internet overlay proxy network that users most frequently access with a special browser called the Tor Browser. Tor hidden services are outside the reach of a standard Internet browser and are set up so that both the hosts and the visitors of those sites are anonymous.

While many measures and discussions of the dark web focus on these hidden services, the dark web is bigger than just hidden services. A large number of sites customers classify as the dark web don’t live on Tor at all; they are hosted on the regular Internet and can be accessed from any browser. Often, these sites are simply hosted in countries where running, for example, a marketplace for stolen credit cards is not grounds to be shut down. For instance, many major fraud markets that claim to provide credit cards or bank accounts live, technically, on the clear web, and popular exploit sites exist steadily on even the most common top-level domains, even dot-coms.

At Terbium, we think about the dark web as anywhere on the Internet that our customers wouldn’t want to see their data appear, either for sale or for vandalism purposes. These locations range from forums on Tor hidden services to clear web carding marketplaces hosted in Eastern Europe to paste sites such as Pastebin that may be used largely for legitimate purposes.

What makes the dark web dark?

Recently, a global poll conducted by Ipsos showed that 70 percent of people believe the dark web should be shut down. The idea of shutting down the dark web is likely to struggle to gain traction. Because, at the end of the day, the dark web is simply a series of independent websites hosted on both the clear web and as hidden services within the Tor network. Even if it were possible, or advisable, to shut down Tor entirely, doing so would do little to curb sites that may not even be hiding, except by virtue of being in countries with laws that do not touch them. Thinking of the dark web as a single entity isn’t entirely accurate, and shutting it down is not as simple as flipping a switch.

The issue is further complicated by the ambiguous nature of Tor itself. In addition to hosting a variety of illicit content, the Tor network is also a tool for free speech in regions that may seek to censor citizens. As a result, in addition to a lot of the so-called dark web existing outside the Tor network, much of the activity occurring within the Tor network is largely considered positive.

Criminal activity does occur on the dark web, but it’s hardly as shady or mysterious as it’s often made out to be. The dark web, criminally speaking, is primarily a place of business, and it mirrors traditional ecommerce far more than you might expect. Major markets allow you to browse through subsections of their websites to see what kinds of offers are available. You can filter your search results by vendor rating or sort your results by price. Vendors rely on reputation, advertisements and sales—Black Friday is a big sale day among fraud dealers, while drug dealers favor Halloween or New Year’s Day—just like traditional retailers.

Shining reviews from satisfied customers are a popular way for vendors to promote their shops, and fraudulent transactions—yes, even for fraud purchases—are quickly flagged as scams to warn other customers. Forums discuss the best places to find deals, who you can trust and how satisfied buyers are with their latest orders—not unlike typical forums on the clear web.

The reality is that the dark web—some of it on Tor, some of it on clear web sites—is made up of sites and activities that pretty closely mirror offline crime statistics. The majority of it is drugs, and of what remains, much of it is financial- or fraud-related crime. At times, the dark web is a disappointingly mundane corner of the Internet. Therefore, at Terbium our approach is to provide our customers with an affordable, automated and fully private tool to monitor for the appearance of data or discussions that they wouldn’t want to be public. Precisely because of how routine the transactions on the dark web can be, not ignoring them as well as not overreacting to them is equally important.