According to Google’s DoubleClick Ad Planner, which tracks users across the web with a cookie, dozens of adult destinations populate the top 500 websites. Xvideos, the largest porn site on the web with 4.4 billion page views per month, is three times the size of CNN or ESPN, and twice the size of Reddit.
LiveJasmin isn’t much smaller. YouPorn, Tube8, and Pornhub — they’re all vast, vast sites that dwarf almost everything except the Googles and Facebooks of the internet. While page views are a fine starting point, they only tell you that X porn site is more popular than Y non-porn site. Four billion page views sure sounds like a lot, but it’s only when you factor in what those porn surfers are actually doing that the size and scale of adult websites truly comes into focus.
THE CALCULATION
The main difference between porn and non-porn sites is
The average duration of a visit:
a) For a news site like Engadget or Mashable, an average visit is usually between three and five minutes; enough time to read one or two stories.
b) The average time spent on a porn site, however, is between 15 and 20 minutes. Then you need to factor in that most websites are predominantly text and images, while the largest porn sites push streaming video.
When you load the CNN home page, you’re talking about a couple of megabytes, and then maybe 500 kilobytes if you load an article.
When you stream porn, assuming a low resolution of 480×200, you’re looking at around 100 kilobytes per second X 15 minutes = 90 megabytes. Then you need to multiply 90 megabytes by the number of monthly visits, which is around 350 million for Xvideos. This comes to around 29 petabytes of data transferred every month, or 50 gigabytes per second. To put this into comparison, your home internet connection is probably capable of transferring a couple of megabytes per second, which is about 25,000 times smaller.
In short, porn sites cope with astronomical amounts of data. The only sites that really come close in term of raw bandwidth are YouTube or Hulu, but even then YouPorn is something like six times larger than Hulu.
STORAGE & INFRASTRUCTURE
While it obviously varies from site to site, most adult sites will probably store in the region of 50 to 200 terabytes of porn. This is quite a lot for a website, only something like Google, Facebook, Blogger, or YouTube would store more data, but in a world where 2TB drives are cheap and plentiful, this isn’t ultimately a very large amount.
Actual hardware requirements are almost impossible to derive [which aren't publicized], but in case of a large porn site we are probably talking about racks of quad-CPU servers, gigabit switches, and load balancers. Software-wise, most large porn sites will use a very-high-throughput database such as Redis to store and serve videos, and a light-weight HTTP server like Nginx to serve up the web pages.
Finally, bandwidth. Referring back to our Xvideos example (based on an Ad Planner estimate), a large porn site will have to have enough connectivity to serve up 50 gigabytes per second, or 400Gbps. Bear in mind this is an average data rate, too: At peak time, Xvideos might burst to 1,000Gbps (1Tbps) or more. To put this into view, there’s only about 15Tbps of connectivity between London and New York.
PORNO- THE REAL WORLD EXAMPLE
YOUPORN
YouPorn hosts “over 100TB of porn”, and serves “over 100 million” page views per day. Which equates to an average of 950 terabytes of data transfer per day, almost all of which is streaming video. This is around 28 petabytes per month, which means our 29PB estimate for Xvideos is on the low side; it probably serves 35 to 40PB per month.
It gets better! At peak time, YouPorn serves 4000 pages per second, equating to burst traffic in the region of 100 gigabytes per second, or 800Gbps. This is equivalent to transferring more than 10 dual-layer DVDs every second.
On the software-side of things, YouPorn’s primary data store is 100% Redis, with MySQL used as an admin tool to manage and add data to the Redis cluster. The site used to be primarily programmed in Perl with a MySQL backend, but in 2011 Perl was switched out for PHP and MySQL replaced with Redis. Nginx acts as the HTTP server, with both HAProxy and Varnish both used to load balance.
The Redis server deals with 4 Million queries per second, and between 8-15GB of data is logged every hour (visitor logs, behavior data, and so on). We’re told that this software stack should be capable of scaling up to 200 million views per day.
Sadly, we didn't get any information about the hardware infrastructure.
A single porn site accounts for almost 2% of the Internet’s total traffic. There are dozens of porn sites on the scale of YouPorn, and 100's that are the size of TechCrunch or your favorite news site.
It is probably not unrealistic to say that porn makes up 30% of the total data transferred across the internet. The internet really is for porn.
