CDN Performance Summary 2011-2014

This is the first in what I hope to be monthly series summarizing CDN performance using data from our cloud speedtest. Because it's the first post, I've also summarized data from all tests run to date - 2011 to present.

The cloud speedtest is a tool we created to measure network latency and throughput between real users and cloud services. When a user starts the test, we use Maxmind's GeoIP database to determine their location (city, state, country) and ISP. We then store the results of every test in a database. Since 2011, we've captured results from over 20 million CDN tests from 5.1 million unique users on 27,000 different ISPs.

The Tests

The speedtest lets users run 3 types of CDN performance tests. Each test includes a brief warmup followed by multiple test iterations. We capture multiple data points from every test including mean, median and standard deviation. The metric the analysis in this post is based on is median.

CDN 101

CDN is an abbreviation for Content Delivery Network. A CDN is essentially a bunch of web servers you can host web content on. CDN servers are called points of presence (POPs), and are usually located in multiple data centers globally. A CDN speeds up web requests by responding to them using servers that are closer to the user.

For example, consider a situation where you have users in US, Europe and Australia, and a web server in the US. Your website responsiveness would likely be acceptable in the US, slower in Europe, and nearly unresponsive in Australia (where bits are traveling ~8,500 miles each way). Using a CDN in this scenario could dramatically improve performance if the CDN uses POPs on the same continent. Other CDN benefits are fault tolerance and offloading of traffic from your web servers.

Picking the Right POP

The most important task for a CDN is to speed up user requests by responding them using the fastest POP in its network for that user. Often, this POP is the one that is physically nearest the user - but this is not always true. CDNs use different methods to choose a POP when a request is received. The most common methods are:

1 IP Anycast for CDN POP routing should not be confused with IP Anycast for DNS. Most, if not all CDNs use IP Anycast for DNS

Geographical Presence

The performance improvement a CDN can provide is constrained by the number and geographical distribution of its POPs. In North America and Europe it is relatively inexpensive and easy for a CDN to provision new POPs and bandwidth. Where this becomes more challenging and costly is in less connected regions like Asia, Oceania and South America.

CDNs Covered

This table lists each of the CDNs included in this post including the POP selection methods used and continent based POP counts. The CDN name links to a more detailed profile we've created for each service including pricing.

CDN Routing Methods POPs per continent
North America South America Europe Asia Oceania Africa
AkamaiDNS/ProprietaryAkamai claims to have over a hundred thousand servers in 75 countries. Many of these servers are located within ISP facilities
Amazon CloudFrontDNS/EDNS/Proprietary14210910
Azure CDNDNS818710
CacheFlyIP Anycast/DNS/EDNS11110922
CDNetworksDNS/Proprietary1272142114
EdgeCastDNS/EDNS/Proprietary911111220
Internap CDNDNS/Proprietary702200
Level 3 CDNDNS/Proprietary19313810
LimelightDNS/Proprietary11304310
MaxCDNIP Anycast/DNS90314140

1 Some CDNetwork Asia POPs are located in China and Russia and require special permission to use

2 Some EdgeCast Asia POPs are located in China and require special permission to use

3 Limelight POP numbers and locations are estimated because specifics are not disclosed

4 MaxCDN POPs outside of North America and Europe require special provisioning and an additional monthly fee

CDN Performance - 2014 Summary

Latency

In this chart, each continent is represented by a different segment the length of which represents median latency for that region. A shorter segment represents lower (better) latency. The segments for North America and Europe are relatively similar for each CDN. Where we see larger differentiation is in the segments representing Asia, Oceania, South America and Africa with some services having much longer segments (higher latency) than others.

CDN Latency - 2014 (milliseconds)
ServiceNorth AmericaEuropeSouth AmericaAfricaAsiaOceania
Akamai56.8848.1594.8484.06114.24103.53
Amazon CloudFront61.0758.78119.09198.45105.7592.31
Azure CDN53.6356.71160.10160.83118.9594.15
CacheFly54.7162.63105.07177.78138.2176.11
CDNetworks58.9164.57169.37115.82107.9398.10
EdgeCast52.4952.58170.06184.82126.2279.06
Internap CDN66.6782.86192.73203.58179.72229.42
Level 3 CDN67.3069.31149.99100.79179.24215.00
LimeLight53.7459.97155.32173.33104.95121.35
MaxCDN56.0965.86184.99183.56255.82240.92

Small File Throughput

In this chart, each continent is represented by a different segment the length of which represents median small file throughput (1-80KB files) in megabits per second for that region. A longer segment represents higher (better) throughput. Small file testing is more sensitive to latency because the requests are brief.

CDN Small File Throughput - 2014 (Mb/s)
ServiceNorth AmericaEuropeSouth AmericaAfricaAsiaOceania
Akamai2.072.010.771.601.070.65
Amazon CloudFront2.201.560.850.531.191.04
Azure CDN (Microsoft)2.822.000.670.991.371.03
CacheFly3.883.212.110.731.972.13
CDNetworks2.382.360.530.901.251.32
EdgeCast3.372.880.910.481.631.93
Internap CDN1.571.340.740.440.680.39
Level 3 CDN2.051.910.980.831.460.47
LimeLight2.501.600.610.551.020.97
MaxCDN2.782.440.400.620.530.55

Large File Throughput

In this chart, each continent is represented by a different segment the length of which represents median large file throughput (198KB-5.1MB files) in megabits per second for that region. A longer segment represents higher (better) throughput.

CDN Large File Throughput - 2014 (Mb/s)
ServiceNorth AmericaEuropeSouth AmericaAfricaAsiaOceania
Akamai15.1113.714.411.837.395.73
Amazon CloudFront10.118.174.491.134.792.47
Azure CDN (Microsoft)15.5413.333.402.355.624.88
CacheFly18.6218.513.822.578.269.38
CDNetworks15.9717.522.611.766.196.21
EdgeCast17.4515.172.791.187.396.98
Internap CDN13.1810.652.461.514.452.28
Level 3 CDN12.329.863.461.454.221.95
LimeLight14.409.483.140.926.334.53
MaxCDN16.2216.891.891.592.932.52

CDN Performance: North America - 2011 to 2014

The remaining sections provide historical performance data based on test results collected between 2011 (when we first released the speedtest) and present.

North America - Latency

CDN network latency has consistently improved since 2011 based on our test results. Today North America latency test results for all CDNs in this post are closely grouped - around 40-70 milliseconds.

North America - Small File Throughput

Since 2012, CacheFly, EdgeCast and MaxCDN have been top small file performers in North America. Both CacheFly and MaxCDN use Anycast CDN networks in this region which likely helps to achieve better performance.

North America - Large File Throughput

Since 2013, CacheFly, EdgeCast and MaxCDN have also been top large file performers in North America.

CDN Performance: Europe - 2011 to 2014

Europe - Latency

CDN network latency has consistently improved in Europe based on our test results. Today Europe latency is tightly grouped at around 40-80 milliseconds for all CDNs in this post.

Europe - Small File Throughput

Since 2013, CacheFly, CDNetworks, EdgeCast, and MaxCDN have been top small file performers in Europe. Both CacheFly and MaxCDN use Anycast CDN networks in this region which likely helps to achieve better performance.

Europe - Large File Throughput

Since 2013, CacheFly, CDNetworks, EdgeCast, and MaxCDN have also been top large file performers in Europe.

CDN Performance: South America - 2011 to 2014

Testing activity in South America is much lower than North America and Europe. Because of this, the data provided here is based on smaller test populations and thus less robust and reliable compared to analysis in other regions.

South America - Latency

Latency improvements in South America have been less pronounced since 2011. Only 2/3rds of the CDNs in this post currently have POPs in South America. Akamai, Amazon CloudFront and CacheFly have all shown consistent latency improvement in this region with all 3 having 1 or more POPs there. Lack of a POP in South America is less detrimental than Asia or Oceania because distances to US based POPs are shorter.

South America - Small File Throughput

As the only CDN with both a regional POP and IP Anycast in South America, CacheFly is the top performer for small file throughput in South America since 2013.

South America - Large File Throughput

Since 2011, Akamai has consistently been a top large file performer in South America. During this time, Amazon CloudFront and CacheFly have expanded their networks to include POPs in this region and are now positioned with Akamai in the top 3 positions.

CDN Performance: Africa - 2011 to 2014

Testing activity in Africa is the lowest of any continent. Because of this, the analysis provided here is based on a smaller test population and is thus less robust than other regions.

Africa - Latency

Since 2011 Akamai and CDNetworks have consistently provided the lowest latency in Africa.

Africa - Small File Throughput

Akamai and CDNetworks have also been consistent top performers for small file throughput in Africa.

Africa - Large File Throughput

For large file throughput, Akamai and CDNetworks have been top performers since 2011. However, we have also observed good performance from CacheFly and Azure CDN.

CDN Performance: Asia - 2011 to 2014

Asia - Latency

There is a distinct separation between CDNs with Asia POPs and those without. For services with, median latency results generally fall around 100-140 milliseconds. Akamai has been a top performer in this region, but in the past few years it has become more competitive. CacheFly appears to be using mixed DNS and Anycast POP selection in this region (as opposed to straight Anycast in North America, Europe and South America). Our MaxCDN account did not have Asia POPs enabled, and this is likely the cause for its higher latency in this region.

Asia - Small File Throughput

Since 2013, CacheFly, CDNetworks and EdgeCast have been top performers for small file throughput in Asia.

Asia - Large File Throughput

Since 2013, Akamai, CacheFly, CDNetworks and EdgeCast have been top performers for large file throughput in Asia.

CDN Performance: Oceania - 2011 to 2014

Oceania is the continent where lack of a CDN POP presence is most pronounced in our test data. Due to its distance from other major Internet regions (~8000 miles from the US), a regional POP is necessary for a CDN to provide acceptable performance.

Oceania - Latency

There is a notable separation between those services with Oceania POPs and those without. For services with, median latency results generally fall around 80-120 milliseconds. Our MaxCDN account did not have Asia POPs enabled, and this is likely the cause for its lower throughput in this region.

Oceania - Small File Throughput

Since 2013, CacheFly and EdgeCast have been top performers for small file throughput in Oceania.

Oceania - Large File Throughput

Since 2013, CacheFly and EdgeCast have also been top performers for large file throughput in Oceania.