Comparison and Analysis of Managed DNS Providers

Introduction

This blog post is the culmination of year's effort researching and developing methods for analyzing and comparing managed DNS services. During this time, we provided access to our analysis and solicited feedback from any DNS provider that would listen. We would like to thank our contacts with UltraDNS, Cotendo, Amazon Web Services and NetDNA for the feedback they provided. As always, it is our intent to provide objective, fair and actionable analysis. We have not been paid by anyone to conduct this testing or write this post. If you have feedback, we'd like to hear it.

This blog post is also intended as an introduction to a new monthly report entitled State of the Cloud - DNS available for purchase on our website. The full version of the August 2012 edition of this report is available for free in pdf format (10MB). Future editions of this report will include both free and premium editions where the free version will include everything but some of the more advanced content such as marketshare analysis. This blog post provides an introduction to and summary of the August 2012 edition of this report.

View Full Report (PDF)

What is DNS?

Domain Name System or DNS for short, is the part of the Internet that lets users access websites (and other Internet services) using easy to remember words and phrases called hostnames like amazon.com or google.com. Without DNS, users would be required to use cryptic numeric-based identifiers called IP addresses (e.g. 72.21.194.1). When a user types a hostname into their browser address bar, one of the first steps undergone is to translate the hostname to an IP address. This translation process involves querying a DNS server that has been assigned responsibility for that hostname. These are called authoritative DNS servers. If the authoritative DNS server is not accessible, the browser will be unable to resolve the IP address and display the website.

DNS server software is freely available and is not overly complex to setup or run. The core functionality of a DNS server is simpleā€¦ the translation of hostnames to IP addresses. It requires only minimal bandwidth and CPU resources to maintain a DNS server. Many organizations host their own DNS servers without much effort.

Managed DNS

Managed DNS is a service that allows organizations to outsource DNS to a third party provider. There many reasons why an organization may elect to outsource DNS hosting... here are a few:

Whatever the reasons are, managed DNS is a fast growing sector in the cloud.

Enterprise versus Self Service

Managed DNS providers can be generally divided into two categories:

After speaking with multiple enterprise providers, it is our impression that they generally consider self service providers as non-competitors targeting a different customer demographic.

Comparing Managed DNS Services

Comparing DNS services is not as simple as running a few benchmarks and calling it good. There are multiple criteria where comparisons may be drawn. In this post, we'll present some criteria we believe to be relevant, the techniques we have used to implement them, and the resulting analysis. The following DNS providers are included:

End-User Performance

There are many factors that affect DNS performance. When a user types a hostname into a browser address bar the path taken for resolving that hostname to an IP address varies between users. Generally, the first point is the user's ISP DNS resolver. These are specialized DNS servers that cache lookups and are used only for DNS resolving. If this DNS server does not have an answer, it will query the next level up (often another ISP DNS server), and this process continues until the authoritative server is queried. This process is referred to as a recursive DNS lookup. From an end-user's perspective, DNS performance is the total lookup time including the entire recursive chain. To improve performance, managed DNS providers typically deploy mnay DNS servers globally and use an Anycast network to reduce latency and thus reduce lookup times. A well designed DNS network will generally provide better and more consistent performance for end-users globally. However, because DNS lookups are almost always cached, after the first lookup, DNS generally has little impact on website performance.

To measure end-user DNS performance, we developed a browser based test that measures the time difference between downloading a small (4 byte) file using both cached and non-cached hostnames. We use a special type of DNS record called a wildcard name that allows the test to generate random hostnames that are guaranteed to require an authoritative DNS lookup. Multiple measurements are taken during each test and the median is recorded. We've run this test using thousands of unique users globally every month. We also utilize a Geo IP database to determine where the user running the test is located which allows us to generate the region specific analysis for each DNS service show below.

The following charts display the results of our end-user DNS performance analysis for July 2012. The bars in each chart represent a managed DNS service and a specific geographic region. The vertical axis is the median DNS lookup time in milliseconds (1000 milliseconds equals 1 second) for all users and tests in that region (typically hundreds or thousands of unique users). The line spanning horizontally on the chart represents the median lookup time for all regions. In general, DNS lookup times in the 50-200 millisecond range are very good.

Synthetic Performance

Another method of measuring DNS performance is to query authoritative DNS servers directly (bypassing recursive lookup chains). This method utilizes test agents located in data centers and provides a more repeatable, consistent and controlled testing environment. However, because these tests are conducted by just a handful of servers located in data centers, and bypasses recursive lookup chains, it is less relevant to the performance an end user would experience. For our testing, we are utilized a network of 110 servers (57 US, 28 EU, 25 AsiaPAC and other locations) conducting tests every 5 minutes. The charts below use the same format as the end-user charts.

Availability

Managed DNS provider networks consist of many DNS POPs (a single POP consists of 1 or more DNS servers) distributed globally. When used in combination with Anycast routing, these DNS servers can be configured to automatically failover to other POPs. Additionally, the DNS protocol can include more than one authoritative server and incorporates retry logic such that DNS clients will try to query multiple DNS servers until a response is received. These factors enable DNS to provide continual availability as long as at least 1 DNS POP is available.

To calculate availability, we utilized the 110 monitoring agents querying provider DNS servers every 5 minutes throughout the month. Every authoritative DNS server was queried during each test. Two metrics are were captured... availability of at least 1 DNS server and availability of all DNS servers. The former is the more important as it reflects what an actual user would generally experience. All providers offered very high availability. The following table represents the results on this analysis for July 2012:

DNS Propagation Latency

Another comparison criterion is the time required to push a DNS record update to a provider's DNS network. We refer to this metric as DNS Propagation Latency. This criterion is more relevant to an organization making frequent or automated DNS updates, where those updates are critical to some functionality. To measure DNS propagation latency, we used 110 global test agents. At the exact moment a DNS change was submitted, those agents were instructed to directly query provider DNS servers and record the amount of time that change took to complete. Testing was conducted for both primary and secondary DNS (secondary DNS is a replication-only service). The tables below show the results of this analysis (Propagation Latency is a median value for all 110 test agents and all provider DNS POPs):

DNS Provider Marketshare

Marketshare is a good indicator of the robustness of a provider DNS network. Providers that manages DNS for popular websites have a demonstrated capability to effectively support very high DNS volume.

To capture DNS provider marketshare we determined which DNS providers are used for the top 10,000 Alexa sites (Alexa publishes a list of the most popular websites). Some DNS providers allow customers to mask their DNS servers using custom hostnames (a featured referred to as vanity DNS servers). To include these, we correlate vanity DNS servers using IP address matching (matching class C IP addresses to those of known provider DNS servers). To verify correlations, we perform authoritative lookups of the hostnames using actual provider DNS servers.

We also track provider marketshare distribution changes by comparing provider marketshare this month to that of the previous month. The following are the results of this analysis for the month of July 2012:

Alexa Top 1,000 DNS Marketshare - Aug 2012
Alexa Top 10,000 DNS Marketshare - Aug 2012

Top 20 Provider Alexa 10,000 Changes - July 2012

Because the makeup of the top 10,000 Alexa websites changes from month to month, the marketshare change analysis may not represent actual provider change. The change metrics above, represent the number of actual confirmed provider changes between July 1 and Aug 1 2012. During this time, we observed significant usage growth for both Route 53 and Dyn.Top 20 Provider Alexa 10,000 Changes - July 2012

Confirmed Alexa Top 10,000 Changes - July 2012

Confirmed Alexa Top 10,000 Changes - July 2012
Because the makeup of the top 10,000 Alexa websites changes from month to month, the marketshare change analysis may not represent actual provider change. The change metrics above, represent the number of actual confirmed provider changes between July 1 and Aug 1 2012. During this time, we observed significant usage growth for both Route 53 and Dyn.

Features

Advanced features are one of the biggest ways that DNS providers distinguish their services. The following is an overview of a few common features and their associated support with each DNS provider:

Health Checks - DNS Failover

DNS Health Checks - Failover

DNS failover involves dynamic DNS resolution based on the availability of target hosts. These hosts are monitored continually by DNS providers using ICMP (ping) or more advanced methods such as HTTP content monitoring. If a primary target host fails a health check, DNS resolution automatically change to a backup target host.
UltraDNSDynCotendoRoute 53DNS Made EasyeasyDNS
YesYesYesNoYesYes

Health Checks - DNS Load Balancing

DNS Health Checks - Failover

Like DNS failover, DNS load balancing monitors target hosts. However, with load balancing there target hosts are not considered active or failover - instead, all hosts receive a even (or weighted) distribution of traffic. If a target host goes down, the DNS service will stop sending traffic to it by ceasing to resolve that IP address.
UltraDNSDynCotendoRoute 53DNS Made EasyeasyDNS
YesYesYesNoNoNo

Location Based Routing (Geo IP)

DNS Location Based Routing (Geo IP)

Location based DNS routing allows a DNS hostname to resolve dynamically depending on the geographic location of the user (or more specifically, location of the user's DNS resolver). To accomplish this, the geographic location is determined using Geo IP databases like Neustar IP Intelligence (formerly Quova) or MaxMind. This location is then run through custom, user-defined DNS rules that may affect the IP address the hostname resolves to. For example, a hostname might resolve to a server in Singapore for users in Asia, and a server in the US for others thereby improving webpage load times.
UltraDNSDynCotendoRoute 53DNS Made EasyeasyDNS
YesYesYesYes1NoNo
1Route 53 provides a unique feature called Latency Based Routing, where DNS resolves to a target host with the presumed lowest latency to the end user (the target host must be in one of 7 AWS data center regions)

Zone Based Routing (Anycast)

DNS Zone Based Routing (Anycast)

Zone Based Routing is functionally similar to Location Based Routing, but instead of using a Geo IP database to determine the user's geographic location, it uses the DNS server that the user is querying. In Anycast networks, this DNS server will typically reside in the same general geographic region as the DNS client. In practice, this limits the number of location specific rules to the number of Anycast zones in the provider's network (typically in the single digits). For example, Dyn's network consists of 7 Anycast zones, thus allowing up to 7 location specific target hosts. Because of this, zone based routing is more limited relative to location based routing. Additionally, zone based routing can be problematic in some geographic regions where Anycast networking is less predictable (i.e. Asia) or when POPs are taken down for maintenance.
UltraDNSDynCotendoRoute 53DNS Made EasyeasyDNS
NoYes (7 regions)NoNoYes (4 regions)Yes (4 regions)

DNSSEC

DNSSEC

DNSSEC (Domain Name System Security Extensions) is a specification for securing DNS records. DNSSEC was designed to protect clients from forged DNS responses by digitally signing DNS responses. By checking the digital signature, DNS clients can verify the authenticity of those responses. Usage of DNSSEC is growing, but due to complexity and lack of support is still relatively low.

Provider or User Managed DNSSEC

Manual generation and management DNSSEC certificates can be cumbersome. Some providers simplify this by generating and deploying certificates automatically thereby eliminating many of the administrative complexities.

UltraDNSDynCotendoRoute 53DNS Made EasyeasyDNS
Yes (provider managed)Yes (provider managed)NoNoNoYes (user managed)

Pricing

Enterprise DNS providers, generally do not disclose pricing publicly. Part of this likely has to do with the negotiable nature of their pricing. In order to determine pricing for these services (where it was not available publicly), we contacted each provider for a pricing quote. Actual pricing may vary depending on a customer's ability to negotiate.

DNS Query Pricing (monthly)

Provider1 million10 million100 million1 billion10 billion
AWS Route 53$0.50$5$50$500$2,750
UltraDNS$50-$1951$865-$1,2002$2,200-$3,0002$5,1252$17,5002
CotendoNot offered3$500$1,000$5,000$10,0000
Dyn$604$295 (10 QPS)$600 (40 QPS)$2,250 (400 QPS)$5,495 (4000 QPS)
DNS Made Easy$2.505$55$2187$1,5207$7,3707
easyDNS$9.956$20$2008$2,0008$20,0008
  1. $50 plan includes US and EU DNS POPs only
  2. UltraDNS discounts query pricing by up to 50% when bundled with advanced features list below. The prices provided here are based on some bundling
  3. The lowest usage tier for Cotendo DNS is 10 million queries/mo
  4. Dyn Enterprise DNS Lite - includes 1.2 million queries/mo
  5. Must prepay annually
  6. Enterprise Plan - Pricing is for 5 million queries/mo
  7. Based on $1500/yr corporate membership (includes 50 million queries/mo)
  8. Based on Enterprise plan and published overage rate - discounts may be available

Advanced Feature Pricing (monthly)

ProviderHealth Checks - FailoverHealth Checks - Load BalancingLocation Based Routing (Geo IP)Zone Based Routing (Anycast)DNSSEC
AWS Route 53NANA$0.25/million queries10NANA
UltraDNS$2251$5632$5003NAIncluded
Cotendo$1304$1304IncludedIncludedNA
Dyn$1005$2005$4006$2006Included
DNS Made Easy$0.427NANA$558NA
easyDNSIncluded9NANAIncluded9Included
  1. UltraDNS refers to this feature as Sitebacker. Price is based on 10 million query bundle pricing (25% discount) with 2 monitored target hosts. Each additional target host is $113/mo ($150 without bundle discount) up to 5, then $90 up to 10, then $68
  2. UltraDNS refers to this feature as Traffic Controller. Price is based on 10 million query bundled pricing (25% discount) with 3 monitored target hosts. Each additional IP is $188/mo ($250 without bundle discount)
  3. Pricing based on 10 million query bundled pricing (25% discount) with up to 5 target hosts. Each additional target host is $100/mo
  4. Per hostname with up to 10 monitored target hosts
  5. Priced based on # of monitor samples per month, $3 per 1000 samples - pricing provided is based on 3 target hosts, 1 monitoring node and 3 minute monitoring intervals. Pricing reduces to $0.30/1000 for 10 million samples/month
  6. Per hostname
  7. Must prepay annually - the Business and Corporate plans include 3 and 10 failover hostnames respectively and up to 5 monitored target hosts using 2-4 minute monitoring intervals
  8. Requires $60/yr business plan. $1500/yr Corporate plan includes 1 geo-targeted hostname at no additional charge
  9. Only 15 minute monitoring intervals are supported
  10. Query surcharge for latency based routing - only target hosts in AWS data centers are supported. Query pricing reduces to $0.125/ million queries for volume above 1 billion queries/mo

Summary

There is a lot to consider when comparing managed DNS providers. We've included what we believe to be a few relevant, objective and comparable evaluation criteria in this post and our new DNS report (PDF). Organizations should consider the criteria that are most relevant to them when evaluating providers. If advanced features are needed, an enterprise provider may be the best choice. If just plain DNS is needed, a self service provider like AWS Route 53 may be a better and more cost effective service. The most important factor is to make your selection based on relevant and objective criteria, not marketing spin.

View Full Report (PDF)