Cloud Harmony is a company that focuses solely on benchmarking the leading cloud providers (IaaS). The goal is to give the public independent data to determine performance metrics and be able to compare the leading providers. Performance is one of the key factors that affect the choice of cloud provider today. However it has long been difficult to easily compare. Cloud Harmony changes that with both simple to use and very detailed benchmarking solutions.
We will with this post start a series of articles which will go into detail on all levels of performance. Of course comparing City Cloud to other major providers with the help of Cloud Harmony. From basic latency to how our IO compares.
General Performance Test – Cloud Computing
The basic speed test we ran compares City Cloud to Amazon EC2 and Rackspace servers in a very general way. The test goes through two parts to see how fast the servers performs in a common scenario for a regular user. A 5 Mb file is downloaded as well as a medium size website, with 19 files. The servers used are smaller which Cloud Harmony considers fairly equal and differences should not impact results negatively.
While this test also measure latency there is a more detailed test at the bottom of this post. It is however clear that latency consistently is much lower when running the test from northern Europe.
As you can read in two different and separate run tests in the two tables below – City Cloud consistently delivers the highest band widths rates as well as the quickest downloads. It is however remarkable how big the differences are at times. We ran these test several times over several days. Sometimes EC2 and Rackspace reached similar speeds of City Cloud but many times we are talking about beating out EC2 and Rackspace by a factor of 2.
In the table directly below Rackspace takes almost 14 seconds to download a 5 Mb file. Same task for City Cloud is just over 6,5 seconds.Similar for Amazon EC2. Downloading the websites 19 files was a dead heat between City Cloud and Rackspace but this time EC2 takes more than twice the time to do the same task.
Running the same test a few days earlier we experienced a similar situation. City Cloud consistently outperformed both Rackspace and Amazon EC2. Downloading the 5 Mb file took about 7,5 seconds for City Cloud whereas it took 14 and 10 seconds for Amazon EC2 and Rackspace respectively.
Downloading the medium size website of 19 files took just over a half a second in the City Cloud server. This same task took EC2 and Rackspace about twice the time.
You can easily use this same speed test. Compare the 20 plus leading providers available in Cloud Harmony’s system.
There is no secret in the fact that if you have your servers close to you there is normally less latency to worry about. It also depends on what routes your data travels and how well a provider can steer how traffic flows. For the latency test it is important to know that the servers (for this test) are located in Sweden (City Cloud), Ireland (EC2) and England (Rackspace) respectively. The test is carried out from Sweden which gives City Cloud an upper hand as it is closer to the source.
In most cases latency up to 50-100 ms is fine if you run a website or some form of SaaS. It is mostly when you have something like gaming, which is very latency sensitive, where more than 100 ms really could pose a big problem. In all cases you would of course want your service to be as quick as possible. As expected the graph below there show big differences in our latency test. While City Cloud consistently comes in at below 30 ms, Rackspace and Amazon EC2 showed significantly higher response times. Many times you see between 4-6 times longer response times from both Rackspace and Amazon EC2. Rackspace usually has a slightly better time than Amazon EC2.
It is clear that City Cloud outperforms Rackspace and Amazon EC2 in a significant way for these common tasks. Many times City Cloud is two or more times as fast as the competing services.
This is a first of many articles where we will now start diving deeper into the comparison of performance between the leading cloud computing providers. Please stay tuned and do try out Cloud Harmony’s service to see for yourself. Of course performance, or the lack of, translates into higher costs and slower service for your users. Two key criteria which makes cloud computing such an attractive choice for companies and a true paradigm shift in computing in general.