Note: I originally intended this to be one post, but it grew to be a monstrosity of a post, bordering on article-length, so I decided to divide it into two parts and post this part tonight. The other part will go up tomorrow. Today’s post focuses on the new technology and the math behind why it’s so incredible. Tomorrow will focus on more commentary and what this could do to the industry.
I’m no soothsayer, and I’m not prescient enough to know exactly what this is going to do, but Amazon Web Services just announced a new “feature” linked to S3 called AWS Import/Export.
Instead of taking hours or days to upload your data over the Internet to Amazon’s data centers, you mail them a hard drive, server, or rack of servers (up to 8U and 50 lbs.) they hook it up to their servers, load the data for you, and your data ends up on S3 automatically.
I’ve gotta note that the name of the service is currently a bit misleading. This is really only AWS Import; it’s a service where your data is loaded into AWS. There’s currently no way to get the data back out the same way, though that feature is pending. So at this time, a more appropriate name would be AWS Import. We’ll have to wait and see how long it takes them to get the Export functionality up and running. But in either case, it’s going to be fast.
How fast? Well, let’s consider this in terms of MB/sec? Here’s the math* :
Suppose you’re the average Joe (or Brian) like me and you’d like to upload 1 TB to S3. You’ve got a speedy cable (upload) connection at 1 MB/sec (8 Mbits/sec).
You’re uploading a million megabytes.
1,000,000 megabytes ÷ 1 megabytes/sec. = 1,000,000 seconds
1,000,000 seconds = 16,666.67 minutes = 277 hours = 11.6 days.
So in order to upload that one terabyte of data, you’ve gotta leave your computer on for 11.6 days of continuous uploading. That doesn’t mention the potential headaches of needing to restart failed downloads, or the sapping of your speed by your neighbors, or the kids downloading HD movies on iTunes.
Now let’s compare that with the deceptively fast method of sending data via “snail mail.”
On the fast end, we could send the 1TB hard drive via overnight mail. Then let’s conservatively estimate that Amazon doesn’t get around to transferring your data until the following evening. That gives you 48 hours of “wasted” travel time. But once they begin transferring, it goes at the speedy, but somewhat conservative clip of 100MB/sec.
1,000,000 megabytes ÷ 100 megabytes/sec. = 10,000 seconds
10,000 seconds = 166 minutes = 2.7 hours
2.7 hours + 48 hours (travel time)= 50.7 hours
So by mailing our hard drive across the country and “wasting” time in transit, we’ve actually saved 226.3 hours or 9.4 days. What a time savings! Even if you didn’t send your drive overnight, and it took a few days to get there, you’d still be ahead of trying to upload it yourself. Kinda crazy to think that, in a sense, sometimes the fastest transfer method is really one of the oldest.
*For ease of math, let’s assume all numbers are base10, meaning 1MB=1million bytes. I know…I know on most operating systems, 1 MB = 1048576 bytes, but this is for ease of math.