Brian @ Wongside

A couple of days ago I mentioned that I got a new 1TB hard drive and ran stress tests on it using Bst5. I figured I’d post the results as if you’re kinda geeky like me they might be of some interest.

Sequential Write Average transfer: 85.6MB/s
Sequential Read Average transfer: 86.9MB/s

RandomWrite Average transfer: 15MB/s
RandomRead Average transfer: 14.8MB/s

Half Stroke Write Average transfer: 71MB/s
Half Stroke Read Average transfer: 19.5MB/s

I’m no expert, and I frankly have no idea what a half stroke write or read is. (Does that mean the hard drive did a half stroke across the drive? I have no idea.) But that seems to be pretty compelling evidence to keep my hard drive defragged. When the drive wrote sequentially, it was nearly 6 times faster than writing to random locations about the drive, and even more so for the difference in read speed.

Anyways, just a few interesting tidbits on a Sunday for you.

I got a new hard drive from NewEgg two weeks ago.  It’s a whopping 1TB of space, and it was a steal at $75 with free shipping. Of course the bummer is that the marketing makes it sound bigger than it is.  When formatted, I lose about 70GB of space and am left with 931GB of space.  Talk about shrinkage. I remember the days when a 256MB hard drive was 256 megabytes. They didn’t have any of this funky messing with bits and bytes, decimal and base 2. Oh well. Times have changed.

In any case, I’m very excited to start filling it.  My current hard drive is 250GB (really 228GB). One of the partitions is 180 GB, and it is nearly full of pictures and other data. Between my 10.1 megapixel dSLR and 8 megapixel point-and shoot, I’ve got over 110GB of photos. I’ve also got about 30 GB of iTunes data including music and videos. I can’t imagine what I’d do if I had the 5D Mark II or the Sony Alpha A900. I’d be shooting 1080P HD video (4.8MB/sec = 289MB/min = 17,370MB/hour) and 21 megapixel twenty-four megabyte photos. Ouch!

The one thing I was worried about was if the drive came DOA (dead on arrival). If you look at the user reviews on the right of that NewEgg page, there’s a fair number of people who have complained about that. So in order to mitigate that risk, I took out a couple of insurance policies. The first one is almost literally an insurance policy. I paid extra for the extended warranty.

The second one is that I took the time to stress test the drive. Frankly, I had no idea how to do it, or what utility to use, but that’s what Google is for. The first entry on the list was for something called Barts Stuff Test. Doesn’t sound that compelling from the title, but I figured I’d give it a look. And apparently Google found exactly what I was looking for:

Bst5 (Bart’s Stuff Test v5) is a small win32 application for long term heavy stress testing storage devices. Bst5 supports testing at file and device level.

That ought to do it! And it even supports hard drives of my size and larger:

Bst5 supports very large volumes, up to 16 exabyte (17.179.869.184 Gigabyte) enough to last for at least 30 years.

Perfect!

So I downloaded it, started it up, and let it run. And run. And run. It ran for almost 5 days straight. Nothing broke during that time, so I’m assuming that we’re good.

Thanks Barts! Time to start filling this sucker up.  See ya!

Note: This is Part II of my post/article on a new service from Amazon Web Services. If you didn’t read it, you should go back to yesterday’s post and read the introduction.

The first time I heard of something like this (transferring data via courier/mail rather than over the Internet because it was faster) was back when I lived in SLO. A guy I knew at church worked for a visual effects company that happened to be working on King Kong. As you may recall, most of the effects were done by Weta in New Zealand. Well, Weta needed to offload some of the digital footage to my friend’s company in Santa Maria. They originally intended to send it over the Internet, but that would have been too slow. So instead they made him drive to LA. Huh?

It seemed crazy to me at the time that they’d make him drive the ~6 hours round trip to LA just to pick up the footage. But in the end it really was quicker for them to send the footage to LA via some high speed optical method, and then have my friend drive down to LA to pick up the hard drive. Once in possession of the fully loaded hard drive (or drives), he hopped back in his car to Santa Maria, and then unloaded the hard drives when he got to the office. It proved much more expedient than waiting days for the files to download.

In any case, we’ll see what this does to AWS and S3. It might revolutionize storage; it might change the way companies like SmugMug do business. It’ll be a heck of a lot faster for companies to perform initial backups of data to S3 with their terabytes of data. After that, they can do their incremental backups over the Internet like everyone else. And heck if they have large incrementals, companies could still send update hard drives in the mail.

I wonder if Amazon will eventually accept a sort of “drop shipped” arrangement so that providers built on AWS like Jungle Disk or SmugMug can have their customers mail drives into Amazon, and have their data loaded into the service of choice.

On the enterprise side, can you imagine a corporation sending a shipment of these or these with multi-terabytes worth of data? If you sent over two of those Sun storage boxes, that could be 96 TB of data. Based on the calculations above, that could be 13.1 days via the mail, or 3 years via the Internet. Crazy! What an invaluable savings!

I doubt this is going to be an economic, or business game-changer. (Though I could be wrong, and it wouldn’t be the first time.) But I definitely think it will have an effect–not only on AWS and their ability to cater to customers with massive amounts of data, but also on the entire cloud-computing ecosystem.

Note: I originally intended this to be one post, but it grew to be a monstrosity of a post, bordering on article-length, so I decided to divide it into two parts and post this part tonight. The other part will go up tomorrow. Today’s post focuses on the new technology and the math behind why it’s so incredible. Tomorrow will focus on more commentary and what this could do to the industry.

I’m no soothsayer, and I’m not prescient enough to know exactly what this is going to do, but Amazon Web Services just announced a new “feature” linked to S3 called AWS Import/Export.

Instead of taking hours or days to upload your data over the Internet to Amazon’s data centers, you mail them a hard drive, server, or rack of servers (up to 8U and 50 lbs.) they hook it up to their servers, load the data for you, and your data ends up on S3 automatically.

I’ve gotta note that the name of the service is currently a bit misleading. This is really only AWS Import; it’s  a service where your data is loaded into AWS. There’s currently no way to get the data back out the same way, though that feature is pending. So at this time, a more appropriate name would be AWS Import. We’ll have to wait and see how long it takes them to get the Export functionality up and running. But in either case, it’s going to be fast.

How fast?  Well, let’s consider this in terms of MB/sec?  Here’s the math* :

Suppose you’re the average Joe (or Brian) like me and you’d like to upload 1 TB to S3. You’ve got a speedy cable (upload) connection at 1 MB/sec (8 Mbits/sec).

You’re uploading a million megabytes.
1,000,000 megabytes ÷ 1 megabytes/sec. = 1,000,000 seconds
1,000,000 seconds = 16,666.67 minutes = 277 hours = 11.6 days.

So in order to upload that one terabyte of data, you’ve gotta leave your computer on for 11.6 days of continuous uploading. That doesn’t mention the potential headaches of needing to restart failed downloads, or the sapping of your speed by your neighbors, or the kids downloading HD movies on iTunes.

Now let’s compare that with the deceptively fast method of sending data via “snail mail.”

On the fast end, we could send the 1TB hard drive via overnight mail. Then let’s conservatively estimate that Amazon doesn’t get around to transferring your data until the following evening. That gives you 48 hours of “wasted” travel time. But once they begin transferring, it goes at the speedy, but somewhat conservative clip of 100MB/sec.

1,000,000 megabytes ÷ 100 megabytes/sec. = 10,000 seconds
10,000 seconds = 166 minutes = 2.7 hours
2.7 hours + 48 hours (travel time)= 50.7 hours

So by mailing our hard drive across the country and “wasting” time in transit, we’ve actually saved 226.3 hours or 9.4 days. What a time savings!  Even if you didn’t send your drive overnight, and it took a few days to get there, you’d still be ahead of trying to upload it yourself. Kinda crazy to think that, in a sense, sometimes the fastest transfer method is really one of the oldest.

*For ease of math, let’s assume all numbers are base10, meaning 1MB=1million bytes. I know…I know on most operating systems, 1 MB = 1048576 bytes, but this is for ease of math.

I had every intention of making this post earlier this week, but didn’t get around to it until today. Charlene has work this week, but I was fortunate enough to get the time off.  In my attempt to be a loving house-husband , I told her I’d make breakfast, lunch and dinner for her every day.

So here’s our menu for the week:

Monday: Blackbean Chicken and Broccoli
Tuesday: Macadamia Crusted Mahi Mahi and Roasted Cauliflower
Wednesday: Cream of Broccoli Soup with Turkey and Provolone Panini
Thursday: Spaghetti (hey, every meal doesn’t have to sound gourmet around here )
Friday: Chicken Divan

It’s my goal to post all the recipes this week, but we’ll see whether or not that happens.

In the meantime, I’m really excited about dinner tonight.  Not just because I like cream of broccoli, but because I get to play with our new stick blender.

And we’re using our panini press again. Frankly, the best part about all this cooking is that I get to use all the kitchen gadgets we got for our wedding. Yesterday, I got to open our toaster. Sunday we got to use our griddle/panini maker. Tomorrow should be pretty low key since it’s just spaghetti, but I’m looking forward to using our Corning Ware on Friday when I make chicken divan.

Charlene totally married a lover of kitchen gadgets and cooking.  Can’t wait…

Ars Technica has a great article on managing one’s e-mail inbox.  I think the only thing I’d add or clarify is to point number one on that article.

My “good friend” Mark Horstman (we’ve never actually met, but I love the Manager Tools podcast he does with Mike Auzenne, and since I’ve been a listener almost from day one, I totally feel a connection) always says that it’s ineffective to leave one’s Outlook running all day. E-mail should be checked a couple of times a day, and never first thing in the morning.

He also points out that there’s no such thing as an “urgent e-mail.”  If people really need to get a hold of you, they should pick up the phone and call.

At any rate, I’m the low man on the totem pole right now, so I don’t have much of a say as to whether or not my boss(es) or clients are sending me “urgent e-mails” that require immediate attention. But one day, when I’m a manager you can bet that I won’t send any “urgent e-mails.”

In addition to the previously mentioned articles on B&H, they also have a great new article on Adobe Lightroom 2. Lightroom is Adobe’s “digital darkroom” software. It has some of the functionality of the venerable Photoshop, but has been reduced to the essentials that professionals use when editing their photos.  I’ve never used the software (I’ve got Photoshop and Bridge), but I know some photography professionals swear by it.

Google released their own, open source web browser, Google Chrome today. I downloaded it this evening, so I’ll play with it and see what all the fuss is about.

Those Google guys always have a lot of fun, and for this release, they’ve written and illustrated a Google Comic Book detailing the how and why they decided to build their own web browswer. It was very interesting and informative, but not for the faint of heart as it is pretty technical. (Mind you, it took me about 20 minutes to read, and I’m a pretty fast reader.  It’s 38 comic book pages that are packed with information.)

Ars Technica has a review I haven’t yet read. One cool feature they mentioned in the comic book was that it always opens up to a customized, user-specific, page that has the nine most visited sites and a list of recently searched terms. The idea is that the browser should make it quick and easy for the user to get where he’s going. Nifty idea. It’s basically a browser-based portal or home page in that it’s generated client side rather than server side.

So maybe I spoke a little too soon. I’m not sure if it’s because of Firefox 3, or some new updates that have been made to Google Maps, but Google Maps has pretty much refused to work on my computer ever since installing Firefox 3. It will work from IE, but in Firefox, it just sits there and tells me “Loading…”

Update: I tried uninstalling Firefox 3 and reinstalling it. That didn’t seem to do anything. So I tried going back to Firefox 2 (in this case the latest version of 2, which is 2.0.0.14). Google Maps works again. So I’m highly inclined to believe that it’s a Firefox 3 thing. If I give Mozilla the benefit of the doubt, maybe it’s because they’ve got enhanced CSS and JavaScript rendering that Google Maps doesn’t really conform to. In that case, it’s really Google’s fault. But regardless, Google Maps doesn’t work in FIrefox 3 on my machine, so I’ll be reverting back to Firefox 2 until it’s fixed.

The Firefox Download Day 2008 Challenge is officially over. It appears that over 8 million copies were downloaded in 24 hours. That’s a pretty amazing feat. If you haven’t downloaded it, you really should.

(I won’t post the links again because you should be able to find them in the posts below.)