Too much data

Sitting on campus, next to a fountain, it’s about 75 degrees, watching a red-tailed hawk, and reviewing some documents. Then, I decided to hookup ScribeFire to my work Blog, since that’s how I author this blog, and it’s a bit more flexible than the web interface. And, I don’t have to remember to put in html paragraph breaks.

Thinking about computers and software as a production mechanism today (well, most days, but today as well). I’m a Mac bigot, and my reason for becoming so was, when I was at work, I coded, I made computers bend to my will, and added value to what they could do. But, when I used a computer outside of work, I didn’t want to administer it, I didn’t want to optimize it, and I didn’t want to have to configure it to do things. I just wanted to write, do email, design a newsletter, play a game (well, it’s a Mac, so not much gaming to be done, unless you really like old games ported from the PC). And, with my Mac, it always more or less did what I wanted.

Now, I manage my photos, my music, my home movies, build multimedia DVD’s for friends and family, write, email, twitter, manage all my contacts, scan then file all my documents electronically (so I can find them at tax time), manage all my finances, and soon, and also play all my audio (and soon video) through my house. My home computer is kinda like my swiss army knife. A specialized tool for a variety of tasks.

But, there are some drawbacks. Since I have all this vital data on my home server, I also have to keep it safe. And, there’s a lot of it. I have about 1TB of pictures, audio, and video right now. It’s all kept on a RAID store I put together. Ahh, but I’m using RAID 5, and if *two* disks fail at the same time, or if my house burns down, or if my RAID card blows up, that data could be lost. So, first, I store it on some old spare external drives. But then, there’s too much data, and I’d have to buy new drives. If I’m going to buy new drives, I don’t want to have another pile of old external drives, and a myriad of Firewire and USB tangle. So, I get another RAID store, so I can backup one to the other. And, it gives me the ability to copy all the data off, increase the size of the disks in one, and have a bigger RAID store later. But, there’s still the offsite problem. OK, for photos, I shove stuff up on Amazon S3, but for a home user, with slow uplink DSL, that’s only good for small amounts of data (well, 100GB in this case, but still). Took 5 days to get the data upstream, then a daily backup. Oh, and I still do a weekly backup to a single 500GB drive, and put it offsite. What I really need is an offsite datacenter to manage all my data securely. But, how to do that with a relatively slow internet connection?

Humm.

Mar 15 2007