January 7, 2006
too much data (and cpu)
Got a bit frustrated this evening with the amount of data that I've made myself deal with.
There's just so much of it. The mp3 collection and photos are so small and manageable, and
the full recordings of all of our jams and rehearsals are somewhat tolerable, but in keeping
all of the NINJAM recordings that we host, too much data gets created. And for some extremely
stupid reason, I insist on trying to keep it all. And what's the point of keeping it unless
you do it redundantly. and I don't just mean on RAID. I mean on RAID, and on another drive,
preferably one that doesnt spend all of its time over 100F, etc. So much data, bleh. Nevermind
the data that is less big, but scattered around (backing up web servers like this one, cvs
databases, etc). There's a lot of irreplaceable bits out there.
Oh yeah my other complaint was that in processing the NINJAM autosongs it ends up using alittle
of CPU on my main workstation (which has plenty of CPU to spare), but uses way too much disk
bandwidth. So I'm setting up a box just for that processing.
Back to my first paragraph, perhaps it just makes sense to say fuck it. You can't keep information
forever. Or if you can, it's not worth bothering. If only we had our 100gb blu-ray recordable
discs. if only I can hang on til then, I'll be able to put everything I will have by then on 20
of them or so. times two. plus the online (RAID) version. sigh.
I guess it's not really that much data.. I mean, everything fits within my 1.2tb array. so
maybe I should just get another, and keep them synchronized. and set up rsyncs to backup other
stuff to them. or just keep getting more firewire drives for the seocnd copies. rOn got one
of those usb->IDE cables, that might be a good way to go, and just buy a few 400gb IDE drives..