Lately, I have been dealing with our data backup solution in South Burlington. Our old RAID system was giving us some trouble and needed to be replaced. Our beloved IT department gave us a Drobo S model RAID to replace it.
Our old RAID was a Sans Digital eight bay hardware RAID that gave us nothing but issues (and had been replaced three times). The main difference between the Sans Digital and the Drobo is the Drobo is only a five bay RAID, reducing the overall storage amount by 4TBs (or 1024GB).
I have been spending a good amount of time trying to move data from one RAID to another and because of the space difference, we had to start using compression to make it all fit. While this transfer was happening, we had to use a temporary solution for our backup services — enter stage left the LaCie 2Big dual bay RAID. It’s a hardy device indeed, as it’s tasked with keeping up with the demand of the busiest Small Dog location in the company. However, it also began getting full during the transition from the Sans Digital RAID to the Drobo S.
Since I needed to transfer the data from the LaCie 2Big over to the Drobo which doesn’t have enough space for all of it, this is where compression helps — big time. After some extensive research and IT input, we decided to move everything into individual sparse images per repair and then compress the images. Not a bad idea, but time consuming (especially when you need to compress individual pieces of data); nevertheless, it must be done.
My method for compression? Until recently, I used OS X UI and compressed the file(s) to create a .ZIP file. Our IT recommended using bzip2 instead, which is also built into OS X. After doing a little reading, it looked promising, but the process doesn’t do directories, so the manual backups needed to be packaged into sparse images. Once those backups are bzip2, the process to compress those sparse images is more complete and faster than the UI compressor.
All-in-all, it’s a several step process that should make our backups relatively easy to maintain, once I’m done with the long transferring and compressing portion. Something that is 300GB in size can take a day or two to compress, especially if the computer you’re using is having to do other tasks. For example, at the time of writing this article, I have both the Drobo and the LaCie 2Big connected to our service Mac mini by USB 3.0. This Mac mini is also serving the Drobo up for sharing across our internal service network so we can back up to it. While it’s sharing and processing new data that is being saved to it, it’s also compressing and moving data which is taxing the CPU to almost max out, so I can only do a few compressions or transfers at a time.
In short, this is an area of IT that I haven’t explored much at all. I continue to look for new techniques and tools to help speed up this process of data maintenance, but some things just require time and due diligence. I will say that this process would go much quicker on a new Mac Pro with Thunderbolt. The processors would be able to keep up with the demand for resources for compressions and file sharing that I would have nearly the amount of slowdown that I have been experiencing on this little Mac mini.
What tools have you the readers found for a task like this? I’m interested in hearing from the community. Send me your responses!