11 Nov 2024

Building a 168TB RAID Storage Drive

This weekend I built my first of four 168TB RAID storage drives and thought I would share with you why I built this, how I have it set up, and what all parts I used.

I built my first RAID drive back in 2015. It consisted of four 2TB HDD drives that were striped together as RAID 0 for speed. This created a single 8TB drive that I used for editing off of. At the time, 8TB was enough for a working drive, but with 4K and 8K, things have changed in the last 10 years!

My two 16TB SSD RAID 0 editing drives.

Two years ago when I purchased my fully loaded MacBook Pro M1 Max, I built two new editing RAIDs to go with it. To build them, I used Promise Technology Pegasus32 R4 enclosures with four 4TB SSD drives setup as RAID 0 which gave me two 16TB editing drives to work off of. One was my primary editing drive, the other was a mirrored backup to that drive, using Carbon Copy Cloner to auto mirror the drives.

I use Carbon Copy Cloner (CCC) to auto backup my editing raid as well as my MacBook Pro.

How we’ve worked for years now has been we go shoot a project, we create backups of the material on portable SSDs, then when back at the office the content goes onto the primary RAID editing drive which auto copies the content to the backup RAID. Once the edit or project is complete, the files got moved off of the editing drives and onto duplicate Seagate Desktop HDD drives which is where the projects live forever after completion. I have 30 of these large Seagate Desktop drives in my office currently (15 + 15 backups), all completely full of projects.

Pictured here is all 30 of my Seagate Desktop storage drives that I’ve used for storing content over the years.

This has long been a pain point for me in my business as with this process, I’ve had to catalog all of my projects on a spreadsheet so that I knew what drives they were on, and then if I needed to access those projects for any reason (update a video for a client, use footage for my own video reels, etc.), I would have to look at the catalog to see what drives the projects were on, then grab those drives, plug them in, then work off of them temporarily to get whatever I needed. This process is not only time consuming, but also slow because the storage drives themselves are very slow. As such, I’ve been planning to build high capacity storage RAIDs for years to solve this problem and I finally got around to building my first one. I plan to build four in total which I’ll get into in a bit.

Example of how I have been cataloguing my projects over the years.

The core of my new storage RAIDs are OWC ThunderBay 8 enclosures. These enclosures have 8 bays, so you can load them up with 8 drives. For the drives themselves, I’m using the largest capacity HDDs you can currently buy, which right now are Seagate IronWolf Pro 24TB 7200 RMP drives. The total cost of 8 of these drives plus one enclosure is just shy of $5,000, which brings the cost per terabyte to only around $30 / TB. Not bad!

OWC ThunderBay 8 enclosure with 8 Seagate IronWolf Pro 24TB 7200 RMP drives.

The total cost per terabyte for this build comes out to only around $30 / TB.

To build these drives, all you need is a screwdriver and some sort of software to setup the RAID configuration. I use Promise Utility to setup my editing RAIDs, but for this new drive I purchased a copy of OWC’s own SoftRAID. To install the drives, you simply remove the front panel using the included key, then use your thumb to unscrew the housing for bay 1. With the housing, you simply slide in the first HDD drive, then use the included screws that OWC gives you with the enclosure to screw the drive into the housing. You then slide the housing back into bay 1 and lock it back in with your thumb. Once you’ve done this with all 8 bays, you can then put the front panel back on, lock it, and remove the key.

You screw the drives into the drive housings using the screws provided by OWC.

Once the drive is screwed into the housing, you slide the housing back into the drive bay.

The drive bays are marked by letters and lights, making monitoring the individual drives easy.

Once the panel is installed, you can plug the enclosure in and fire it up.

 

To fire up the enclosure, you plug it into the wall for power and then use a USB-C / Thunderbolt 3 cable to connect it to your computer. Once connected, you can launch your software of choice and configure the RAID however you see fit.

There are a lot of options for how you can setup your RAID array, with RAID 0, 5, 6, & 10 being the most common. My editing RAIDs are currently setup as RAID 0 (striped) which gives you the fastest speeds. It also allows you to use all bays together to create one big drive. With this build, if you set it up as RAID 0, you will get one 192TB drive. I tested this drive at RAID 0 just to see what the speeds were and was getting 1524.5 MB/s write and 1466.0 MB/s read which is great. For comparison, my SSD RAID which is setup as RAID 0 clocks in at around 1654.8 MB/s write / 2100.2 MB/s read, so for an HDD RAID to come in at 1524.5 MB/s and 1466.0 MB/s respectively is fantastic.

The new RAID setup as RAID 0 clocks in at 1524.5 MB/s write and 1466.0 MB/s read speeds.

My SSD editing RAID setup as RAID 0 clocks in at 1654.8 MB/s write and 2100.2 MB/s read speeds.

That said, while RAID 0 is great for an editing drive, it’s not so ideal for a storage drive. The reason is that when a drive is setup as RAID 0, if one drive fails, the whole RAID goes down. You can fix the entire RAID by replacing the drive that failed, but you will lose all of your data in the process. That’s why on my editing system I run two RAID 0 drives that mirror each other. This way if I ever lose one of the RAIDs, I have a second mirrored copy to fall back on. I can simply rebuild the downed drive and copy everything back over to it. This is fine and well for editing RAIDs, but when you’re talking about 192TB of data on storage drives, it’s best to go another route.

Setting the drive up as RAID 0 gives you 192TB of total storage space.

RAID 5 & 6 are very popular for RAID arrays because they have a built-in fail safe and can handle losing a drive without losing all of your data. With RAID 5, you are still striping your drive similarly to RAID 0, but instead of using all of the storage space available, it saves about one drive’s worth of storage for what’s called “parity”. Parity is a technique that uses extra data to protect your data in case a drive fails. In laymen’s terms, when using RAID 5 in my build, seven drive’s worth of space would be combined to create one 168TB drive, while one drive’s worth of space would be reserved for parity. With RAID 5, if one drive goes down, the entire RAID is still operable and you can replace the downed drive and have everything back up and running without losing any data.

Setting the drive up as RAID 0 gives you 168TB of total storage space.

RAID 5 also still produces relatively fast speeds. To demonstrate this, I tested this drive’s speed at RAID 5 to see what the speeds were and was getting approximately 1059.4 MB/s write and 1205.9 MB/s read which is not all that far off of what I was getting with RAID 0, but with a built-in fail safe. For another comparison, I ran a speed test on one of my SanDisk Extreme PRO V2 Portable 4TB SSD drives and they were clocking it at roughly 833.2 MB/s write / 852.8 MB/s read which shows that even at RAID 5 using HDD spinning disk drives, I’m still getting much faster speeds than what I get on a single portable SanDisk SSD drive.

The new RAID setup as RAID 5 clocks in at 1059.4 MB/s write and 1205.9 MB/s read speeds.

My SanDisk Extreme PRO V2 Portable 4TB SSDs clock in at 833.2 MB/s write and 852.8 MB/s read speeds.

RAID 6 is very similar to RAID 5, but instead of using one drive’s worth of space for parity, it uses two. With this build that would mean that I would have a 144TB drive with 48TBs worth of space being reserved for parity. You lose a lot of space doing this, but if two drives go down, you are still safe. RAID 6 is becoming increasingly popular because of its added safety, but I personally like RAID 5 over RAID 6 because RAID 5 gives me more total storage space which is important to me in a storage drive. Safety from losing my content is also important, but I’ll cover my workaround for this in a minute.

Lastly you have RAID 10. With RAID 10, you are taking half of your drives and are striping them as RAID 0 and the other half is being used as RAID 1 which means they are being used for mirroring. Essentially, half of the drive would be a working drive and the other half would be a mirrored backup. Because one half of the drive is setup as RAID 0, in theory you should be getting RAID 0 speeds with a duplicate mirrored copy on the other half of the drive. But, I tested this drive at RAID 10 and was getting 806.6 MB/s write and 570.5 MB/s read speeds which are significantly less than not only what I was getting in RAID 0, but even with RAID 5. Plus, you lose half of your storage space with RAID 10, so RAID 10 isn’t my personal choice for RAID arrays.

The new RAID setup as RAID 10 clocks in at 806.6 MB/s write and 570.5 MB/s read speeds.

So that brings us to what I chose to go with for my new storage drive and that is RAID 5. I chose RAID 5 because it gives me a failsafe that I don’t get with RAID 0, but without sacrificing two drives worth of data like you do with RAID 6. It also gets much higher speeds than RAID 10, which again only gives you half of the data to work with. So for my build, RAID 5 was ideal.

I used OWC SoftRAID to create the RAID 5 array as well as to actively monitor each individual drive.

Now, I mentioned earlier that I wanted to build four of these. You might be asking why. The reason is because for every RAID I build, I want a backup RAID. This is because if I ever did lose an entire RAID for any reason (more than one drive fails, my kids spill milk over the entire enclosure, etc.), I want to ensure I have a backup of everything. Therefore, I would need to build a backup RAID for my storage RAID.

But why then build four instead of just two? Well, for me it’s because I’m going to fill up this current 168TB RAID with all of the projects I currently have on file. I never delete anything in my business. In fact, part of what you pay for when you hire my company for a video or photo project is hard drive storage space. I charge $100/TB for every TB a project takes with a 1TB minimum per project. This way my customers are paying my storage costs, but with the added benefit that I will always have copies of everything we create. My customers can call me years after we complete a project and I can still pull up everything we’ve ever done together, this way it’s always there if we need it for any reason. In my office I currently have 12-13 years worth of projects that have never been deleted, including photos, videos, music, SFX, project files, etc. As such, I am going to fill up my new 168TB drive pretty fast.

I’m going to now need to build a second 168TB RAID for future projects, plus two additional 168TB RAIDs to back these up. This way I will have a total of 336TB worth of storage space in my office with 336TB worth of mirrored backup storage thanks to Carbon Copy Cloner. That should allow me to be good on storage for the next several years. Plus, not only will I have duplicate RAIDs for all of my storage, but all four RAIDs will be configured as RAID 5, so I will have multiple failsafes in place for all of my content.

My current setup includes two 16TB RAID 0 editing drives, plus this first of four RAID 5 storage drives.

I also plan to upgrade my editing RAIDs by keeping the enclosures, but bumping the individual SSDs up from 4TB to 8TB, thus making my editing drives 32TB instead of 16TB. This will be nice since I am often working on many projects at once. That said, this is last on my list because the storage drives with backups are needed more at this time.

One question you might have is, “Can you edit off of these 168TB RAID 5 drives?” The answer is yes! With speeds of 1059.4 MB/s write and 1205.9 MB/s read, these RAID 5 drives are giving me faster results than my single SSD field drives are, so absolutely they are fast enough to edit off of.

Why then would I still have the SSD editing RAIDs? That’s a fair question to ask, and the answer is 8K. While the new RAID 5 drives I’m building are fast enough to handle 8K without bottlenecking your computer, when you start stacking 8K timelines for multi-cam edits, you really want the fastest speeds you can get. While I typically am shooting 4K on my FX6 and FX3 cameras, I do also shoot a fair amount of 8K on my Sony A1 cameras, and that 8K is in the very taxing H.265 codec. As a result, I want to use the fastest drives possible for editing, so the SSD RAID 0 drives are still the ticket for me. That said, you could 100% get by with editing everything off of these large capacity RAID 5 storage drives if you don’t want to have separate editing drives.

My new RAID 5 HDD drive is giving faster speeds than my SanDisk Extreme PRO V2 Portable 4TB SSD drives.

One last thing I’ll bring up is the 3-2-1 rule. Most production houses abide by the 3-2-1 rule which is the recommendation that you keep three copies of all of your data, using two different types of media, and storing one copy off-site in case you have fire, flood, or theft. I agree with this rule which is why I plan to keep all 30 of my old Seagate Desktop storage drives and use them to house copies of all of my projects off-site. I plan to take all 15 of my A-drives and move them off-site immediately while the remaining 15 B-drives will be used for backing up future projects moving forward, getting moved off-site as they get filled up. This way I have three copies of everything I do, two locally in my office and one off-site in an undisclosed location. Plus, I already own all 30 of these painfully slow drives, so I might as well make use of them for something!

The 3-2-1 rule recommends you keep 3 copies of your data, using 2 types of media, and storing 1 copy off-site.

To wrap up, is building a RAID right for you? It depends! There’s a lot of storage solutions out there, from NAS systems that work on a network to just piling up desktop drives like I have been doing for years. But, if you’re looking to get everything consolidated onto one or two drives, RAIDs are a cost effective and efficient way to do it. Have a great week, y’all!

Leave a Comment