combatentropy
Veteran
Interesting. My guess is that the Samsung X5's native write speed is about 700 MB/sec. but has a buffer of about 20 GB
Nvme Enclosures
- Thread starter Peter C.
- Start date
AndreeOnline
Veteran
I did the following install today:
The long card in the middle, a Sonnet Fusion. 4x NVMe, each with a dedicated x4 lanes.
It's only half full with two 2TB NVMes in a 4TB Raid-0, so I need to settle for 6000 MB/s for now.
=)
The long card in the middle, a Sonnet Fusion. 4x NVMe, each with a dedicated x4 lanes.
It's only half full with two 2TB NVMes in a 4TB Raid-0, so I need to settle for 6000 MB/s for now.
=)
Grug
Veteran
Those “10Gbps” enclosures top out at about 700-900MB/s, it just seems to be the case with these small, bus-powered enclosures (whether you’re using TB3 or USB3.2), you take a substantial hit to the theoretical top speeds of these drives.
They’re still stupid fast regardless though, so it’s not a big issue on the whole.
I go back a while, but I remember one of my earliest drive enclosures was a 4 gigabyte SCSI based unit that cost $3200. It was external and required an internal SCSI board that was another $500.00.
Can even remember how fast it was, but you could edit Sony DV SD footage off of it.
G
Guest
Guest
Oh, thank you!
What kind of cards do you use?
I have some Kippertie Redmags, and they have a USB-C port option. Given it is a Samsung NVME SSD inside, it should be faster than using RED's mag reader, which is limited to 500mbps. But even then, I am often downing between 200Gb to 1.5TB transfers at a time.
Peter C.
Veteran
I'm not crazy about software raids, especially the non redundant ones, every drive you add your doubling the risk of failure. For reliability/cost I'd use 4tb drives and swap them when needed. That chart isn't illustrating much of a benefit.
As far as the USB 4 that's an interesting article although there's no speed benefit just allocates bandwidth better, which isn't a bad thing just nothing ground breaking and if you want something now. How long do you have to wait for a usb 4 enclosure and when they first come out they'll be selling at a premium.
I'm not sure about is the benefits of the faster enclosures if your using mainly to do edits and not doing large sustained file copies. If you look at random read writes for small files all these drives don't use a lot of bandwidth.
btw I updated my first post with list of the enclosures. The OWC is the best buy at $68 for thunderbolt.
As far as the USB 4 that's an interesting article although there's no speed benefit just allocates bandwidth better, which isn't a bad thing just nothing ground breaking and if you want something now. How long do you have to wait for a usb 4 enclosure and when they first come out they'll be selling at a premium.
I'm not sure about is the benefits of the faster enclosures if your using mainly to do edits and not doing large sustained file copies. If you look at random read writes for small files all these drives don't use a lot of bandwidth.
btw I updated my first post with list of the enclosures. The OWC is the best buy at $68 for thunderbolt.
Last edited:
At my level of backups, I’m not really scared of data loss. Especially considering these are SSDs with no moving parts. They’ll generally die quickly or last for quite a long time. Also, there’s nothing inherently more risky about software RAID. The failure factor of RAID 0 is there whether you’re using hardware or software. On a hardware RAID, until you spend big bucks, you’re adding the additional failure point of the controller itself.
Anyway, my video RAID is backed up 3-4 times per day to a local drive and nightly to a 10GbE NAS that’s running RAID 6 on spinny disks. That NAS is backed up to another NAS nightly. The backup NAS is mirrored to a cloud storage provider as well.
At worst... I lose a few hours’ work. Likely not even close to that. I’ll take that risk to not have to screw around with “which drive was that on again?”
Also, bare 4TB NVMe sticks are still in the $700 range. My entire 4M2 setup was $1200 or so. So it’s faster and cheaper than a pile of 4TB drives... and it doesn’t dead-end one of my limited number of Thunderbolt ports, has real cooling, and it’s own power supply.
Horses for courses, but I’m pretty happy with my 4M2 setup. I probably don’t need the speed for editing, but it sure is nice dumping a pile of CFExpress cards.
Anyway, my video RAID is backed up 3-4 times per day to a local drive and nightly to a 10GbE NAS that’s running RAID 6 on spinny disks. That NAS is backed up to another NAS nightly. The backup NAS is mirrored to a cloud storage provider as well.
At worst... I lose a few hours’ work. Likely not even close to that. I’ll take that risk to not have to screw around with “which drive was that on again?”
Also, bare 4TB NVMe sticks are still in the $700 range. My entire 4M2 setup was $1200 or so. So it’s faster and cheaper than a pile of 4TB drives... and it doesn’t dead-end one of my limited number of Thunderbolt ports, has real cooling, and it’s own power supply.
Horses for courses, but I’m pretty happy with my 4M2 setup. I probably don’t need the speed for editing, but it sure is nice dumping a pile of CFExpress cards.
Last edited:
Delkin POWER 512GB CFExpress in a Sonnet dual Thunderbolt 3 reader.
Peter C.
Veteran
If you thought software raids and ssd were reliable then you wouldn't need to do backups you're doing.
AndreeOnline
Veteran
I think that is a bit of a stretch. Not worrying about failure and truly believing in a device's reliability shouldn't stop anyone from taking basic precautions.
The loss of data is unacceptable, so you make sure that doesn't happen.
You trade speed for increased (but still small) risk and use backup as you would anyway.
Peter C.
Veteran
My point was that as reliable ssd are, running non redundant ssd raid is risky without a backup. The other point was every additional drive you add to a non redundant raid you are increasing the risk. In my experience it’s preferable to avoid any type of raid if possible. They’re more expensive, complex, more to go wrong and when they do go wrong are a headache. People like to often say anecdotal things like ssd are more reliable than spinners because they don’t have moving parts, where I believe their failure rates to be similar.
combatentropy
Veteran
Oh that's right! If you have a RAID-0 of 4 drives, and one drive fails, then the data is lost for all 4 drives.
There are 7 kinds of RAID setups. This specifically applies to the one called RAID 0, which is what I understand y'all to be considering.
'RAID 0 (also known as a stripe set or striped volume) splits ("stripes") data evenly across two or more disks, without parity information, redundancy, or fault tolerance. Since RAID 0 provides no fault tolerance or redundancy, the failure of one drive will cause the entire array to fail; as a result of having data striped across all disks, the failure will result in total data loss.' --- Wikipedia
There are 7 kinds of RAID setups. This specifically applies to the one called RAID 0, which is what I understand y'all to be considering.
'RAID 0 (also known as a stripe set or striped volume) splits ("stripes") data evenly across two or more disks, without parity information, redundancy, or fault tolerance. Since RAID 0 provides no fault tolerance or redundancy, the failure of one drive will cause the entire array to fail; as a result of having data striped across all disks, the failure will result in total data loss.' --- Wikipedia
puredrifting
Major Contributor
Yep. I am with JB. As long as you have good back ups, it wouldn't sink you if you had to move to another RAID or rebuild it in case of failure. RAID 0 is always a bigger risk but RAID 5 w redundancy requires a larger capacity and substantially
more expensive RAID to get the same capacity. I have three other spinning drive RAIDS so if the NVME RAID goes down, I can rebuild and continue on with one of the other RAIDs. Not that big of a deal, I transfer editing projects
onto and off my RAIDs weekly, they are JUST for editing mainly.
That's silly. I back up single drives just as religiously and stringently... if the data that's on them warrants it. One of my drives is backed up to over 6 different places including two redundant cloud providers and a spare NAS at a relative's house across the state. The primary version is a single SSD. Does that mean I think a single SSD is less reliable than a RAID 0 of four SSDs? No, not at all. It's that the data on that drive is very precious and it would be really really bad if I lost it.
Here's the thing... with my current RAID 0 in the 4M2 and it's associated backups... if the RAID were to fail, I could switch immediately to working off of a copy of that drive that was at most a few hours old. It would suck to lose an hour or two of work, but I'd have near zero downtime. So, I'm not super worried about the overall reliability of the RAID... because even with four drives it's probably still pretty damn reliable. Especially with no moving parts in the member drives.
If I have my stuff spread across multiple 2TB or 4TB external drives, in order for those to be backed up reliably and regularly I need to have them hooked to something that's capable of backing them up. So, I'd have to chew through 2-4 Thunderbolt ports (my iMac Pro has 4 total) to have them all connected. Also, very few affordable external SSDs have a pass-through port to connect another device. So, between my two external monitors and my two 4TB SSDs, my Thunderbolt capabilities are completely full up. That sucks because I have a Thunderbolt audio interface, external PCIe enclosure with a SAS card in it for my LTO drive...
Oh, you say... just connect them as I need them to save Thunderbolt ports. Sure... but that means that they have to be hooked up when the backup tasks run OR vice-versa. Since I don't want to run backup tasks on my working drive while it's being used for editing, that means I have to remember to hook up each drive and let a backup task run on it at regular intervals. Since that likely won't be several times a day for each drive, that means my backup frequency just dropped down to maybe once a day... more likely less than once per day. The best backup solution is one that's transparent and requires no human interaction. This really fails that test.
The way I run my data storage, that RAID failing is really not a huge concern. The whole drive is hot-replaceable with a copy. If those two both go down together, I can rebuild from the NAS. If for some reason the whole house burned down, I can pull a copy from cloud. If all that fails, I still have my footage (it's written to LTO tape and stored off-site). If I lose all of those... chances are pretty good that editing drive going down isn't my biggest problem.
Peter C.
Veteran
Is it ok for me to not to find value in an external nvme raid and to caution those reading the risks of a raid 0? Normally in a raid 0 with spinner drives there is a speed advantage but with ssd they already exceed the bandwidth of thunderbolt. IMHOP its an expensive solution just to get one large volume. For me 1 2tb or 4tb nvme drive that gets backed up and projects get archived as they are completed makes more sense. Sounds like it works for you so let's leave it at that.
Last edited:
High1ander
Well-known member
I'm with jbregar on this one.
I myself have, depending on material, but for important stuff:
NAS backup
Single Ext HDD locally and on another continent (for a feature)
Raid 0 with 4 SSDs for "online"-work on one workstation
Raid 1 with 2 mechanical HDD for "online"-work on another workstation
Basically everything is duplicated on 3 different physical places, and stored on minimum 5 different mediums. So as long there isn't an Extinction event stopping the earth from spinning it is pretty safe. But this is all for "working on the project", storing for later will be LTOs, single HDDs and maybe cloud. Archiving is a different beast.
I myself have, depending on material, but for important stuff:
NAS backup
Single Ext HDD locally and on another continent (for a feature)
Raid 0 with 4 SSDs for "online"-work on one workstation
Raid 1 with 2 mechanical HDD for "online"-work on another workstation
Basically everything is duplicated on 3 different physical places, and stored on minimum 5 different mediums. So as long there isn't an Extinction event stopping the earth from spinning it is pretty safe. But this is all for "working on the project", storing for later will be LTOs, single HDDs and maybe cloud. Archiving is a different beast.
Sure you can find value or not in whatever you want.
I concede (though it was never in question) that RAID 0 is riskier to the entire dataset than four independent volumes. I'm kind of assuming if someone's spending thousands of dollars on storage, they already know that you need to back up an editing RAID 0 though.
As for price... the cheapest individual 4TB SSD sticks I could find are right around $700. Add an enclosure they're right over $700 each. If we're talking off-the-shelf 4TB single-drives with enclosure, they're even MORE expensive at $1000 (OWC Envoy) up to $1300 (Glyph Atom Pro).
I'd need two of them to equal the 8TB I have in my 4M2. That's $1400 if I put them together myself and $2000 for an off-the-shelf solution. My 4M2 setup cost me right under $1200. So, which is the "expensive solution" again?
Even if we're talking about 4x2TB disks then it will end up right around $1000 total. I'll gladly pay $200 for the other benefits of a single volume. Especially since I have a single project that's around 3TB by itself.
In addition to the price advantage, my setup also means it can be hooked up at all times so automated backups can run. It uses fewer Thunderbolt ports. It has a built-in DisplayPort port so I can run my second monitor off of it. It has a fan to cool the drives, because they do get warm when you're using them hard.
Last edited:
Thomas Smet
Veteran
A single NVMe can fail as well which puts you pretty much in the same place of needing a backup no matter if you have a RAID 0 or if its a single NVMe. I really think of a RAID 0 as a single drive that has more individual parts that can fail killing the entire drive. If I treat it like a single drive that can fail and lose everything then there is really nothing extra to worry about. It either fails or it doesn't just like a single drive.
The beauty of the RAID 0 is if one drive fails you only have to pay to replace one affordable HDD and transfer back over from the backup. If a large NVMe fails you have to pay to replace the entire thing and still transfer from the backup. For example lets say you have a 8 TB drive. A 8 TB SSD would cost around $1,200 to replace if it fails. A single 1 TB HDD in a 8 drive RAID 0 would only cost $50 to replace the failed drive. Even a 2 TB HDD in a 4 drive setup for a really solid drive is only $100 to fix. Even if talking about a 4 TB SSD that's still $600 to replace it vs a single 1 TB hDD for $50 in a four drive RAID 0.
Plus flash storage wears out over time. They have a ton of write cycles but the thing about video editing is we tend to fill up those write cycles constantly, erase and repeat. A system drive with Trim will spread the write cycles around so a 512 GB SSD that only uses 64 GB will spread that 64 GB across the write cycles before it uses them again. We tend to not fill up and erase system drives daily or many times a day so they can last for many years. A video SSD however is filled and wiped a lot. You may find yourself replacing that $1,200 8 TB Ssd a lot sooner than you think. All depends how many video projects you do and how often that 8 TB is filled up. If you use a smaller SSD like 2 TB you will eat up the write cycles all that much faster and the write cycles tends to increase as the drive gets larger.
The whole "SSDs have a limited lifespan" thing is largely overblown. Even in a video production environment. The bigger the SSD, the more terabytes written (TBW) it is expected to survive... and with modern controllers and reserve flash, the lifespans are significant.
My 4M2 is filled with 2TB HP EX950 blades. Each blade is expected to survive 1400TBW based on wear. Since a RAID 0 stripes across the drive set, that means this 8TB RAID can be reasonably expected to survive 5600TBW before the drives are shot. The drives in there have a useful life of ~5 years IMO... which means if I'm writing less than about 1100TB per year to the drives, they'll probably survive (based on wear) past the point where I'd replace them anyway. I'd certainly replace spinning drives sooner than 5 years.
So, 1100TB/yr. That means I'd have to erase and re-write my drive completely about every 3 days. That is just simply not gonna happen... even with a heavy video editing workflow.
Thomas Smet
Veteran
I agree it isn't nearly as much of a concern as some make it out to be. It is something to think about however and some cheaper drives may not last as long. Its important to invest in good flash storage so it lasts longer and not assume all flash storage is created equally. Plus a HDD will likely fail long before a SSD wears out anyway. Intel 660 drives for example are enticing due to their cost but they tend to slow down a lot faster than more expensive drives and eat up their write cycles a lot sooner. You get what you pay for really does apply to flash storage.