External SAS connected arrays function in much the same way as SATA based ones, but with a few advantages, that usually come at a significantly higher cost.  SAS is a full duplex interface, and the command set is based on SCSI instead of IDE, allowing higher performance and throughput.  More expensive SAS arrays also support multipath signaling, for greater redundancy in the supporting electronics. (As opposed to the redundancy provided at the disk level by RAID configurations)  SAS also supports much longer cable lengths, up to 10 meters or 30 feet.  This can be advantagious for quiet video editing rooms, since the disk array, which is usually the loudest part of the system, can be located farther away from the users.

A number of vendors have now begun offering external arrays that interface with the host workstation via a direct extension of the PCIe bus.  This allows all of the RAID functionality to be contained within the array, and gives full speed access to the data as if it was contained within the machine.  Among the advantages of removing the RAID functionality from an internal add-on card, are that it can be attached to a laptop via an ExpressCard, which uses the same signaling protocol as PCIe, and that with addition of a few cheap pass-thru cards, an array can easily be moved between systems.  This is definitely not a hot swappable solution, since it accesses the PCIe bus directly, which is initialized at bootup on most systems.  But if your main edit system has a total OS meltdown at a critical point in your project, it should be much easier to access your data from a different system than if you needed to reinstall the PCI SATA RAID card somewhere else, and allow you use your laptop as a backup edit system in certain instances.

Fibre Channel is by far the most expensive option.  Every part of the system is more expensive, the PCIe HBA cards, the fiber cables, and the disk array controllers.  On the otherhand, Fibre Channel offers capabilities that none of the other storage options really do.  It is a hot swappable interface, running on fiber cables that can extend access thousands of feet if desired, and can easily be networked and shared.  Devices can be connected directly together, shared in an Arbitrated Loop, or all attached to a central fibre switch for simplified management.  It is an efficient and low latency interface, and is available in speeds of 1,2,4, or 8Gb per second, and multiple channels can be combined for higher performance.  Higher speed devices are usually backwards compatible with older hardware, similar to the way ethernet works, allowing you to upgrade your storage network one piece at a time.

Choosing the right storage solution depends on your immediate media needs, your available budget, and the direction you anticipate growing in the future. SATA based solutions offer all of the speed you could need if scaled large enough.  SAS can offer similar performance in a smaller package, but at a higher cost.  Sharing data beyond gigabit network speeds requires a storage system that can interface with multiple computers, but that comes at a significantly increased initial cost.  Investing in Fibre Channel storage is usually only worth the expense if you anticipate the need to share your data on a SAN, either immediately or at some point in the future.  I will examine a few popular shared SAN options in my next post.

Dual 3.0 Ghz 5365 Clovertown Intel Xeon Processors (8-Cores)
8GB Ram (Once I get a 64bit OS)
QuadroFX 4600 Graphics Card (To power my 30″ LCD)
2x300GB 15k RPM SAS Drives (125MB/s each)

The XW8400 is currently one step behind the new top of the line XW8600, but the PCI-X slots, wider 3rd party support, and much lower price, weighted in its favor.  I will be doing some tests and benchmarks in the near future, so stay tuned for the results.  I am also curious to see if the Quadro4600 properly allows fullscreen overlay output in Prospect2K’s CineformRT Premiere mode, fixing the Geforce8 overlay problem.

I got a ridiculously good deal on everything, since I waited nearly a year for the perfect opportunity, but I now have a few extra parts that I am not sure how to best utilize:

I already had all the disks I needed, with a 74GB 10k rpm SATA drive for OS and 4x 500GB SATA RAID5 for Data. My Raid5 is only getting 60-80MB/s with the integrated Intel Matrix storage controller, which is less than I was hoping for, but my data will be secure. If I continue to use the Raid5, I won’t have any place for my new, SAS drives worth $1000 bucks apiece. Alternately I could replace my Raid5 with a single 1TB disk for quantity storage with less security, and use the SAS disks in Raid0 for high-speed capture storage.

So that leaves me with a problem/question that I am looking for creative solutions to:
I need a good external enclosure for either 2 SAS drives, or 4 SATA drives. With SAS, the key factor I would be looking for is speed, and with SATA, the key factor would be security. Drobo comes to mind for the SATA drives, but is there a cheaper option with the same level of security, or better bandwidth?  Anyone know any good ways of externally connecting two SAS drives in Raid0?

Anyhow, export and render benchmark results to come, specifically with Premiere, AE, RedCine, and Cineform, plus a few FPS benchmarks for fun.

The industry has responded with many different solutions, that vary in concept beyond recognition and in price by many orders of magnitude.  The earliest solutions involved video tape, analog replaced by digital recording.  Hard disks were introduced for random access to data, and now those are slowly beginning to be replaced by solid state flash chips.  Since this site is targeted to PC users, we will focus on hard disk based solutions, and the interfaces with which they can be accessed by a media workstation.

Hard disks are produced with five popular interfaces:  IDE/ATAPI, Serial-ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), and Fibre Channel (FC).  IDE and SCSI interfaces are currently being phased out and replaced by their more capable and flexible Serial varients.  I know little of true Fibre Channel hard disks, but that format is rarely used in this industry.  That leaves only two options, which are now somewhat similar and compatible, SATA and SAS.  With identical connection cables, and both offered in 3.5″ and 2.5″ form factors, it is hard to tell the two options apart visually.  Their interfaces both support 300MB/s, dedicated buses for each drive, and port splitting when that is not required.

The biggest differences between SATA and SAS are performance and cost, which eventually distill down to one issue: size.  SAS disks have slightly more capable and efficient electronics, run fewer platter, with less data, and much higher RPMs and faster I/O and transfer rates.  SATA drives usually have much more storage capacity, lower speeds, and are always much cheaper.  At first glance, high end post production work would seem suited for SAS drives, since moving picture footage requires a higher data transfer rate than almost any other application of computing technology.

There are four other factors, which when combined, weight much more heavily in favor of SATA.  The first is price.  Since the difference in price per Gigabyte is currently so great, and SATA drives are not that different in their design or performance, a few quick calculations will reveal that while SAS disks have higher performance per drive, SATA disks deliver more performance per dollar, regardless of their storage capacity.  Second is that the infrastructure needed to aggregate the performance of multiple disks (Raid arrays) will be required, regardless of which disk solution we choose.  This is due to the fact that HD resolutions and larger require much higher data transfer rates than any single drive can provide (unless compressed, and even then, fast disk access is beneficial).  The marginal cost to increase the number of drives being aggregated will be low in many cases.  The third factor is that digitalized footage requires a tremendous amount of storage space, once again contributing to the need for many hard drives to be combined.  Lastly, most of the popular solutions to improve reliability, do so by utilizing even more capacity, to store redundant information in the form of parity, or straight backups.

These factors, when combined make a strong case for SATA disks, which have higher capacity at the expense of performance per drive.  If we are combining drives anyway, the performance benefits of SAS will usually be negated by combining more SATA drives for less money.  This is a case where quantity can clearly overcome quality in most instances.  As a side benefit, SATA drives usually have much greater capacities.

The only time when SAS may be favorable, will be when there is little need for high capacity, and when there is value to smaller solutions.  Fewer SAS disks are required to reach a given level of performance, and will therefore be more portable, require less power, and frequently generate less heat and noise.  For visual effects, were a few seconds of footage are manipulated at very high quality, or short commercials, SAS may be a more efficient option.

In most cases though, the numbers come down in favor of SATA by along shot.  Let’s imagine a two hour movie, with a 10:1 shooting ratio, giving us 20 hours of footage, and for the sake of example, let’s assume a data rate of 100MB/s.  With 3600 seconds in an hour, that is 360000MB an hour, or 360GB.  20 hours of footage would require 7.2TB of storage.  Add 10% to avoid disk fragmentation, and you need an 8TB array.  With 1TB SATA disks you need 8, plus two more to support Raid 50.  You will have the bandwidth of eight drives, and assuming 50MB/s each for SATA disks, and an efficient controller interface, that is 400 MB/s, more than enough for our 100MB/s files.  10 SATA drives at 1TB currently costs ago $3,000, and the Raid hardware will be required by both SATA and SAS, so it does not necessarily need to be factored in.  Now when onlining a production, not all footage is usually captured, but when you factor in captures, conformed exports, film and video colored versions, testless and texted masters, a 10:1 ratio will not be an inaccurate estimate.  Now I used round numbers, so that if the datarate of your format of choice is higher or lower, you can ajust accordingly. 200MB/s footage would need 20 disks, but could get double performance.  50MB/s footage would only need 5 disks, but could still expect 200MB/s of performance.  Have less footage, I left a 4x overhead in this example with 20 hours of source, but I also used 1TB drives for my calculations.  With 10 hours, 500GB drives show SATA to be even more economically favorable.

Now for a quick comparison to SAS, we start by noting that the maximum capacity is 300GB, and you can expect to pay at least $500 per disk.  Our 8TB example would require about thirty disks, assuming a Raid 50, striping together three Raid 5 arrays of ten disks. 27 data disks is 8.1TB for a cost of $15,000 in drives alone, not counting that it requires hardware for three 10 bay array enclosures instead of one.  From a performance perspective, assuming 80MB/s per disk, you can get over 2GB/s if you want to pay for an interface that fast, but remember that this is all for footage that is 100MB/s.  2GB/s might be good if you want to share it between multiple systems, but with that many users, usually multiple productions will be processed concurrently, requiring much more storage capacity anyway.  By multiplying up and down for different formats, it becomes clear that there is no way that SAS can economically catch up.

So I hope this successfully establishes that SATA disk drives will almost always be the drive type of choice for post-production environments.  I plan to examine the different options for connecting these drive arrays to a workstation or group of systems in my next post.