Well now that we have established\u00c2\u00a0the idea that SATA drives are usually the ideal choice, we have to deal with the next logical question:\u00c2\u00a0 How should we go about connecting a whole bunch of these drives\u00c2\u00a0to our editing system?\u00c2\u00a0 The primary considerations I will be examining are cost, throughput, reliability, and shared access.\u00c2\u00a0 The most popular solutions, offered\u00c2\u00a0by multiple vendors, are SCSI, Fibre\u00c2\u00a0Channel,\u00c2\u00a0Ethernet,\u00c2\u00a0iSCSI, eSATA\u00c2\u00a0and the recently implemented External PCIe.\u00c2\u00a0 There are few other proprietary options available, but those are the ones that are widely available.<\/p>\n
Let’s start with SCSI<\/a>, since it is the easiest to dismiss.\u00c2\u00a0 While we are discussing the connection of SATA drives, many of the first generation SATA arrays had intergrated controllers and Raid hardware, and then needed a fast connection to the host.\u00c2\u00a0 These arrays were designed to replace much more expensive SCSI drive based arrays, so the target customers trusted the SCSI interface, and already had high end SCSI controllers in their systems.\u00c2\u00a0 That made SCSI the optimal connection solution for early SATA arrays.\u00c2\u00a0 The SATA Raid controller masquerades the entire array as a single SCSI disk, allowing connection to systems through existing SCSI cards.\u00c2\u00a0 With up to\u00c2\u00a0320MB\/s of bandwidth, a single SCSI channel can efficiently support 5-7 SATA disks without much impact on\u00c2\u00a0performance.\u00c2\u00a0 The biggest reason to dismiss SCSI as a serious possibility is that eSATA is a better option for most, and the remaining will be much better served by a Fibre Channel interface, allowing for economical upgrading to a full SAN in the future. \u00c2\u00a0There are many products available that share storage directly to an ethernet\u00c2\u00a0network connection.\u00c2\u00a0 The consumer varients hardly have the performance to support DV editing, let alone anything more demanding.\u00c2\u00a0 The higher end options, with prices similar to SCSI and FC do offer some interesting possibilities, but will rarely be the optimal choice for a given situation.\u00c2\u00a0 Any gigabit ethernet connection is limited to 125MB\/s, and in reality, the achievable performance is usually about\u00c2\u00a0half of that.\u00c2\u00a0 Gigabit network solutions will not be a solution for uncompressed work at HD or higher resolutions.\u00c2\u00a0 10Gb Ethernet would\u00c2\u00a0offer the desired performance, but is not\u00c2\u00a0currently an\u00c2\u00a0economical solution.\u00c2\u00a0 If compressed files are used, regualr gigabit ethernet\u00c2\u00a0can be used to transport the data in realtime, but I would still argue that arrays interfacing directly to ethernet are not the most efficent solution.\u00c2\u00a0 Any similar array directly connected to a workstation\u00c2\u00a0through a different interface will give much better performance to that system, and can still be shared on an ethernet network via that workstation.\u00c2\u00a0 There will be a performance hit on that station when sharing data to other system, but a network card with a TCP\/IP Offload\u00c2\u00a0Engine (ToE) can minimize that effect, and the increased performance on that system do to the high speed storage directly attached should more than offset whatever is remaining.\u00c2\u00a0 This would involve using an array with one of the other interfaces we are examining.<\/p>\n A\u00c2\u00a0recent technology that uses ethernet to transfor data, is iSCSI<\/a>.\u00c2\u00a0 Promoted as having many of the advantages of Fibre Channel SANs, iSCSI gives initiator devices (workstations) block level access to their target device (arrays).\u00c2\u00a0 This\u00c2\u00a0allows the target device on the network to emulate a local device on the initiator’s system.\u00c2\u00a0 The downsides are that maintaining data intergrity on shared target drives, requires most of the same expensive software infrastructure that a Fibre SAN does, and the inefficiencies of the TCP\/IP protocol are still present to limit the realistically achievable maximum transfer rate.\u00c2\u00a0 If you have to deliver identical data\u00c2\u00a0to a large number of systems, and don’t want to spend money on the performance that Fibre Channel hardware can deliver, then iSCSI might be of benefit to you.\u00c2\u00a0\u00c2\u00a0These products are targeted at large corporations, and don’t scale down in size without losing performance, and maintaining\u00c2\u00a0deployment complexity.\u00c2\u00a0 I don’t see this being the solution of choice for most desktop PC workstation professionals in post-production field.<\/p>\n \u00c2\u00a0The next solution is offered in a staggering varietly different solutions, eSATA<\/a>.\u00c2\u00a0 This can be fairly confusing due to the number of variations of this technology on the market.\u00c2\u00a0 eSATA is a very flexible standard, but not all implementations will deliver optimal results.\u00c2\u00a0 For example, some products support port multiplying to increase the number of drives without increasing the complexity of the interface cables or the Raid controller.\u00c2\u00a0 This solution is good for high volume solutions, but will not deliver the same level of performance as direct connection based\u00c2\u00a0solutions.\u00c2\u00a0 The simplest, professional level, eSATA array will be an external drive enclosure that passes each drive’s data interface directly back to the controller, which will usually be some varient of PCI card, inside the workstation.\u00c2\u00a0 This gives the card direct full-speed access to each disk drive, and all Raid processing is done on the controller card inside the workstation.\u00c2\u00a0 This will be the fastest and most efficient solution for the cheapest price, and I highly recommend it.\u00c2\u00a0 The limitations are the cables which usually have a 6 foot maximum length, and the fact that Fibre channel is easier to share.\u00c2\u00a0 But for the independent, budget conscious, single workstation user, this is the way to go.\u00c2\u00a0 Eight disks gives you enough storage for almost any concievable independent project, and eight drives should support uncompressed HD if desired, and may even work for 2K with an efficient Raid controller.\u00c2\u00a0 Solutions that use port multipliers to connect more drives, will increase storage but not performance, and usually require more expensive SAS compatible controller cards to support the port multiplying.\u00c2\u00a0 If you need more than 8TB of storage on your system, these might work well for you.<\/p>\n The most recent development in this area is the advent of the use of External PCI Express<\/a> as an array interface.\u00c2\u00a0 A small PCIe passthru card is all that is required in the host system.\u00c2\u00a0 An x4 slot can transmit and recieve 10Gb\/s of data, which is 1.2GB\/s, and there is much less overhead than most other interfaces.\u00c2\u00a0 An x8 slot is capable of twice as much throughput for an insignificant margin cost increase.\u00c2\u00a0 With External PCIe, the drive controller and raid processing electronics are contained within the drive enclosure, and the controller has direct access to the disks.\u00c2\u00a0 As a result, the array could easily be moved to\u00c2\u00a0another system, without having to bring a separate controller card from within the system.\u00c2\u00a0 Each system would need an External\u00c2\u00a0PCIe bracket, but those\u00c2\u00a0are only forth about\u00c2\u00a0ten\u00c2\u00a0dollars. \u00c2\u00a0Due to the nature of the External PCIe interface, the computer has the same level of access to the controller and\u00c2\u00a0its data that it would if those electronics resided on a board contained within the workstation.\u00c2\u00a0 Another benefit of PCIe, is that the new ExpressCard for notebooks is based on the same interface.\u00c2\u00a0 This allows a simple adapter to connect an External\u00c2\u00a0PCIe device to a notebook at x1 speeds (over 250MB\/s will be fast enough for uncompressed HD).\u00c2\u00a0 Currently I am only aware of two vendors offering soluitions using this technology, CalDigit<\/a> and Ciproco<\/a>.\u00c2\u00a0 It will be interesting to watch as this technology continues to develop.<\/p>\n So my recommendation is that high end eSATA solutions are the most economical direct attached storage solutions, and can support uncompressed HD if needed.\u00c2\u00a0 Larger operations that are considering upgrading to a full shared SAN system in the future will probably find the increased initial investment of Fibre Channel arrays to be well\u00c2\u00a0worth the value when they re-utilize the same hardware in their SAN implementation sometime in the future.<\/p>\n","protected":false},"excerpt":{"rendered":" Well now that we have established\u00c2\u00a0the idea that SATA drives are usually the ideal choice, we have to deal with the next logical question:\u00c2\u00a0 How should we go about connecting a whole bunch of these drives\u00c2\u00a0to our editing system?\u00c2\u00a0 The primary considerations I will be examining are cost, throughput, reliability, and shared access.\u00c2\u00a0 The most […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[34,35,32],"_links":{"self":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts\/14"}],"collection":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/comments?post=14"}],"version-history":[{"count":0,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts\/14\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/media?parent=14"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/categories?post=14"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/tags?post=14"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n
\nThe next step for high end SATA arrays was to replace the SCSI emulation with a much more flexible\u00c2\u00a0interface, Fibre Channel<\/a>.\u00c2\u00a0 With up to 400MB\/s, Fibre Channel has few disadvantages to SCSI, and one major benefit.\u00c2\u00a0 SATA disk arrays with Fibre Channel interfaces can usually be connected to switches, and shared between multiple systems, in a SAN<\/a>.\u00c2\u00a0\u00c2\u00a0All connected systems get direct block level access to the disks, which will almost always be faster and more responsive than sharing through an ethernet\u00c2\u00a0network.\u00c2\u00a0 With the proper Shared SAN software, these systems can also\u00c2\u00a0share the data down to the level of individual files.\u00c2\u00a0 For facilities where multiple users do collaberative work, based on the same source data, Fibre Channel is probably worth the added initial investment, even if a SAN is not immediately implemented with the purchased hardware.\u00c2\u00a0 The possible extensible use of\u00c2\u00a0an array beyond a single workstation\u00c2\u00a0should be\u00c2\u00a0well worth the increase in price, and as an added benefit, cable lengths can easily\u00c2\u00a0be increased enough to keep the noisy array out of what should be a peaceful creative environment.<\/p>\n