{"id":221,"date":"2010-12-11T20:33:22","date_gmt":"2010-12-12T03:33:22","guid":{"rendered":"http:\/\/www.hd4pc.com\/techblog\/?p=221"},"modified":"2010-12-11T20:33:22","modified_gmt":"2010-12-12T03:33:22","slug":"shared-san-storage-solutions","status":"publish","type":"post","link":"https:\/\/www.hd4pc.com\/techblog\/2010\/12\/11\/shared-san-storage-solutions\/","title":{"rendered":"Shared SAN Storage Solutions"},"content":{"rendered":"

SANs<\/a> are a hardware solution that allows multiple systems to share access to the performance and security offered by large high speed disk arrays.\u00c2\u00a0 A single array of disks can be partitioned in a way to provide each connected system direct access its own volume, with the each system taking advantage of the redundancy and speed benefits of a\u00c2\u00a0large RAID<\/a>.<\/p>\n

Shared SANs takes the benefits of having all of your storage interconnected with high bandwidth links, and extends it one step further.\u00c2\u00a0 By running special software to synchronize the connected systems, it allows each of the connected systems to access the same the data on the same volume on the SAN, without overwriting each others files or corrupting the data.\u00c2\u00a0 Most SAN software is designed to function as a peer to peer solution for smaller installations, (5-10 systems) or with dedicated servers for larger SANs.
\n
\nAs is probably obvious, there are many benefits to having multiple systems sharing the same set of files on a central high performance disk array<\/a>.\u00c2\u00a0 First off, you don’t have to buy individual arrays for each system, making individual systems cheaper and quieter.\u00c2\u00a0 All the actual data is stored in single physical location, making it easier to protect and secure it.\u00c2\u00a0 With all the data stored on centralized volumes, file management is easier, with a single unified file structure, and you lose the need to duplicate source files across every system that needs local access to them.\u00c2\u00a0 This saves time and storage space.\u00c2\u00a0 It also makes it easier to make thorough backups<\/a>, especially in automated form, which makes your data more secure.\u00c2\u00a0 On the flipside, the initial investment is usually rather high, and all of your eggs are in one basket.\u00c2\u00a0 If the SAN has an issue or problem, your entire production may grind to a halt until the issue is resolved.<\/p>\n

Most all SANs use Fiber Channel<\/a> as their primary physical interface.\u00c2\u00a0 Although this in not inherently required, until recently there was no other standard technology that offered that capability.\u00c2\u00a0 CalDigit<\/a> recently launched a PCIe switch<\/a>\u00c2\u00a0product that they claim offers shared SAN capabilities for their PCIe attached arrays<\/a>.\u00c2\u00a0 While the idea is great, currently the hardware<\/a> is still in a similar price range to entry level fiber solutions<\/a>, and you still need expensive software to keep the connected systems in sync and prevent your SAN data\u00c2\u00a0 from getting corrupted.<\/p>\n

iSCSI<\/a> also offers some of the same capabilities, with block level drive access, but is only a viable competitor in the high end production world when running on 10Gb<\/a> ethernet interfaces, which are still usually prohibitively expensive at this point.\u00c2\u00a0 Running iSCSI<\/a> over Gigabit ethernet may be a viable solution for certain compressed workflows, but offers few advantages over regular network storage, at the expense of needing separate SAN software to share properly.<\/p>\n

There are a number of different software options when creating a Shared SAN.\u00c2\u00a0 I am not familiar with\u00c2\u00a0every one\u00c2\u00a0of them, but the five I describe here should give you a place to start.\u00c2\u00a0 They all serve the same purpose of preventing multiple systems from trying to write data in the same spot at the same time, but they use a variety of different methods to accomplish that objective.<\/p>\n

FiberJet<\/a> is the cheapest option, but does not allow true file level sharing.\u00c2\u00a0 It prevents overwriting and data corruption by only giving one system at a time write access to any given volume.\u00c2\u00a0 On the otherhand, all systems can be given full read access any volume all the time.\u00c2\u00a0 This allows you to share source footage and other media with multiple workstations without the waste of having to duplicate the files.\u00c2\u00a0 It doesn’t allow you to easily share actually project files, since most apps will require write access, and will usually force you to share your files across a number of separate volumes, making it harder to find or backup your data efficiently.\u00c2\u00a0 So\u00c2\u00a0FibreJet<\/a> gives you about half of the benefits of a Shared SAN, as a low cost starting point.<\/p>\n

MetaSAN<\/a> has been available for quite a while now, and is fairly common in PC based post-production environments.\u00c2\u00a0 It supports true file level sharing, allowing all of your systems to read and write files on the same volume simultaneously.\u00c2\u00a0 It supports standard file systems<\/a>, and operates as a separate process over IP<\/a> to keep machines in sync.\u00c2\u00a0 It also allows PCs to access files on Mac formatted drives and vice versa.\u00c2\u00a0 It requires one of the connected systems to host the server process, to manage the distribution of metadata and synchronization information.\u00c2\u00a0 That system does not have to be dedicated to that task, but it can be for maximum performance and stability.\u00c2\u00a0 If you use a user workstation, rebooting that system could cause other users to lose disk access.\u00c2\u00a0 I have used MetaSAN<\/a> for many years, and it is an amazing tool, but it has its quirks that you have to get used to.\u00c2\u00a0 It has a tendency to freeze up workstations if something goes wrong, as it waits for certain requests to timeout, which can make it difficult to troubleshoot when you are in a hurry. (And when the SAN is down, you are always in a hurry)\u00c2\u00a0 On the otherhand, with all of its instability and frusteration, it has never allowed one of my arrays to become corrupted, or for me to lose data, so it clearly performs its function.<\/p>\n

HyperFS<\/a> is a recently released option, primarily offered by Rorke Data<\/a> in the US.\u00c2\u00a0 It has its own proprietary file system, which can be directly accessed from Windows, OSX and Linux based systems.\u00c2\u00a0 The base software is priced similar to MetaSAN, and functions in a peer to peer fashion in smaller installations.\u00c2\u00a0 But if you have more than 8 systems to connect, you will be required to invest in a full dedicated metadata server and license, which significantly increases the deployment cost.<\/p>\n

XSAN<\/a> is Apple’s shared SAN software offering, currently on version 2.2, and it is limited to OSX and requires Xserve<\/a> systems as metadata controllers.\u00c2\u00a0 As a PC guy, I have no experience with XSan<\/a>, but it is used by many Mac based post-production facilities.\u00c2\u00a0 The underlying technology is based on the last option we will examine, StorNext.<\/p>\n

StorNext<\/a> is by far the most expensive option, but it offers higher performance, specifically for frame based media, than any of the other choices.\u00c2\u00a0 Frame sequence<\/a>\u00c2\u00a0based media bog down other SAN software due to the high number of individual files that are being opened, accessed, and closed, in rapid sequence.\u00c2\u00a0 Each individual frame requires the same amount of metadata and synchronization data as an entire video file, overloading lower end software options.\u00c2\u00a0 StorNext is an enterprise level product with a variety of options and tiers, with versions that support every different OS, and even ones that interoperate with Apple’s XSan.\u00c2\u00a0 It is clearly an expensive option, but you are paying for stability and performance, putting it at the core of many DI facilities that have a DPX<\/a> based workflow.<\/p>\n

Shared SANs are one off the most complicated and expensive\u00c2\u00a0investments available in the post-production world.\u00c2\u00a0 Lower cost network based alternatives are a better place to start, for smaller\u00c2\u00a0oragnizations and compressed workflows,\u00c2\u00a0until you are sure you need the performance that SANs can offer.\u00c2\u00a0 Once you are working with uncompressed high definition video, or 2K frame sizes, especially with multiple users, a SAN will probably be worth the investment.\u00c2\u00a0\u00c2\u00a0The effect that they can have on your workflow and level of collaboration is dramatic, making them worth the effort it takes to get them up and running.<\/p>\n","protected":false},"excerpt":{"rendered":"

SANs are a hardware solution that allows multiple systems to share access to the performance and security offered by large high speed disk arrays.\u00c2\u00a0 A single array of disks can be partitioned in a way to provide each connected system direct access its own volume, with the each system taking advantage of the redundancy and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,9],"tags":[16,24,34,7],"_links":{"self":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts\/221"}],"collection":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/comments?post=221"}],"version-history":[{"count":4,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts\/221\/revisions"}],"predecessor-version":[{"id":225,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/posts\/221\/revisions\/225"}],"wp:attachment":[{"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/media?parent=221"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/categories?post=221"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hd4pc.com\/techblog\/wp-json\/wp\/v2\/tags?post=221"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}