I downloaded the Lite version and have been playing around with it, but it is not a very intuitive program.  I am about fifty pages into the manual, but have yet to be able to get a sequence of content I cut together in CS5.5 into Resolve successfully.  I can’t export it into any format Resolve will read, besides uncompressed HD, which my array is not designed to handle.  Using an EDL to link to my Canon 5D source files is giving me all sorts of issues as well, which is something we experienced at the office when we first switched from Speedgrade to Resolve.  Clearly it is going to take some time to learn how too get anything useful done in the program.  If I do get it to work, the automatic tracking tools are the features I am most interested in playing with.

In other major news that doesn’t directly affect me, Blackmagic-Design also acquired Teranex last week.  The first major change they made was cutting the price of Teranex’s primary flagship product, the VC100, from $90,000 to $20,000.  This is similar to what happened when they acquired DaVinci last year, but much more extreme.  It will be interesting to see if some of that high end image processing technology makes it into future Decklink cards, or any similar products.

After announcing it last week, today Avid released the next version of Media Composer, and it is a major update.  The key thing that will affect all users is that the program will now be a natively 64bit application.  This will exclude users who still have 32bit systems, but allow the software to use more RAM, and therefore effectively do larger projects, on 64bit systems, which are pretty prevalent at this point.  From what I can tell, re-coding an entire application for 64bit is also good for cleaning up the code for existing features, and streamline performance and threading.  Avid’s primary competition: Adobe Premiere Pro, saw a huge improvement in both performance and stability when the first 64bit version (CS5) was released.  This has led to a huge increase in that application’s popularity, especially for larger projects.  The level of competition presented by that change may be a key factor in why we are now seeing so many major changes from Avid.

The next most significant new feature, that will affect many users, especially new ones, is “Avid Open I/O” which is Avid’s new approach to integrating hardware with Media Composer.  Avid now supports hardware I/O via products from AJA, Blackmagic, Matrox and others.  We have seen hints of this coming, with support for the Matrox MXO2-Mini and AJA ioExpress in versions 5.0 and 5.5 respectively.  This change has two major benefits, it will allow most users access to professional hardware I/O solutions without paying for Avid’s overpriced hardware, and nearly as important, it will allow edit systems to run multiple applications on the same system and hardware.  Add this to advances in project compatibility and interchange, and users will be able move their editing projects between Avid and Premiere Pro (and FCP on the Mac side) on the same systems, using the same video output hardware.

Avid also added a myriad of new features to enhance more advanced workflows, with better support for stereoscopic content, more advanced color correction tools, and surround sound mixing options on the audio side.  They also have a 4:4:4 RGB version of the popular DNxHD codec, and natively support ProRes files.  Their Symphony finishing toolset no longer requires Nitris DX hardware, but I have yet to find the pricing structure for that option.  It looks to be a solid feature set, and I am looking forward to trying it out.

The solution that I use for my stereoscopic work is Cineform’s Neo3D, which allows 3D content to be processed through a fairly standard post production workflow.  They also have a creative way of supporting stereoscopic content, that allows stereoscopic finishing work to be done during any step in the process, even concurrently on a separate system if desired.  This process uses the same active metadata workflow that they have made available for color correction for the last few years.  All of the stereoscopic adjustments, as well as the muxing of the left and right streams is done on the fly by the decoder, so changes don’t have to be permanently rendered into the media, and the target application receives a premuxed single stream of video.  This allows certain 2D apps that have no integrated support for stereoscopic post, to be utilized for 3D editorial, and still give editors a live stereoscopic preview.  I discussed the options for connecting 3D preview displays, as well as some of the display options available, in earlier articles in the series.  I am sure the process will continue to evolve as new products are developed to simplify the workflow.

Shared SANs takes the benefits of having all of your storage interconnected with high bandwidth links, and extends it one step further.  By running special software to synchronize the connected systems, it allows each of the connected systems to access the same the data on the same volume on the SAN, without overwriting each others files or corrupting the data.  Most SAN software is designed to function as a peer to peer solution for smaller installations, (5-10 systems) or with dedicated servers for larger SANs.

As is probably obvious, there are many benefits to having multiple systems sharing the same set of files on a central high performance disk array.  First off, you don’t have to buy individual arrays for each system, making individual systems cheaper and quieter.  All the actual data is stored in single physical location, making it easier to protect and secure it.  With all the data stored on centralized volumes, file management is easier, with a single unified file structure, and you lose the need to duplicate source files across every system that needs local access to them.  This saves time and storage space.  It also makes it easier to make thorough backups, especially in automated form, which makes your data more secure.  On the flipside, the initial investment is usually rather high, and all of your eggs are in one basket.  If the SAN has an issue or problem, your entire production may grind to a halt until the issue is resolved.

Most all SANs use Fiber Channel as their primary physical interface.  Although this in not inherently required, until recently there was no other standard technology that offered that capability.  CalDigit recently launched a PCIe switch product that they claim offers shared SAN capabilities for their PCIe attached arrays.  While the idea is great, currently the hardware is still in a similar price range to entry level fiber solutions, and you still need expensive software to keep the connected systems in sync and prevent your SAN data  from getting corrupted.

iSCSI also offers some of the same capabilities, with block level drive access, but is only a viable competitor in the high end production world when running on 10Gb ethernet interfaces, which are still usually prohibitively expensive at this point.  Running iSCSI over Gigabit ethernet may be a viable solution for certain compressed workflows, but offers few advantages over regular network storage, at the expense of needing separate SAN software to share properly.

There are a number of different software options when creating a Shared SAN.  I am not familiar with every one of them, but the five I describe here should give you a place to start.  They all serve the same purpose of preventing multiple systems from trying to write data in the same spot at the same time, but they use a variety of different methods to accomplish that objective.

FiberJet is the cheapest option, but does not allow true file level sharing.  It prevents overwriting and data corruption by only giving one system at a time write access to any given volume.  On the otherhand, all systems can be given full read access any volume all the time.  This allows you to share source footage and other media with multiple workstations without the waste of having to duplicate the files.  It doesn’t allow you to easily share actually project files, since most apps will require write access, and will usually force you to share your files across a number of separate volumes, making it harder to find or backup your data efficiently.  So FibreJet gives you about half of the benefits of a Shared SAN, as a low cost starting point.

MetaSAN has been available for quite a while now, and is fairly common in PC based post-production environments.  It supports true file level sharing, allowing all of your systems to read and write files on the same volume simultaneously.  It supports standard file systems, and operates as a separate process over IP to keep machines in sync.  It also allows PCs to access files on Mac formatted drives and vice versa.  It requires one of the connected systems to host the server process, to manage the distribution of metadata and synchronization information.  That system does not have to be dedicated to that task, but it can be for maximum performance and stability.  If you use a user workstation, rebooting that system could cause other users to lose disk access.  I have used MetaSAN for many years, and it is an amazing tool, but it has its quirks that you have to get used to.  It has a tendency to freeze up workstations if something goes wrong, as it waits for certain requests to timeout, which can make it difficult to troubleshoot when you are in a hurry. (And when the SAN is down, you are always in a hurry)  On the otherhand, with all of its instability and frusteration, it has never allowed one of my arrays to become corrupted, or for me to lose data, so it clearly performs its function.

HyperFS is a recently released option, primarily offered by Rorke Data in the US.  It has its own proprietary file system, which can be directly accessed from Windows, OSX and Linux based systems.  The base software is priced similar to MetaSAN, and functions in a peer to peer fashion in smaller installations.  But if you have more than 8 systems to connect, you will be required to invest in a full dedicated metadata server and license, which significantly increases the deployment cost.

XSAN is Apple’s shared SAN software offering, currently on version 2.2, and it is limited to OSX and requires Xserve systems as metadata controllers.  As a PC guy, I have no experience with XSan, but it is used by many Mac based post-production facilities.  The underlying technology is based on the last option we will examine, StorNext.

StorNext is by far the most expensive option, but it offers higher performance, specifically for frame based media, than any of the other choices.  Frame sequence based media bog down other SAN software due to the high number of individual files that are being opened, accessed, and closed, in rapid sequence.  Each individual frame requires the same amount of metadata and synchronization data as an entire video file, overloading lower end software options.  StorNext is an enterprise level product with a variety of options and tiers, with versions that support every different OS, and even ones that interoperate with Apple’s XSan.  It is clearly an expensive option, but you are paying for stability and performance, putting it at the core of many DI facilities that have a DPX based workflow.

Shared SANs are one off the most complicated and expensive investments available in the post-production world.  Lower cost network based alternatives are a better place to start, for smaller oragnizations and compressed workflows, until you are sure you need the performance that SANs can offer.  Once you are working with uncompressed high definition video, or 2K frame sizes, especially with multiple users, a SAN will probably be worth the investment.  The effect that they can have on your workflow and level of collaboration is dramatic, making them worth the effort it takes to get them up and running.

External SAS connected arrays function in much the same way as SATA based ones, but with a few advantages, that usually come at a significantly higher cost.  SAS is a full duplex interface, and the command set is based on SCSI instead of IDE, allowing higher performance and throughput.  More expensive SAS arrays also support multipath signaling, for greater redundancy in the supporting electronics. (As opposed to the redundancy provided at the disk level by RAID configurations)  SAS also supports much longer cable lengths, up to 10 meters or 30 feet.  This can be advantagious for quiet video editing rooms, since the disk array, which is usually the loudest part of the system, can be located farther away from the users.

A number of vendors have now begun offering external arrays that interface with the host workstation via a direct extension of the PCIe bus.  This allows all of the RAID functionality to be contained within the array, and gives full speed access to the data as if it was contained within the machine.  Among the advantages of removing the RAID functionality from an internal add-on card, are that it can be attached to a laptop via an ExpressCard, which uses the same signaling protocol as PCIe, and that with addition of a few cheap pass-thru cards, an array can easily be moved between systems.  This is definitely not a hot swappable solution, since it accesses the PCIe bus directly, which is initialized at bootup on most systems.  But if your main edit system has a total OS meltdown at a critical point in your project, it should be much easier to access your data from a different system than if you needed to reinstall the PCI SATA RAID card somewhere else, and allow you use your laptop as a backup edit system in certain instances.

Fibre Channel is by far the most expensive option.  Every part of the system is more expensive, the PCIe HBA cards, the fiber cables, and the disk array controllers.  On the otherhand, Fibre Channel offers capabilities that none of the other storage options really do.  It is a hot swappable interface, running on fiber cables that can extend access thousands of feet if desired, and can easily be networked and shared.  Devices can be connected directly together, shared in an Arbitrated Loop, or all attached to a central fibre switch for simplified management.  It is an efficient and low latency interface, and is available in speeds of 1,2,4, or 8Gb per second, and multiple channels can be combined for higher performance.  Higher speed devices are usually backwards compatible with older hardware, similar to the way ethernet works, allowing you to upgrade your storage network one piece at a time.

Choosing the right storage solution depends on your immediate media needs, your available budget, and the direction you anticipate growing in the future. SATA based solutions offer all of the speed you could need if scaled large enough.  SAS can offer similar performance in a smaller package, but at a higher cost.  Sharing data beyond gigabit network speeds requires a storage system that can interface with multiple computers, but that comes at a significantly increased initial cost.  Investing in Fibre Channel storage is usually only worth the expense if you anticipate the need to share your data on a SAN, either immediately or at some point in the future.  I will examine a few popular shared SAN options in my next post.

Speaking of DSLR files, Adobe has totally reinvented the way they are handled in CS5.  Most applications, including the CS3 and CS4 versions of both Premiere and After Effects, use QuickTime importers to access the content of Canon DSLR files.  This makes sense, since they are stored in an MOV wrapper, but leads to two issues.  One is that is specific to Adobe is that on a PC, QuickTime files go through a few extra steps before they are accessed by the application, so there is a performance hit, and with lots of files accessed at once, there are usually stability issues as well.  The other issue effects all applications that use QuickTime to access DSLR files, and that is that ever since QuickTime 7.6.2 was released, Canon DSLR files have been decoded in a much more flat and washed out color space than they were designed to be viewed in.  Prior to version 7.6.2, they were decoded in a way that clipped the highlights and shadows, which was even worse.  In CS5, Adobe worked with Mainconcept to create an importer that reads the DSLR source files without involving QuickTime at all.  This alleviates both the performance hit on PC systems, and the color space issues of QuickTime’s default decoding.  A lot of work was put into getting the decode matrix and color space exactly correct based on the processing that Canon’s hardware does to the file in the camera.  This should allow CS5 to decode the files more correctly than any other application that I am aware of, and give more options for color processing at later stages in the workflow pipeline, since more of the original color data is preserved.

The Mercury playback engine has a significant impact on the user experience, with most frequently used tools being available in real time.  Supposedly most of the decode and playback improvements are based on the new code written for native 64bit execution, with the GPU offload limited to effects processing.  While many editors don’t use very many discrete effects in their work, there are some intrinsic playback functions like scaling frame sizes and adapting frame rates that are considered effects and offloaded to the GPU.  This allows content of different frame rates and resolutions to be intercut seamlessly on the timeline, and it is truly seamless.

I have occasionally found myself editing in the wrong sequence frame rate without even noticing it, since the software makes the conversion on the fly.  Even more frequently I have found a clip shot in the wrong frame rate on a tapeless camera almost escaping detection because gone are the red render bars and playback glitches that used to stand out.  This allows editors to import media from many different sources without prerendering everything to a normalized format.  I used to spend about a quarter of my time at work converting strange source footage into 1080p24 intermediate files, because any footage not matching the timeline format would cause previous versions of Premiere to glitch during playback and occasionally crash.  Now I would recommend carefully converting any footage used in a final piece to the correct format for maximum control, but this step can now be put off until the online stage, since it is no longer required for playback and stability.  Since 90% of footage usually ends up on the cutting room floor, putting off these time consuming conversions until after the creative edit is finished, will drastically reduce the amount of footage you end up needing to process.

So this clear increase in performance leads to the question of: how far can you push it?  A few months back I processed an ISO noise test in After Effects CS4 for Shane Hurlbut.  We were comparing the image noise produced at twelve different ISO levels on the Canon 5D, and the project involved twelve streams of video with masks, levels, and position adjustments for a tiled view.  I was getting about two frames per second when rendering previews, which seemed reasonable considering the amount of processing involved.  When I saw the list of GPU accelerated effects in Premiere Pro CS5, and was asked to create torture test for Adobe to show off at NAB, this jumped to mind.  I recreated the entire project, using twelve layers of native DSLR footage, each layer having a motion effect and a 4-point garbage matte to create the tiling, and a color correction applied to exaggerate the noise to a clearly viewable level.  The same basic setup that was getting 2fps in AE CS4 played back in real-time in Premiere Pro CS5.  (Dual Xeon X5365 CPUs and 16GB RAM with a QuadroFX 4800) Needless to say I was quite impressed with the outcome, since it was deliberately beyond Adobe’s ten-layer playback claim, and using a complex format to decode and playback.  Clearly GPU acceleration can have a dramatic impact on application performance.

There has been much discussion and debate on tech forums and blogs about the specifics of Premiere Pro’s hardware support for CUDA acceleration.  Adobe has severely restricted the number of cards for which they officially support CUDA based GPU acceleration, to maintain control over the hardware environments upon which their accelerated code is tested, supposedly for stability reasons.  The official list is limited to QuadroFX 3800, 4800, and 5800, as well of the discontinued GeForce 285 GTX, with certain limitations, for those on a lower budget.  There are currently no officially supported mobile GPUs, even though notebook CPUs are usually more in need of a performance boost than desktop chips.  This may be due to the fact that even the newest mobile QuadroFX 3800M is still based on the G92 core from the GeForce 8000 series, but I don’t like seeing software artificially limited in regards to performance or hardware support, and this is an example of both.  Don’t confuse legitimate limitations and artificial limitations, since clearly a powerful GPU is necessary for optimal performance in CS5, but there are cards of equal capability that are specifically excluded from the list, supposedly for stability reasons.  Luckily Adobe has left an option for knowledgeable users to override some of those artificial limitations, and I anticipate seeing them being dropped completely in a future update.  I anticipate a more reasonable requirement of any NVidia card supporting CUDA 1.1 or 1.3, with at least 785MB of video memory, at some point in the future.

While Premiere Pro CS5 is not perfect, it is a complete reversal from the previous fiasco that was CS4.  It clearly demonstrates the possibilities provided by GPU acceleration, resting solidly on 64bit code with proper multithreaded programming, it scales to take advantage of whatever hardware is made available for it.  Since Adobe has made a practice of introducing significant improvements in incremental dot releases, I am looking forward to seeing how else they refine it in the coming months.

FTC Disclosure: I have been on Adobe’s beta team for many years, and Adobe has provided me a copy of CS5 for this review.  NVidia has provided me with graphics hardware in the past, which I utilized in this review.  My only admitted personal bias is my preference of Windows over OSX, because I like full control over every aspect of my computing experience.  If for some reason that bothers anyone, there are plenty of other sources of information on the internet, but I try to provide unique insight on how each of these tools fits into the larger post-production picture.  Any relevant critique or response is welcomed.

The other major new feature for this release of After Effects is the Rotobrush.  Based on Photoshop image processing functionality, extended to account for the information made available from multiple frames, this is the closest thing you are going to find to magic in a current generation compositing application.  It allows you to automatically separate foreground and background objects in a video image, otherwise known as rotoscoping.  Of course it is not perfect, but it is a huge advance, compared to previous tools.  Automated tools like this are usually much more impressive if you aren’t expecting a miracle going into the process, but as long as you have reasonable expectations, Rotobrush can give you usable results in a rush, or a good starting point when more precise work is required.  Much of the “magic” of the new tool is in the processing of the automatically defined edges, and this capability is available without using the Rotobrush, in the form of the “Refine Matte” effect.  This effect can be used to polish the edges of standard keys, or manually rotoscoped footage.  Most of the other major feature enhancements for this release come in the form of integrated third party plug-ins, including Color Finesse for grading, Mocha for tracking, and DigiEffects Freeform for 3D simulation.  Each of these are feature rich plug-ins that include functionality that is beyond the scope of this overview, but are well worth experimenting with if you spend a lot of time working in After Effects.

There are quite a few new features in Premiere Pro CS5 that don’t hinge on the new Mercury Playback Engine.  These relate primarily to metadata and content organization, as opposed to the media itself.  Speech detection tools were introduced in CS4, and now those can be used in conjunction with the features of Adobe Story, to link your actual footage to your original plans, including scripts, storyboards, and shot lists.  Much of this footage processing links to OnLocation CS5′s features, which besides monitoring the technical details of a captured media signal, allow ingest of metadata during the acquisition process.  Building up as much information as possible during production, on a per-shot and per-take basis, will further streamline the organization process during editorial.  All of this information is searchable, with the intent being that an editor will have a much easier time finding what they are looking for, as this metadata is passed down throughout the post production workflow, based on source timecode.  This content logging metadata is not the only way that CS5′s handling of non-media information has improved.

Adobe has also been working hard to streamline the process of exchanging existing projects with Premiere from Final Cut Pro and Avid Media Composer.  They have further refined the FCP XML import and export functionality that was first introduced in version 4.0.1, which offers some interesting options for Mac based workflows.  Working at a PC based facility, I have had more opportunity to take advantage of Premiere’s support for exchanging timelines with Avid.  My current workflow is to use EDLs to move my sequences from Avid to Premiere, but with CS5′s improved support for AAF import and export, hopefully I will soon be able to leverage the additional functionality offered by AAF exchange.  (One caveat with using EDLs to move sequences, is that occasionally Avid adds spaces to the end of every line, and CS5 will not relink to the source footage automatically, unless you manually remove those spaces)  I have yet to find the optimal settings to switch to AAF files, but I have talked with people who have it up and running.  Combining this sequences exchanging feature with CS5′s support for DNxHD in both MXF and MOV wrappers, offers some interesting possibilities, but the truly revolutionary workflow will finally arrive when Media Composer 5 is released next month, allowing Avid to link to external QuickTime files thru AMA.  Hopefully this will allow Premiere Pro CS5 and Media Composer 5 to exchange sequences that all reference the same source files, without any conversions or transcoding.  Since Canon DSLR files will be supported natively both applications, this should totally streamline my current workflow.

Now why would someone want to move their project between different NLE applications?  An editor’s familiarity with a specific toolset is usually important to them, so certain offline creative editors are not going to switch their primary editorial app no matter what.  To its credit, Avid has a solid reputation for handling extremely large projects with no decrease in performance.  While Premiere Pro has taken a huge step forward in that regard, it is going to take a long time for it to build a stable reputation, since trust usually develops very slowly.  On the other hand, Premiere makes a perfect interactive online conform tool, especially compared to Avid.  It can ingest most digital formats in their native form, supports SDI capture and playback over a number of different hardware solutions, and scales to 2K and 4K resolutions, in RGB color space if desired.  It is compatible with DPXs for color grading, and can playback surround sound for reviewing final mixes.  The fact that Premiere can use Dynamic Link to ingest your visual effects changes from After Effects, and to output to Encore for adding interaction becomes an added bonus for this conform solution.

Dynamic Link is now on its fourth iteration, and has matured into a functional tool, even at HD resolutions.  It is designed exchange media between apps, without having to waste time or disk space by rendering.  While it started as a method for importing AE comps into PPro, it became the backbone of PPro’s export capabilities thru Adobe Media Encoder as well.  While Media Encoder CS5 is a dramatic improvement from the previous version, Adobe also added back in the option to export files directly from within Premiere.  This is much quicker for rendering out small parts of large projects, avoiding the overhead of syncing the entire project with Media Encoder before rendering a couple of frames.  They also brought back the ability to easily export single frames from the timeline, a function that had been removed in CS4, an inexcusable oversight that has now been rectified.  Exporting a Premiere Pro sequence thru Media Encoder, while not transparent, is now a much faster and more stabile process than it was in CS4.  Media Encoder also now supports DPX sequences and AVC-Intra MXF files, for both input and output.  Still sequence support in both Premiere Pro and Media Encoder is still missing crucial options, for relinking footage in PPro and manually overriding the frame rate in AME.  Hopefully we will have more user control of these settings in future versions.

One thing that I would like to see handled differently in Media Encoder, as well as the Premiere exporter, is the default settings for each export plug-in.  Adobe is never going to be able to predict what settings every user is going to want, but it would be nice if it defaulted to whatever I selected last time I used that particular output option.  As it stands, Media Encoder defaults to the same output as the last file queued, but if I choose any other output (AVI, MOV, DPX) it returns all of the individual settings to the original default, which is usually DV based.  Now it is possible to save presets, but I hesitate to do so unless I anticipate using that exact configuration frequently, because sorting through too many presets can become harder than manually defining the individual settings.  On a relate note, be careful where you save your .prproj files, since selecting “Save As” does not default to the current project location as it should.  Instead it defaults to the folder containing the last piece of media that you imported.  I usually follow a strict project organizational pattern, but I have saved projects in the wrong folders daily since switching to CS5 thanks to this “feature.”  Hopefully this can be fixed in a minor update.

Now a subsidiary application to Premiere Pro, Encore CS5 is the next logical step in that application’s evolution as an interactive authoring tool.  The most significant new feature for larger productions would have to be support for DDP output, so that your Encore projects can now be replicated at a professional facility without any special hardware required on the authoring side.  Adobe has also continued to develop the Flash export options, allowing Encore users to output their interactive experiences directly to the web, now with most of the BluRay interactivity features, including the new multipage menus, supported as well.  Encore projects are now completely cross platform compatible, between the Mac and PC versions.  There is also better support for 23.976p and 24p source files, and an easier subtitling process.  AVCHD files can now be authored directly to BluRay discs without any transcoding, decreasing processing time, and increasing output quality.

Previous versions of Encore have used Dynamic Link to import Premiere Pro sequences as source clips, and more significantly in my opinion, to import After Effects compositions as motion menu backgrounds.  I was never really that impressed by Dynamic Link for Encore source clips, because your Premiere Pro sequence is usually finished by the time you make a DVD, and since you are going to have to transcode to MPEG2 at some point anyway, little time is saved.  Motion menu design on the other hand seems like a much better fit for the workflow benefits offered by Dynamic Link, since the menu needs to be edited in both Encore and AE during the authoring process.  Encore now also uses Dynamic Link to offload transcoding of those sequences, or any other incompatible source footage, to Adobe Media Encoder.  This will free up Encore to continue interactive authoring work, while asset transcoding proceeds in the background, and also allows third party accelerated encoding plug-ins that are compatible with Adobe Media Encoder, to be utilized for these internal transcodes.  While I am unaware of any CS5 compatible encoders, this was an issue with the Elemental Accelerator in CS4.  The files it generated at 24p were re-transcoded in Encore anyway, and Encore couldn’t utilize the accelerated plug-in directly.  Anyhow, while no CS5 version of Elemental Accelerator has been announced, Matrox’s CompressHD should be able to accelerate Encore’s BluRay H.264 encodes, once Matrox releases CS5 compatible drivers.

The one thing still missing from the Adobe package is a good intermediate format for exchanging files between other applications, systems, or facilities.  This capability is available from 3rd parties like Cineform or Matrox, but until Adobe integrates support for one directly into the suite, there will not be a single universal standard format, that can be counted on to be compatible everywhere.  It needs to be compatible with both Mac and PC, and store at least 10 bit HD footage with sufficient compression to playback smoothly on a laptop.  Apple and Avid have both recognized this need, and developed ProRes and DNxHD respectively.  AVC-Intra is Adobe’s current recommendation to meet those workflow needs, but that format has a complicated file structure, and is not a codec optimized for smooth playback.  This universal format would tie in well with OnLocation, if Adobe ever added support for HD-SDI capture capabilities.  That would allow ingest, either live or from tape playback, into a compressed format for editorial, with log notes entered in real-time similar to their current tapeless solution options.  And if a professional selection of scopes could be viewed of the SDI input signal, you would have an all-in-one onset workstation solution.  OnLocation CS5 currently offers most of these capabilities, but is limited to HDV, XDCam, and P2 acquisition for most of its features.  Oh, and if it supported my Canon DSLR, that would be pretty cool too; maybe next time.

As a whole the CS5 release is a major turning point for Adobe, especially with the problems their users experienced with CS4.  While I don’t yet fully utilize many of the new features, CS5 has already revolutionized the way I work.  Premiere Pro CS5 has replaced After Effects CS4 as central application from which I manage my projects.  I was lucky enough to be on the beta team, so most of the major issues I encountered with the software in my specific workflow, have already been resolved during the development process.  While CS5 is by no means perfect, it is a huge step forward that will begin to rebuild user confidence in Adobe products.  Once third party hardware and plug-ins have had time to be updated to be compatible with the new 64bit native applications, I see no other disadvantages or caveats to upgrading to CS5, which I highly recommend.

FTC Disclosure: I have been on Adobe’s beta team for many years, and Adobe has provided me a copy of CS5 for this review.  My only admitted personal bias is my preference of Windows over OSX, because I like full control over every aspect of my computing experience.  If for some reason that bothers anyone, there are plenty of other sources of information on the internet, but I try to provide unique insight on how each of these tools fits into the larger post-production picture.  Any relevant critique or response is welcomed.

Twixtor is a plug-in from ReVision Effects that allows you to change the frame rate of your footage thru motion compensated frame blending.  While it can be used to add frames for slow motion effects, I have found that I get much better results when removing frames, such as when dropping from 30fps to 24fps.  Regardless of the specific settings, Twixtor takes a lot of time to render.  In our first tests on 8-Core Xeon systems, processing one minute of source footage required one minute of render time.  Now with Intel’s new Nehalem based CPUs, and recently their even newer Gulftown 6-core chips, we have seen that reduced by about fifty percent, to a half hour per minute of source footage, which is still a long time, but feels great compared to where we were a year ago.  Since our footage re-link process is based on frame counts, we have to process our entire source clips in order to take advantage of that level of workflow automation, even if we are only using the last ten seconds of a 14 minute take.  Obviously there are ways around this, but we currently have more render time available to us than man hours, and it gives us more flexibility later on anyway, so we just let it go.  We took advantage of every night and weekend during creative editorial to Twixtor every clip that made it into the rough cut, and now we just have to link to that bank of processed footage to conform our cuts in CS4.  The fact that all of Canons DSLRs now support 24p should alleviate most of the frame rate and Twixtor issues in future projects.

Besides frame rate issues, Canon DSLRs present another unique challenge, in regards to color space and bit depth.  Many professional video codecs store the color values in the range between 16 and 235, of the 256 possible 8bit options.  (The reasoning for this is fairly complicated, and relates primarily to legacy analog video signal issues)  This limits pixels to 220 levels for each color in most 8bit codecs, but the MOV files from the Canon DSLRs use the entire 256 possible options (0-255) for each color.  This increases the number of possible values for each three color pixel by over 50%, (220^3 vs 256^3) but also means that converting your DSLR footage into most other 8bit formats will result in one of two issues: either the extreme values will be clipped, losing detail in the highlights and the shadows, or all of the dynamic range will be squeezed into the reduced sample-space, meaning certain intermediate values are going to be merged together if you edit in an 8bit codec.

Clipping was the most likely possibility in most existing applications prior to the release of Quicktime 7.6.2 in mid 2009.  Previous to that point, Quicktime displayed Canon clips incorrectly (clipping the values beyond 16-235) but after that update was released, most applications that used Quicktime to decode DSLR footage, were able to access the entire dynamic range of the source clips.  This support is not a foregone conclusion though, since DSLR files could be imported with a more generic MPEG4 decoder without Quicktime, and still be displayed incorrectly.  Even with properly calibrated import processes, compressing the 255 possible values for each color channel into the limited 220 values that most 8bit video formats offer, will lead to a loss of precision, and a potential increase in color banding, especially if you plan to color correct the footage later.  A 10bit video format will offer four times as many possible legal color values, and will be able to store all of the original image data with precision to spare.  Once you have color corrected your footage, and any visual effects are complete, an 8bit distribution format may be sufficient for most uses, but any image processing that takes place on the original files before you apply the “look” that you want, should definitely be processed in at least 10bit color space to preserve as much of your original image information  as possible.

When editing DSLR footage in Avid, DNxHD is the recommended intermediate format.  DNxHD files can be encoded in either HD 709 (16-235) or RGB (0-255) color space, but any DNxHD files encoded in RGB are converted to HD 709 upon import into Media Composer, regardless of the original output setting.  Therefore any DNxHD MOV files generated elsewhere for ingest into Avid should be exported at 16-235 to match Avid’s target color space, for a lossless “Fast Import.”  On the other hand, when importing DSLR footage into Avid, you should select “Computer RGB (0-255)” as the SOURCE color space, in the “File Pixel to Video Mapping” options.  (Rec. 709 is always the TARGET color space for DNxHD MXFs in Avid)  While importing with the 0-255 setting retains the full dynamic range, it still squeezes the entire range into the 16-235 gamut.  That loss of precision should not be as significant as viewable dynamic range for an offline edit, but if you planning to export your Avid sequence as your master without a separate conform, you should consider using a 10bit codec in Avid, like DNxHD 175x.  That will allow you to maintain both the original dynamic range and the bit depth, at the expense of higher storage space requirements.

Once you have a re-linked timeline of high quality 24p footage, there are still a few more steps that can be taken to cleanup the footage.  Dead pixels should be the first thing on the list to deal with.  Dead pixels can be caused by physical debris on the sensor or lens of the camera, or by an electronic malfunction with one of the photo-receptors on the CMOS sensor.  The result is the same regardless of the cause, with one of more pixels locked at a static value throughout the shot.  The simplest way to fix this is to cover the effected pixels with information from the surrounding area.  One procedural way to fix this is to duplicate the layer of footage in an AE comp, and mask out a similar section nearby and cover it.  (If you have a horizontal row of three dead pixels, mask the three pixels above them on a second layer, and then drop the top layer down one to cover the spot)  In most cases the duplicated data will be totally invisible, but be sure to QC the result.  If you are Twixtoring your footage to a different frame rate, fix the dead pixels before applying the rate change, otherwise the motion compensation process will cause the dead pixels to move around, making them much more difficult to remove in a procedural fashion.  The next step is to look for any rolling shutter artifacts, caused by the slight difference in time between when the top and bottom of the frame are sampled.  This difference in time can manifest itself in a number of interesting ways, including distortion, with the top of the frame seeming to “lead” the bottom.  It can also cause horizontal bands of brightness with quick flashes of light only being recognized by part of the sensor.  The Foundry has a plug-in called Rolling Shutter that can help reduce the image distortion caused by motion of the camera on smoother shots.  The horizontal bands have to be removed manually in a VFX process if you want to get rid of them, borrowing data from the preceding or following frames if needed.  The Canon DSLRs also exhibit some moiré and other aliasing issues due to the way they sample the low resolution video from the high resolution sensor.  The only way to really get rid of those artifacts is to selectively mask and blur the effected sections of the frame.  Lastly, if you are using Twixtor, QC the output for corrupt frames caused by the interpretation engine being unable to guess the proper motion of the moment in the shot.  If re-rendering with different settings doesn’t help, covering the bad frame with an original frame of footage from that moment usually solves the single frame issues.  Luckily the most difficult sections of footage to calculate motion compensate for, are usually segments of where using frame dropping conversion instead is undetectable, since the extreme motion should hide any stutter caused by the missing frames. (This is coming from a guy who is processing a lot of handheld combat footage)  Once these steps, as well as the rest of your visual effects work, are finished, you are ready to export and color, which should be similar to most other workflows at this point.

Adobe announced all of the features of the new highly anticipated Creative Suite CS5, which will include native 64bit, CUDA GPU acceleration, and better support for formats like DPX and DSLR clips.  I will have many more CS5 details in future posts, delving into how that will change the landscape of many workflows.  There are lots more new developments in the post world, but those are the primary things that stick out to me right now.  We still have two more days, and I will be spending a good bit of time demonstration Cineform’s Neo3D features at their booth at the back of the Lower South Hall.  So if anyone wants to catch up with me, feel free to stop by.

So once the lenses issue was dealt with, were left with a selection of AVCHD encoded MOV files.  AVCHD is a processing intensive format that does not playback or edit very well.  While Quicktime would play the files, it clipped the blacks and the whites at incorrect levels.  16 and 235 were being stretched to 0 and 255 on decode, lowering the dynamic range.  This was caused by Quicktime incorrectly interpreting one of the header fields on the file.  The solution to this was to use CoreAVC to decode the files when converting into a different, and ideally more edit friendly, compression format.  Shortly after this workaround was developed, Apple released a Quicktime update (7.6) that fixed this particular issue entirely.

Beyond the clipping issue, there are other tricks to maximize the dynamic range of the 5D.  The picture style is used to control the way that the camera converts the 14bit RAW still into an 8-bit JPEG.  The same picture profile settings are applied to the 8-bit recorded video.  This allows you to do things to get the maximum detail out of the available 8-bits of color depth.  The first few projects I worked on that used the 5D, we used a custom picture profile that I got from Stu Maschwitz’s ProLost blog, High Gamma 5.  We did a number of comparison tests, and while High Gamma 5 gave us a wider total dynamic range, for our feature film, we eventually decided to use Neutral, one of the default Canon presets.  Neutral gave us a file that was closer to the final look we were going for, and with only 8bits of color depth, burning in your look, at least to a degree, should result in better picture quality at the end of the day.

Every file the camera records is named MVI_####.mov, with an auto-incrementing number, and no real override options.  That makes things simple on tiny projects, with one camera since each file has a unique name.  On larger projects, and ones that use more that one camera, (We usually have 15) file management can be a bit more work, to keep things straight throughout the post production process.  Our solution was to rename each MOV file with a unique 8 digit identifier as the new filename, and store the key to the original card and filename in a database.  This allows each clip to have a consistent name throughout the process, to show up on EDLs as a tape name or clip name as desired, without truncating unique values after the 8th digit for certain formats.  By the time we are done ,we usually have a source MOV, an Avid MXF, and an online Cineform AVI, all with the same content and file name.

Next up was the framerate problem, at 30p.  The first few projects I did with 5D we posted at 29.97, so the issue was solved with a simple reinterpretation of the framerate, when converting from the source AVCHD into an editing codec, and tweaking the audio .1% to match.  Unfortunately 29.97 footage doesn’t intercut with film very well, and won’t print back for theatrical masters either, so sometimes a 24p workflow is required.  For 24p projects, the conversion solution is much more complicated, involving motion compensated frame blending.  After extensive testing we concluded that this was best done with the Revision Effects Twixtor plugin for AE, or using Optical Flow in FCS Compressor on OSX.  Having a PC centered workflow, I favor the AE based solution.  With render times at around an hour per minute of source footage, it is impractical to convert all of the source footage on large projects, which necessitates an offline edit.  Since we don’t have timecode and keycode, relinking for the online requires a bit more creativity.  We have found some interesting options that are unique to Premiere Pro CS4, related to the way it links EDLs to existing source footage that make this much simpler than our first tedious tests, which involved manually rebuilding projects at 24p back in Premiere Pro CS3.  The new CS4 version can convert the TC-In on an EDL to a framecounted In-point of an existing media file, with makes the onlining of 5D footage a relatively simple automatic process after a few find-replace edits (.mov to .avi in our case) to the EDL.  In the future, it looks like Canon is going to support 24p recording on all of their DSLR offerings, so all of these crazy 30p workarounds will soon be an obselete thing of the past.

Although it is much better in rough environments than most other electronics, Canon DSLRs do have their weaknesses.  I have operated a 5D in temperatures of 20 below zero, and in the desert at over 120 degrees fahrenheit.  While we had no issues in the cold, where solid state recording has a huge advantage over tape, there are some issues at higher temperatures.  The camera sensor itself is a large piece of silicon, that generates a lot of heat on its own, and when combined with a high external temperature, in the worst cases is shuts off the camera.  You probably have to be over 150 degrees to get to that point, leaving the camera in a black metal box in direct sunlight for an extended period of time, but we have seen it happen.  A much more frequent problem, that is harder to detect, is that as the sensor begins to overheat, there will be much more video noise in the recorded picture, especially in the darks.  This is probably due to a higher latent voltage on the chip as its electrical resistance changes with the temperature increase.  This has only been a problem for us when shooting with the same camera for many hours in a hot environment, and our solution is usually just to swap the camera bodies for one that has not been used in a while.  This obviously requires having multiple cameras on set, which isn’t always an option on lower budget projects.

The last issue, that we are still finding new ways to deal with, is rolling shutter.  Having a large format CMOS sensor, DSLRs are subject to rolling shutter, or inconsistencies between the top and bottom of the frame.  I have spent the last few months working on a project that put the 5D into some of the most intense situations.  As a fairly lightweight device, it is subject to more jitter and shake than a larger camera with more inertia, and with the camera moving, the rolling shutter results in the recorded picture being slightly geometrically skewed, depending on the direction of the motion.  We also shoot high speed objects, like helicopter rotor blades, which are known to cause strange artifacts in certain instances.  So far we have been lucky with that, and haven’t found any of those types of issues in our footage.

The type of rolling shutter artifact we are struggling with the most, is gunfire muzzle flashes, especially at night.  In the dark, the flash blows out the imager, but the flash does not last as long as even a single frame.  So with the rolling shutter, the top half of a frame will be totally blown out, with the bottom part looking normal, because the flash had subsided by the time that part of the chip was sampled, or vice versa.  Setting the shutter speed lower than the frame rate causes it to screw up more of the frame or frames, and setting it higher causes it to narrow the flash into a distinct horizontal band in the footage, neither of which is desirable.  One thing we have found that helps is setting the shutter on the 5D to 1/30th.  (We usually set it to 1/50 to get similar motion plur to film shot with a 180 degree shutter)  With the 30p framerate, the flash either affects an entire frame, or matching parts on two subsequent frames.  (Bottom part of one frame, and the reverse area on the top of the next one)  This gives us an entire over exposed frame if we stitch the two parts together.  This can be hand cut back into footage that has been brought from 30p to 24p by manually selecting frames.  It remains to be seen if this solution can be scaled practically to our entire movie.  The best way to avoid this issue is to avoid recording gunfire at close range in very dark environments.  The farther you are from the muzzle flash, and the more ambient light there is, the less it is going to flare out your camera, minimizing the degree of the resulting rolling shutter artifact.

So that should convey some of the challenges are in faced in using DSLRs for filmmaking, especially on large scale projects, but it is by no means an exhaustive list.  As the tools evolve to suit the cameras, and the cameras evolve to suit the tools, many of these issues will become much easier to solve and require fewer workarounds.  The AVCHD decoding issue was solved by a new release of Quicktime, the manual lense control was solved with a new firmware release from Canon.  The 30p conversion process is the next issue I see becoming a thing of the past, if Canon can get a 24p recording option onto the 5D.  I am looking forward to that day, but in the mean time I have 2TB of 30p footage, divided into 5,000 shots, to cut into a 24p film, so I have a lot of work ahead of me.