The other major new feature for this release of After Effects is the Rotobrush.  Based on Photoshop image processing functionality, extended to account for the information made available from multiple frames, this is the closest thing you are going to find to magic in a current generation compositing application.  It allows you to automatically separate foreground and background objects in a video image, otherwise known as rotoscoping.  Of course it is not perfect, but it is a huge advance, compared to previous tools.  Automated tools like this are usually much more impressive if you aren’t expecting a miracle going into the process, but as long as you have reasonable expectations, Rotobrush can give you usable results in a rush, or a good starting point when more precise work is required.  Much of the “magic” of the new tool is in the processing of the automatically defined edges, and this capability is available without using the Rotobrush, in the form of the “Refine Matte” effect.  This effect can be used to polish the edges of standard keys, or manually rotoscoped footage.  Most of the other major feature enhancements for this release come in the form of integrated third party plug-ins, including Color Finesse for grading, Mocha for tracking, and DigiEffects Freeform for 3D simulation.  Each of these are feature rich plug-ins that include functionality that is beyond the scope of this overview, but are well worth experimenting with if you spend a lot of time working in After Effects.

There are quite a few new features in Premiere Pro CS5 that don’t hinge on the new Mercury Playback Engine.  These relate primarily to metadata and content organization, as opposed to the media itself.  Speech detection tools were introduced in CS4, and now those can be used in conjunction with the features of Adobe Story, to link your actual footage to your original plans, including scripts, storyboards, and shot lists.  Much of this footage processing links to OnLocation CS5′s features, which besides monitoring the technical details of a captured media signal, allow ingest of metadata during the acquisition process.  Building up as much information as possible during production, on a per-shot and per-take basis, will further streamline the organization process during editorial.  All of this information is searchable, with the intent being that an editor will have a much easier time finding what they are looking for, as this metadata is passed down throughout the post production workflow, based on source timecode.  This content logging metadata is not the only way that CS5′s handling of non-media information has improved.

Adobe has also been working hard to streamline the process of exchanging existing projects with Premiere from Final Cut Pro and Avid Media Composer.  They have further refined the FCP XML import and export functionality that was first introduced in version 4.0.1, which offers some interesting options for Mac based workflows.  Working at a PC based facility, I have had more opportunity to take advantage of Premiere’s support for exchanging timelines with Avid.  My current workflow is to use EDLs to move my sequences from Avid to Premiere, but with CS5′s improved support for AAF import and export, hopefully I will soon be able to leverage the additional functionality offered by AAF exchange.  (One caveat with using EDLs to move sequences, is that occasionally Avid adds spaces to the end of every line, and CS5 will not relink to the source footage automatically, unless you manually remove those spaces)  I have yet to find the optimal settings to switch to AAF files, but I have talked with people who have it up and running.  Combining this sequences exchanging feature with CS5′s support for DNxHD in both MXF and MOV wrappers, offers some interesting possibilities, but the truly revolutionary workflow will finally arrive when Media Composer 5 is released next month, allowing Avid to link to external QuickTime files thru AMA.  Hopefully this will allow Premiere Pro CS5 and Media Composer 5 to exchange sequences that all reference the same source files, without any conversions or transcoding.  Since Canon DSLR files will be supported natively both applications, this should totally streamline my current workflow.

Now why would someone want to move their project between different NLE applications?  An editor’s familiarity with a specific toolset is usually important to them, so certain offline creative editors are not going to switch their primary editorial app no matter what.  To its credit, Avid has a solid reputation for handling extremely large projects with no decrease in performance.  While Premiere Pro has taken a huge step forward in that regard, it is going to take a long time for it to build a stable reputation, since trust usually develops very slowly.  On the other hand, Premiere makes a perfect interactive online conform tool, especially compared to Avid.  It can ingest most digital formats in their native form, supports SDI capture and playback over a number of different hardware solutions, and scales to 2K and 4K resolutions, in RGB color space if desired.  It is compatible with DPXs for color grading, and can playback surround sound for reviewing final mixes.  The fact that Premiere can use Dynamic Link to ingest your visual effects changes from After Effects, and to output to Encore for adding interaction becomes an added bonus for this conform solution.

Dynamic Link is now on its fourth iteration, and has matured into a functional tool, even at HD resolutions.  It is designed exchange media between apps, without having to waste time or disk space by rendering.  While it started as a method for importing AE comps into PPro, it became the backbone of PPro’s export capabilities thru Adobe Media Encoder as well.  While Media Encoder CS5 is a dramatic improvement from the previous version, Adobe also added back in the option to export files directly from within Premiere.  This is much quicker for rendering out small parts of large projects, avoiding the overhead of syncing the entire project with Media Encoder before rendering a couple of frames.  They also brought back the ability to easily export single frames from the timeline, a function that had been removed in CS4, an inexcusable oversight that has now been rectified.  Exporting a Premiere Pro sequence thru Media Encoder, while not transparent, is now a much faster and more stabile process than it was in CS4.  Media Encoder also now supports DPX sequences and AVC-Intra MXF files, for both input and output.  Still sequence support in both Premiere Pro and Media Encoder is still missing crucial options, for relinking footage in PPro and manually overriding the frame rate in AME.  Hopefully we will have more user control of these settings in future versions.

One thing that I would like to see handled differently in Media Encoder, as well as the Premiere exporter, is the default settings for each export plug-in.  Adobe is never going to be able to predict what settings every user is going to want, but it would be nice if it defaulted to whatever I selected last time I used that particular output option.  As it stands, Media Encoder defaults to the same output as the last file queued, but if I choose any other output (AVI, MOV, DPX) it returns all of the individual settings to the original default, which is usually DV based.  Now it is possible to save presets, but I hesitate to do so unless I anticipate using that exact configuration frequently, because sorting through too many presets can become harder than manually defining the individual settings.  On a relate note, be careful where you save your .prproj files, since selecting “Save As” does not default to the current project location as it should.  Instead it defaults to the folder containing the last piece of media that you imported.  I usually follow a strict project organizational pattern, but I have saved projects in the wrong folders daily since switching to CS5 thanks to this “feature.”  Hopefully this can be fixed in a minor update.

Now a subsidiary application to Premiere Pro, Encore CS5 is the next logical step in that application’s evolution as an interactive authoring tool.  The most significant new feature for larger productions would have to be support for DDP output, so that your Encore projects can now be replicated at a professional facility without any special hardware required on the authoring side.  Adobe has also continued to develop the Flash export options, allowing Encore users to output their interactive experiences directly to the web, now with most of the BluRay interactivity features, including the new multipage menus, supported as well.  Encore projects are now completely cross platform compatible, between the Mac and PC versions.  There is also better support for 23.976p and 24p source files, and an easier subtitling process.  AVCHD files can now be authored directly to BluRay discs without any transcoding, decreasing processing time, and increasing output quality.

Previous versions of Encore have used Dynamic Link to import Premiere Pro sequences as source clips, and more significantly in my opinion, to import After Effects compositions as motion menu backgrounds.  I was never really that impressed by Dynamic Link for Encore source clips, because your Premiere Pro sequence is usually finished by the time you make a DVD, and since you are going to have to transcode to MPEG2 at some point anyway, little time is saved.  Motion menu design on the other hand seems like a much better fit for the workflow benefits offered by Dynamic Link, since the menu needs to be edited in both Encore and AE during the authoring process.  Encore now also uses Dynamic Link to offload transcoding of those sequences, or any other incompatible source footage, to Adobe Media Encoder.  This will free up Encore to continue interactive authoring work, while asset transcoding proceeds in the background, and also allows third party accelerated encoding plug-ins that are compatible with Adobe Media Encoder, to be utilized for these internal transcodes.  While I am unaware of any CS5 compatible encoders, this was an issue with the Elemental Accelerator in CS4.  The files it generated at 24p were re-transcoded in Encore anyway, and Encore couldn’t utilize the accelerated plug-in directly.  Anyhow, while no CS5 version of Elemental Accelerator has been announced, Matrox’s CompressHD should be able to accelerate Encore’s BluRay H.264 encodes, once Matrox releases CS5 compatible drivers.

The one thing still missing from the Adobe package is a good intermediate format for exchanging files between other applications, systems, or facilities.  This capability is available from 3rd parties like Cineform or Matrox, but until Adobe integrates support for one directly into the suite, there will not be a single universal standard format, that can be counted on to be compatible everywhere.  It needs to be compatible with both Mac and PC, and store at least 10 bit HD footage with sufficient compression to playback smoothly on a laptop.  Apple and Avid have both recognized this need, and developed ProRes and DNxHD respectively.  AVC-Intra is Adobe’s current recommendation to meet those workflow needs, but that format has a complicated file structure, and is not a codec optimized for smooth playback.  This universal format would tie in well with OnLocation, if Adobe ever added support for HD-SDI capture capabilities.  That would allow ingest, either live or from tape playback, into a compressed format for editorial, with log notes entered in real-time similar to their current tapeless solution options.  And if a professional selection of scopes could be viewed of the SDI input signal, you would have an all-in-one onset workstation solution.  OnLocation CS5 currently offers most of these capabilities, but is limited to HDV, XDCam, and P2 acquisition for most of its features.  Oh, and if it supported my Canon DSLR, that would be pretty cool too; maybe next time.

As a whole the CS5 release is a major turning point for Adobe, especially with the problems their users experienced with CS4.  While I don’t yet fully utilize many of the new features, CS5 has already revolutionized the way I work.  Premiere Pro CS5 has replaced After Effects CS4 as central application from which I manage my projects.  I was lucky enough to be on the beta team, so most of the major issues I encountered with the software in my specific workflow, have already been resolved during the development process.  While CS5 is by no means perfect, it is a huge step forward that will begin to rebuild user confidence in Adobe products.  Once third party hardware and plug-ins have had time to be updated to be compatible with the new 64bit native applications, I see no other disadvantages or caveats to upgrading to CS5, which I highly recommend.

FTC Disclosure: I have been on Adobe’s beta team for many years, and Adobe has provided me a copy of CS5 for this review.  My only admitted personal bias is my preference of Windows over OSX, because I like full control over every aspect of my computing experience.  If for some reason that bothers anyone, there are plenty of other sources of information on the internet, but I try to provide unique insight on how each of these tools fits into the larger post-production picture.  Any relevant critique or response is welcomed.

Apple recently released a free ProRes playback decoder for both OSX and Windows, which is pretty significant.  This with allow Final Cut Pro editors to share their footage using this efficient codec with Windows users and Mac users who don’t have Final Cut Pro.  Since there is still no way to encode to ProRes on PC, another format is still required to get footage back from the PC to the Mac.

Reading between the lines of some of NVidia’s Nvision, I believe I have acertained the specs on Nvidia’s next top end workstation graphics card.  The NVidia Quadro FX 5800 will have 4GB of memory, with a PCI-Express 2.0 interface, for 8GB/sec of bandwidth in both directions.  It should have two dual link DVI connectors and one DisplayPort connector, besides the usual Quadro stereo port.  It should support SLI Mosaic Mode, using four connections from two cards to support a four quadrant image, which will eventually allow full resolution display of 4K footage in realtime.

Speaking of 4K, Red has announced that Adobe will be releasing a plugin to edit R3D files natively in the CS3 suite of applications.  Coinciding with that due to the release of the first version of the Red SDK, Cineform has finally released an “official” version of their Red4K convertor.  They have also begun to advocate an offline/online workflow for the first time, under the brand name of Cineform Express.  My only comment is, once Adobe supports Red natively, shouldn’t the online be done straight from the R3D files?  In a perfect world, a Cineform offline with a relink to R3D files, exported as an EDL to SpeedGrade for color would be the ideal way to do post with Red footage.  If only life was that simple; maybe someday it will be.

Originally the only way to generate Cineform AVI files at this resolution was to scan film to DPXs, and then convert those image sequences, presumably in After Effects.  Interestingly, while being at 2K resolution, these files were being downsampled to YUV colorspace, which improved compression rates and performance, at the expense of the quality certain color information.  Eventually the SI-2K camera provided a second root source of Cineform2K imagery, that did not involve converting from an uncompressed source file at any point in the process.  I anticipate we will see 2K data being captured over SDI directly to Cineform files in the near future, if they are not doing so already, as both AJA and BMD have products with the required hardware capability.

The SI-2K brought with it one other new option to the format, which was an alternate pre-debayer RAW ‘colorspace’ instead of the YUV data that was normally being compressed.  CineformRAW video files had even better compression ratios, but the only way to take advantage of the benefits of pre-debayered images was by tapping directly into a single sensor imager, as the SI-2K did.  This eventually led to a third underlying option, with the development of Cineform 444.

Cineform444 allows full RGB encoding, processing, and mastering of image data, which is the norm in a digital intermediate environment.  The RAW files of the SI-2K had the required information for RGB processing, but it hadn’t been fully taken advantage of until that point.  DPX filmscans have full RGB info, as do most still images and CGI files.  Only ‘broadcast video’ files and formats are limited to YUV.  While increasing file sizes and processing requirements, this RGB processing brought the “Cineform Intermediate” to the next level quality wise.  Cineform now shows off their format in comparison the the popular HDCam-SR format, which doesn’t directly compare since SR can’t be accessed natively, but it communicates the idea to people who still think in terms of tape formats.  I believe SR tapes that are recorded in 444 RGB can be captured directly to Cineform444 via BMD hardware, but I have not had the opportunity to test that myself.  I have worked on projects where we captured 444 RGB data uncompressed through a BMD Multibridge, and then later converted parts to Cineform444 to work with.  I have also extracted CineformRAW files from the SI-2K directly to full range Cineform444 files.  Either way, this workflow has allowed me to work on visual effects shots at full 2K resolution in RGB, on my system at home without a dedicated RAID, and even on my laptop if I could suffer the 12inch screen size.

Around the same time that the Cineform444 capabilites were being developed, the option to include an imbedded alpha channel was added, and the maximum bit depth was increased from 10bit to 12bit.  I have yet to take advantage of the 12bit capability, but the embedded alpha channel can be useful for everything from animated titles to visual effects.  I find the biggest gains from this development when using Cineform for my After Effects work.

Cineform has also been working to develop a cross-platform solution, so that Cineform compressed media can be easily shared between Macs and PCs.  Their first step was to develop a Quicktime wrapper for their original AVI codec.  Next, they ported the compression codec to OSX, assisted I am sure, by Apple’s switch to Intel processors.  The last steps remaining to be completed are an accelerated CineformRT engine for the new Mac version of PremierePro CS3, and an OSX port of their HDLink conversion tool.  At this point Cineform is the only reliable compressed format for transfering video between Windows and OSX systems, besides saving to a DPX sequence, which has obvious disadvantages.

As mentioned in an earlier post, Cineform has recently released a beta of their next improvement to the capabilities of their format, with support for 4K imagery, or higher.  Although 4K filmscans could be used as source, their immediate target is data from the Red One camera, with 4K output from the RedCine conversion utility.  With wavelet compression for efficient reduced resolution decoding, RGB colorspace, 10 or 12bits of color depth, support for alpha channels, and cross platform compatibilty, Cineform seems well positioned to meet the needs of this emerging market.

The current Neo4K workflow is to use RedCine to export Cineform4K MOV files from the RedOne camera’s native R3D files.  These compressed files can be played in realtime, usually at 2K display resolution, in Premiere, AfterEffects, and a variety of other programs.  Filmout is the primary application for 4K finishing, and most filmout facilities are still going to require DPX files as the final step, but Cineform can be used for all of the steps prior to that point.  If they optimize their downsampled playback correctly, realtime CCR in Speedgrade should be possible in the near future, with 2K live playback and 4K renders.  Iridas needs to get Speedgrade working correctly in HD and 2K first though, before we get too far ahead of ourselves.

I Updated/Rewrote the following section Saturday morning after further testing and research:

I was able to download 2K and 4K R3D demo files directly from Red, and successfully imported them into RedCine.  I then was able to export them to Cineform2K and Cineform4K MOV files.  The 4K files took 4-6 seconds per frame to render, and the 2K took about 1 second per frame, on my dual 3.2Ghz Xeon workstation with my new 8800GTX GPU.  There is a known error in RedCine that prevents the encoding of files at exactly 4096 pixels wide, so testing is being done at 4088 pixels width instead.  I also converted the resulting MOV files to AVI, so I had four files to test, AVIs and MOVs at 2K and 4K.  After updating Quicktime at David Newman’s suggestion, all my files worked in both Premiere Pro (Prospect2K project) and in AfterEffects.  The 4K files do not really playback in Premiere, but you can scrub them.  In AE, I get 2fps at 4K in full quality.  At half quality (4K->2K) I get 6fps, which is the same as I get when playing back 2K footage at full.  All these AE tests were done at 8bpp, since I foresee doing work at 2K-8bit, and rendering at 4K-16bit.

Clearly there is still work to be done, but the software was all released in the last couple of days, in first public beta form.  Regardless, it is amazing that I am now able to edit 4K video footage in my bedroom on my Xeon workstation from 2005.  Once it is working correctly in AE, I will be all set, because that is where I like to do most of my online work.  It is much more precise than Premiere Pro, and the type of work I do rarely requires much realtime playback.  I am looking forward to getting a chance to do some “real” work with Red footage, but given the current performance, I am hoping that my first steps can be with a shorter project.  It also reassures me of my recent decision to invest in a new workstation, since 4K is a lot of data to be manipulating, no matter how you compress it.

I will try to explain with this analogy.  Most HDV cameras allow you to capture HDV recordings to downconverted DV files in realtime, for backwards compatibility with older NLEs.  Imagine you shot footage in HDV, but you want to offline in DV, and online in HDV.  You capture your HDV footage into a DV project one full tape at a time, and are left with bunch of DV avi files.  You edit the entire piece, and export the final audio, and a DV guidetrack.  Now that you have the final edit in DV, you are ready to conform it at HDV quality.

There are a number of approaches you could take to accomplishing this.  You could throw away the project file and recapture everything in HDV to do an eye match to the DV guidetrack, withno other reference.  That would obviously be a tedious process. Alternatively, you could offline all your media in Premiere Pro, and recapture every entire tape, starting and ending at the exact same timecode values as your DV captures were, giving you an identical set of AVIs, that are at the higher HDV resolution.  This is more efficient, but still requires a lot of storage if your online format is 100 times larger than your offline (not true with HDV) and capture time.  Once finished capturing, there would be very little more work to do. (Resize titles, etc.) You could do a “Remove Unused” in Premiere to totally skip any tapes that were not used in the final edit.  This could save you some space, but only if you have a lot of captures you never ended up using at all.  The next option is to use the project manager to create an offline trimmed project, and recapture only the required parts of clips from HDV (usually with handles).  This process works in Premiere even with high end HD-SDI projects, but will not yet work with data centric workflows.  The alternative that SHOULD work for the Red RAW 4K workflow, among other tapeless formats, is very similar.  In our DV to HDV example, we export an EDL from our DV project and import that EDL into an HDV project to recapture only the needed material.  This allows us to easily capture only the required segments of higher resolution footage, saving tima and disk space.  The new captures will already have a project sequencing them in the correct order, so all that remains is to redo any titles and graphics or effects at the higher resolution, and the project will be ready to send to color correction or whatever the next step in your specific workflow is.