Speaking of DSLR files, Adobe has totally reinvented the way they are handled in CS5.  Most applications, including the CS3 and CS4 versions of both Premiere and After Effects, use QuickTime importers to access the content of Canon DSLR files.  This makes sense, since they are stored in an MOV wrapper, but leads to two issues.  One is that is specific to Adobe is that on a PC, QuickTime files go through a few extra steps before they are accessed by the application, so there is a performance hit, and with lots of files accessed at once, there are usually stability issues as well.  The other issue effects all applications that use QuickTime to access DSLR files, and that is that ever since QuickTime 7.6.2 was released, Canon DSLR files have been decoded in a much more flat and washed out color space than they were designed to be viewed in.  Prior to version 7.6.2, they were decoded in a way that clipped the highlights and shadows, which was even worse.  In CS5, Adobe worked with Mainconcept to create an importer that reads the DSLR source files without involving QuickTime at all.  This alleviates both the performance hit on PC systems, and the color space issues of QuickTime’s default decoding.  A lot of work was put into getting the decode matrix and color space exactly correct based on the processing that Canon’s hardware does to the file in the camera.  This should allow CS5 to decode the files more correctly than any other application that I am aware of, and give more options for color processing at later stages in the workflow pipeline, since more of the original color data is preserved.

The Mercury playback engine has a significant impact on the user experience, with most frequently used tools being available in real time.  Supposedly most of the decode and playback improvements are based on the new code written for native 64bit execution, with the GPU offload limited to effects processing.  While many editors don’t use very many discrete effects in their work, there are some intrinsic playback functions like scaling frame sizes and adapting frame rates that are considered effects and offloaded to the GPU.  This allows content of different frame rates and resolutions to be intercut seamlessly on the timeline, and it is truly seamless.

I have occasionally found myself editing in the wrong sequence frame rate without even noticing it, since the software makes the conversion on the fly.  Even more frequently I have found a clip shot in the wrong frame rate on a tapeless camera almost escaping detection because gone are the red render bars and playback glitches that used to stand out.  This allows editors to import media from many different sources without prerendering everything to a normalized format.  I used to spend about a quarter of my time at work converting strange source footage into 1080p24 intermediate files, because any footage not matching the timeline format would cause previous versions of Premiere to glitch during playback and occasionally crash.  Now I would recommend carefully converting any footage used in a final piece to the correct format for maximum control, but this step can now be put off until the online stage, since it is no longer required for playback and stability.  Since 90% of footage usually ends up on the cutting room floor, putting off these time consuming conversions until after the creative edit is finished, will drastically reduce the amount of footage you end up needing to process.

So this clear increase in performance leads to the question of: how far can you push it?  A few months back I processed an ISO noise test in After Effects CS4 for Shane Hurlbut.  We were comparing the image noise produced at twelve different ISO levels on the Canon 5D, and the project involved twelve streams of video with masks, levels, and position adjustments for a tiled view.  I was getting about two frames per second when rendering previews, which seemed reasonable considering the amount of processing involved.  When I saw the list of GPU accelerated effects in Premiere Pro CS5, and was asked to create torture test for Adobe to show off at NAB, this jumped to mind.  I recreated the entire project, using twelve layers of native DSLR footage, each layer having a motion effect and a 4-point garbage matte to create the tiling, and a color correction applied to exaggerate the noise to a clearly viewable level.  The same basic setup that was getting 2fps in AE CS4 played back in real-time in Premiere Pro CS5.  (Dual Xeon X5365 CPUs and 16GB RAM with a QuadroFX 4800) Needless to say I was quite impressed with the outcome, since it was deliberately beyond Adobe’s ten-layer playback claim, and using a complex format to decode and playback.  Clearly GPU acceleration can have a dramatic impact on application performance.

There has been much discussion and debate on tech forums and blogs about the specifics of Premiere Pro’s hardware support for CUDA acceleration.  Adobe has severely restricted the number of cards for which they officially support CUDA based GPU acceleration, to maintain control over the hardware environments upon which their accelerated code is tested, supposedly for stability reasons.  The official list is limited to QuadroFX 3800, 4800, and 5800, as well of the discontinued GeForce 285 GTX, with certain limitations, for those on a lower budget.  There are currently no officially supported mobile GPUs, even though notebook CPUs are usually more in need of a performance boost than desktop chips.  This may be due to the fact that even the newest mobile QuadroFX 3800M is still based on the G92 core from the GeForce 8000 series, but I don’t like seeing software artificially limited in regards to performance or hardware support, and this is an example of both.  Don’t confuse legitimate limitations and artificial limitations, since clearly a powerful GPU is necessary for optimal performance in CS5, but there are cards of equal capability that are specifically excluded from the list, supposedly for stability reasons.  Luckily Adobe has left an option for knowledgeable users to override some of those artificial limitations, and I anticipate seeing them being dropped completely in a future update.  I anticipate a more reasonable requirement of any NVidia card supporting CUDA 1.1 or 1.3, with at least 785MB of video memory, at some point in the future.

While Premiere Pro CS5 is not perfect, it is a complete reversal from the previous fiasco that was CS4.  It clearly demonstrates the possibilities provided by GPU acceleration, resting solidly on 64bit code with proper multithreaded programming, it scales to take advantage of whatever hardware is made available for it.  Since Adobe has made a practice of introducing significant improvements in incremental dot releases, I am looking forward to seeing how else they refine it in the coming months.

FTC Disclosure: I have been on Adobe’s beta team for many years, and Adobe has provided me a copy of CS5 for this review.  NVidia has provided me with graphics hardware in the past, which I utilized in this review.  My only admitted personal bias is my preference of Windows over OSX, because I like full control over every aspect of my computing experience.  If for some reason that bothers anyone, there are plenty of other sources of information on the internet, but I try to provide unique insight on how each of these tools fits into the larger post-production picture.  Any relevant critique or response is welcomed.

Adobe announced all of the features of the new highly anticipated Creative Suite CS5, which will include native 64bit, CUDA GPU acceleration, and better support for formats like DPX and DSLR clips.  I will have many more CS5 details in future posts, delving into how that will change the landscape of many workflows.  There are lots more new developments in the post world, but those are the primary things that stick out to me right now.  We still have two more days, and I will be spending a good bit of time demonstration Cineform’s Neo3D features at their booth at the back of the Lower South Hall.  So if anyone wants to catch up with me, feel free to stop by.

NVidia and Elemental Technologies report that the accelerator can provide up to a 10x increase in encoding speeds, but mileage may vary in real world use.  With my system, I usually found encoding rates to be 50% to 100% faster with real world work, depending on the output settings, which is still a significant improvement if you do a lot of exporting.  The relative increase in encoding performance that you will experience depends on the speed of your system.  Older systems will see a major improvement, while newer high end workstations are already quite fast, so the change will be less dramatic.  My 3Ghz 8-Core workstation can already encode my HD timelines to MPEG2 for DVD faster than realtime, but with the Elemental Accelerator, I was able to cut the time for a two minute export from 1:27 to 30 seconds.  On a two hour clip, that would be a thirty minutes instead of an hour and a half.  My 1080p H.264 encodes for BluRay output only saw a 50% increase in encoding speed.

Exports of more complicated timelines see less of an improvement because render speed is not accelerated by the GPU as much as encoding speed is.  If you coming from a single flat clip, the increase in export speed will be much more apparent.  I don’t use AVCHD footage, but when encoding from those types of source files, the GPU can accelerate the processing intensive AVCHD decode, as well as the MPEG2 or H.264 encode.  This should lead to more dramatic performance improvements, especially on laptops, where available CPUs are not as powerful as their desktop counterparts.  When I first reviewed the accelerator software earlier in the year, I anticipated that laptops would be even better served by a GPU based accelerator than desktops.  If you frequently make these types of exports from a laptop with a Quadro GPU, this plugin will be worth it.  In the desktop world it isn’t as simple, since the CPUs are not so weak by comparison to the GPU. If you make DVDs or BluRays for a living it will be a no brainer, but otherwise it all depends on how much of your day you spend waiting for exports to complete.

There are still a few other issues to be worked out.  The Adobe Media Encoder CS4 is designed to minimize the decrease in system performance during background rendering by pausing exports during timeline playback.  This may be necessary on lower end systems, but eight-core workstations should have power to spare, before we even factor in the GPU.  This prevents Adobe Premiere CS4 from really utilizing the CPU power being freed up by the GPU acceleration, because any time it is called upon, the render gets paused anyway.  Adding the option to disable that functionality would be beneficial to both Elemental Accelerator users, and anyone else with a high-end workstation, so that they could truly multi-task their system.  Until then, its probably best to work outside of Premiere Pro during your accelerated exports, to truly take advantage of your available system power.

With NVIDIA’s Digital Video Pipeline in the works, its easy to see where this could be going.  Once they have SDI video inputs available, we could see realtime capture directly to H.264 and MPEG2 files, among many other things.  With professional I/O, the available GPU power could be used to turn the video card into a full editing accelerator card.  With SLI Mosaic and other new developments with the QuadroPlex, I can see it being scaled up to 4K frame sizes with realtime performance, and I am definitely looking forward to that.

It has been a long time since I have posted, but more is coming soon.  I spent the last few months working at a summer camp, but I am now getting back in the swing of things.  NVidia sent me some more cool new things to try out, so I will be posting about that soon.  Also at some point I should probably start posting about my extensive experimentation with the Canon 5DMkII.

Convergent Design had both of their digital recording devices on display.  The Flash XDR has been available for a while, but I have been waiting for the smaller NanoFlash.  It can record full 1080p HD-SDI or HDMI onto CompactFlash cards in the same MPEG format as Sony’s new XDCam-HD422 gear.  It is absolutely tiny, at about half the size of a 3.5″ hard drive, and is solid state.  I can envision a number of uses for it in rigging small camera’s on vehicles and in other rough spots.  The most challenging spot is usually as a backpack recorder for a helmet cam.  If Iconix ever releases a smaller CCU for their Studio2K, the backpack may someday become a belt.  The HDMI input allows cheap camcorders to record at much higher quality than their internal storage allows.

One other thing that I didn’t notice until after the show was over, was that NVIDIA announced a new SDI I/O solution, which should become available later this year.  SDI output is nothing new for NVIDIA, and has been available as an option since back when the AGP based QuadroFX 4000 was the top of the line.  I had heard over two years ago that SDI input capability was in development, but the Quadro Digital Video Pipeline has been the first sign of it becoming a reality.  Based on the developments in GPU accelerated encoding with the Quadro CX, I predicted last year that this must be coming soon.  I am still curious to see the details on how this will be implemented at the software level, and more specifically, what types of realtime HD compression will be supported.  The fact that it supports multistream capture is somewhat unique, and will be very useful as stereoscopic video production becomes more common.

That pretty much sums up the main things that stood out to me at NAB.  I have deliberately steered clear of discussing a few products that I will be using a lot in the near future.  I plan to post much more detailed info on those products and the related workflows once I have experienced them first hand on large projects.

Sony has a number of new toys that span the budget gamut.  On the high end, the SRW 9000 is an HDCam-SR camcorder with capabilities similar to the F23, and an all in one shoulder-mount formfactor like the F900.  With option boards, this unit is capable of recording up to 60fps at full raster 1920×1080, with up to 10bit 4:4:4 RGB of color information.  The next step below SR is really XDCam-HD422 at this point, since regular HDCAM is basically obselete.  The new PDW-800 basically replaces all of the functionality of the F900 (30i/p, 25/44p, etc.) plus the advantages of full 1920×1080 recording in 4:2:2 instead of HDCam’s 1440×1080 at about 3:1:1.  The 50Mb files can also be edited in their native form in almost any NLE, and can be accessed in a non-linear fashion directly from the storage disk. (No rewinding, preroll, or realtime capture required)  The new camera even has a network jack right on the side for copying the recorded files to a network, and can dump the proxies to a USB flash drive if you want to go that route.  That won’t be necessary for any workflow I will use, since Premiere, FCP, and even Avid can access the full resolution files directly without import conversions, for truly native editing.  Finally a workflow that is actually designed to “work” and “flow.”  Sony also had a number of new LCD monitors on display, including a 4K one, two circular polarized 3D screens, and two new 30″ and 17″ additions to their BVM line of LCDs.  The 3D screens look great, and while I couldn’t confirm the inner workings, with a single input, it seems to use an interface technology that would be compatible with Cineform’s new Neo3D software that I have been demoing all week.

The only thing I was really impressed with at the Panasonic booth was that they created an Economy line of P2 cards, but I have yet to find the specifics on the new pricing levels.  With $900 being the minimum price, an economy option was definitely needed.  SxS cards at least have SDHC based replacement options, which is especially ironic since the P2 cards are rumored to physically contain 4 SDHC cards.  Panasonic also has a new 3D plasma screen, based on sequential alternating frames, which requires active 3D shutter glasses.  I have also heard they have a new 10bit 1080p projector for $2400 that I should go check out tomorrow.

NVIDIA has released OSX drivers for the QuadroFX 4800, so it seems that that will probably become the highend card of choice for those purchasing the new MacPro, which was recently revised with Nehalem based Xeon CPUs.  I have been pleased with the performance I get from my equivalent QuadroCX card, and I am looking forward to trying a QuadroFX 5800 on my SpeedGradeDI system sometime in the near future.

Other new features in Photoshop CS4 that require OpenGL acceleration will benefit work on even regular sized images.  Images viewed at magnification levels that are not even multiples (1/2, 1/4, etc.) are now displayed at much higher quality, as well as being much faster and more fluid, since the preview is being scaled in the graphics card.  You can smoothly zoom to any level instead of the previous default 100%, 50%, 33%, etc. and the image will look perfectly clear.  You can also rotate the previewing plane without permanently effecting the image quality or dimensions, and continue to interact with the image at that angle.  Although it initially seems trivial, after a bit of thought I can conjure up a variety of important but obscure uses for this capability, most of which relate more to art and design than video post-production.  There is also a new pixel grid that is available to clearly dilineating the boundaries between individual pixels at extremely close zoom levels (>600%)  I am honestly not sure why this requires GPU acceleration, but it is unavailable unless you have a supported OpenGL graphics card.  My only complaint about the new functionality and acceleration found in Photoshop CS4 is that Adobe saw fit to specifically skip its implementation in the 64bit version of Windows XP.  Since this is the highest performing version of Windows available, Adobe’s failure to directly support it will cause its users continued frusteration for some time to come.

The only possible relief on the horizon in that regard is that Windows 7 seems to have been fast-tracked by Microsoft due to the Vista issues.  Speaking of future developments, hopefully we will soon be able to display 10bit color from Photoshop onto a Dreamcolor LCD via DisplayPort.  I suspect that this may not be available until CS5, but I am sure it is coming, based on the increasing level of hardware support on products that are now coming to market, with the Quadro CX leading the way.

The CS4 applications that will see significant performance gains from hardware acceleration, are After Effects, Photoshop, and Premiere Pro.  The improvements in After Effects and Photoshop will also be evident with any other previous generation high end GPU, while the new hardware accelerated H264 encoding support for Premiere Pro is specifically tied to the new Quadro CX card.  NVidia has also recently announced the Quadro FX 4800, with basically identical hardware specifications to the Quadro CX, and which retails for about $200 cheaper.  The extra cost is buying you access to the CUDA based RapiHD H.264 encoder, that is available in no other form besides in conjunction with the Quadro CX card.  If you have no need for accelerated H.264 encoding, you could consider saving $200 with the FX 4800, but I envision the possibility of NVIDIA releasing more CX-only tools for creative professionals, since that card is targeted towards that specific market.  Not to be overlooked, NVIDIA has also released the Quadro FX 5800, with an incredible 4GB of memory, but that should only be needed by applications with the most intense processing requirements, and is a class above NVIDIA’s previous Quadro products.  The Quadro CX is compatible with the same HD-SDI output daughter card that the previous Quadro FX 4600 and 5600 cards used, for broadcast and post-production applications.  Hopefully we will see more software applications directly supporting that interface card in the near future.  The Quadro CX is based on the same core architecture as the new GTX 200 series of consumer cards, while the Quadro FX 4600 was based on the same technology as the GeForce 8800GTX, which is now two generations out of date.  It is to be expected, that there is an all around performance increase with the new cards in almost any application, but Adobe has been specifically adapting their software to leverage the power of these graphics processors.

Of the many applications in Adobe’s new CS4 Suite, After Effects is the one that most fully and effectively integrates the power of GPU acceleration to increase processing power and application responsiveness.  Many of these features are not new, but by nature of the way they are designed, grow more powerful as GPU performance increases.  OpenGL allows most of the 3D processing required for advance compositing to be offloaded to the GPU for dramatic increases in performance and responsiveness.  There are also many plug-ins and effects that specifically take advantage of GPU power.  Synthetic image generation like noise and fractals, as well as artificial 3D blurring are some of the best fits for effective GPU acceleration in AE.  Many of these improvements are only implemented for faster previews unless the user specifically selects OpenGL exporting, due to a possible loss in quality based on the lack of precision of OpenGL, but certain effects such as the new “Cartoon” vectorizing filter, that experiences dramatic (30x) rendering improvements with GPU processing, utilize GPU acceleration both for rapid previews and for accelerating the final export render.  In most of the synthetic object (Noise, fractals, shapes, blurs, etc.) previewing tests I did in AE, OpenGL acceleration with the Quadro CX provided a ten fold increase in performance over CPU based rendering.  This is the difference between an interactive experience, and a plan-next-move-while-rendering workflow.  While any graphics card with OpenGL support can accelerate processing in After Effects, as compositions and projects become larger and more complex, the benefits of the Quadro CX’s increased memory and processing power will come into play.  Complex projects will experience a greater increase in performance than simpler ones, when upgrading to a higher end GPU.

The next program in the CS4 suite to experience major performance improvements through GPU acceleration is Photoshop CS4.  Most of these improvements come from newly added implementation of OpenGL processing, and therefore, like After Effects, they are not specifically tied to the new Quadro CX.  But the power of the new Quadro CX makes the benefits of these improvements more dramatically obvious, especially on larger images.  These improvements in Photoshop are fairly extensive, and I will review them in detail in the next posting in this series.  As far as the Quadro CX is concerned, with its large 1.5GB cache of onboard memory, it is more than capable of handling the largest and most complex operations that almost anyone would attempt in Photoshop.

Adobe’s Premiere Pro CS4 also takes advantage of the GPU in a few less significant ways, for basic effects.  The one totally new aspect that the Quadro CX brings to the table is accelerated encoding, specifically encoding to H.264 with the new RapiHD encoder from Elemental Technologies.  This is the primary marketing piece specific to the CX card, and my third and final post in this series on the Quadro CX will be about CUDA and its implementation in this new encoder.

After all of the Adobe tests, I had two other programs that utilize the GPU that I wanted to try on this new high end card.  The first was Iridas SpeedgradeDI, which is specifically programmed to run on NVidia’s Quadro cards.  The base version running with DVI or 8bit DisplayPort attached monitors worked great, and nothing I could do with my limited knowledge of the program could even get it to drop a frame.  The real test for that application would involve connecting the optional SDI daughter card for true 10bit output.  Hopefully the 10bit color depth supported by the DisplayPort will eventually allow that level of monitoring without the high priced SDI daughter board.  The highest-end customers will still require an SDI output in order to use SDI interfaced external waveform and vectorscope tools, or live broadcast outputs.

Lastly, I ran my favorite program, Battlefield 2 to test out the card’s 3D rendering capabilities.  BF2 was released over three years ago, and therefore is not a cutting edge test, but it is my most recent high performance game.  With all of the settings maxed out, at maximum resolution on my 30″ LCD, I was able to get 99.9 FPS about 90% of the time, with the occasional dip into the low nineties for complex scenes.  Anyhow, the Quadro CX should be more than up to the task for those late night “stress relief” sessions with any modern 3D “application” if desired.

Once I had thoroughly tested the Quadro CX’s acceleration capabilities, I endeavored to verify the capabilities of it’s newly supported output interface.  I hooked my HP Dreamcolor LCD to the card via the new DisplayPort interface, hoping to get some taste of 10bit color output.  Unfortunately, currently none of the major applications I currently have installed are programmed to take advantage of this capability.  I do have a small utility from NVidia that displays 16bit TIFF files in 10bit color depth, and I can confirm that yes, there is a difference, and yes, the combination of the Quadro CX and the HP Dreamcolor does give you full hardware support for 10bit color display.  Hopefully in the future we will see updates and plug-ins that will unlock this feature in useful ways.  There is a 10bit capable SDI plug-in that NVidia released for After Effects 7 quite a while back, as a simple demonstration of their new SDI capability, and I am hoping to see an equivalent DisplayPort version for both After Effects and Photoshop, especially since NVidia and Adobe seem to be working together more closely these days.

Anyhow, if you are in the market for a new high end GPU, the Quadro CX has all of the processing power that most people could possibly need.  At its currently available price of about $1800, it has directly replaced the Quadro FX 4600, and no question is a superior product.  Now if you already have a 4600, the jump to the CX is not immediately necessary unless you are currently pushing your system to the limits, or you encode a lot of footage to H264.  The Photoshop and After Effects GPU support in CS4 will work nearly as well with a 4600, but there is a difference.  With any other previous generation card, you should see a significant all around performance increase with any application that leverages GPU processing power.

That information should give you a good general idea of what the new Quadro CX card is capable of, and I will be detailing the improvements in Photoshop CS4 and the RapiHD encoder in my upcoming posts.

Adobe has added many minor tweaks and improvements to all of their apps in CS4.  My favorite so far would be the new Adobe Media Encoder, which allows you to render and encode Premiere Pro and After Effects sequences  in the background, while you continue to work in the applications themselves.  This should help finally utilize all those extra CPU cores we all have buried in our systems.  There is also supposed to be more support for 64bit processing, especially in Photoshop.  The speech to text engine and enhanced dynamic link are the other features Adobe has been highlighting, but I don’t have as much use for those personally.  One thing that has been made clear by third parties, is that Premiere Pro is not finished, and that we should expect a major update soon.  I am not sure which details of that have been made public, but until that release, most third party extensions of Premiere will not be compatible with CS4.  Specifically I am awaiting support from Cineform for ProspectHD/4K, and from Matrox for the AXIO line of products.  Both companies have stated that their software will require support from the extended SDK in the update, so stay tuned for any news on that from Adobe in the near future.

The announcement of NVidia’s new high end GPU, the Quadro CX came as a bit of a surprise to me, since I had seen the specifications for what I believe will be the Quadro 5800, which should include 4GB of memory, and will be found in the new QuadroPlex.  The new Quadro CX card is very similar to the current Quadro 5600, with 1.5 GB of memory, and supports the same SDI daughter card, for broadcast preview and output.  This brings to mind the way that the GeForce9 series of consumer graphics cards were very similar to the previous line of GeForce8 cards.  The new Quadro CX seems most closely related to the Geforce 260 on the consumer side, but with a lower memory bandwidth for some reason.  One key new feature is the edition of DisplayPort output, which is made even more significant by the fact that is supports true 10bit color.  When combined with the new HP Dreamcolor LCD or similar display, this should allow true 10bit display of HD or 2K content without the need for expensive SDI interfaced components.  This is especially critical for desktop based color correction applications, many of which have been written specifically to be run on NVidia Quadro hardware.

Between better multithreading, and more use of the GPU, we should see significant performance increases.  The possibility of 10bit color out of CS4 is exciting as well, but I have not heard any details about that.  I have recently received one of the HP Dreamcolor displays, so I plan to post some info from my experience with that soon.

Elsewhere on the mobile recording front, I have been using some prototype units that record HD-SDI directly to SATA drives.  We tested them in some pretty extreme operating environments, and when used with Solid State SATA drives, they held up pretty well.  While we weren’t without our share of problems, the units were able to capture some pretty amazing footage when combined with an Iconix camera system.  I won’t post a full review until the creators have had an opportunity to solve some of the issues with the units, that our tests exposed.

Nvidia also released a new high end professional video card, the QuadroFX 4700 X2.  This card has two independent GPUs that can be harnessed together with SLI or used separately to drive 4 separate displays.  The stats are not much more impressive than the current top of the line 4600 and 5600 solutions, so they are really just updating the previous 4500X2 which was made obselete by the new generation of GeForce8 based cards released last year.