Twixtor is a plug-in from ReVision Effects that allows you to change the frame rate of your footage thru motion compensated frame blending.  While it can be used to add frames for slow motion effects, I have found that I get much better results when removing frames, such as when dropping from 30fps to 24fps.  Regardless of the specific settings, Twixtor takes a lot of time to render.  In our first tests on 8-Core Xeon systems, processing one minute of source footage required one minute of render time.  Now with Intel’s new Nehalem based CPUs, and recently their even newer Gulftown 6-core chips, we have seen that reduced by about fifty percent, to a half hour per minute of source footage, which is still a long time, but feels great compared to where we were a year ago.  Since our footage re-link process is based on frame counts, we have to process our entire source clips in order to take advantage of that level of workflow automation, even if we are only using the last ten seconds of a 14 minute take.  Obviously there are ways around this, but we currently have more render time available to us than man hours, and it gives us more flexibility later on anyway, so we just let it go.  We took advantage of every night and weekend during creative editorial to Twixtor every clip that made it into the rough cut, and now we just have to link to that bank of processed footage to conform our cuts in CS4.  The fact that all of Canons DSLRs now support 24p should alleviate most of the frame rate and Twixtor issues in future projects.

Besides frame rate issues, Canon DSLRs present another unique challenge, in regards to color space and bit depth.  Many professional video codecs store the color values in the range between 16 and 235, of the 256 possible 8bit options.  (The reasoning for this is fairly complicated, and relates primarily to legacy analog video signal issues)  This limits pixels to 220 levels for each color in most 8bit codecs, but the MOV files from the Canon DSLRs use the entire 256 possible options (0-255) for each color.  This increases the number of possible values for each three color pixel by over 50%, (220^3 vs 256^3) but also means that converting your DSLR footage into most other 8bit formats will result in one of two issues: either the extreme values will be clipped, losing detail in the highlights and the shadows, or all of the dynamic range will be squeezed into the reduced sample-space, meaning certain intermediate values are going to be merged together if you edit in an 8bit codec.

Clipping was the most likely possibility in most existing applications prior to the release of Quicktime 7.6.2 in mid 2009.  Previous to that point, Quicktime displayed Canon clips incorrectly (clipping the values beyond 16-235) but after that update was released, most applications that used Quicktime to decode DSLR footage, were able to access the entire dynamic range of the source clips.  This support is not a foregone conclusion though, since DSLR files could be imported with a more generic MPEG4 decoder without Quicktime, and still be displayed incorrectly.  Even with properly calibrated import processes, compressing the 255 possible values for each color channel into the limited 220 values that most 8bit video formats offer, will lead to a loss of precision, and a potential increase in color banding, especially if you plan to color correct the footage later.  A 10bit video format will offer four times as many possible legal color values, and will be able to store all of the original image data with precision to spare.  Once you have color corrected your footage, and any visual effects are complete, an 8bit distribution format may be sufficient for most uses, but any image processing that takes place on the original files before you apply the “look” that you want, should definitely be processed in at least 10bit color space to preserve as much of your original image information  as possible.

When editing DSLR footage in Avid, DNxHD is the recommended intermediate format.  DNxHD files can be encoded in either HD 709 (16-235) or RGB (0-255) color space, but any DNxHD files encoded in RGB are converted to HD 709 upon import into Media Composer, regardless of the original output setting.  Therefore any DNxHD MOV files generated elsewhere for ingest into Avid should be exported at 16-235 to match Avid’s target color space, for a lossless “Fast Import.”  On the other hand, when importing DSLR footage into Avid, you should select “Computer RGB (0-255)” as the SOURCE color space, in the “File Pixel to Video Mapping” options.  (Rec. 709 is always the TARGET color space for DNxHD MXFs in Avid)  While importing with the 0-255 setting retains the full dynamic range, it still squeezes the entire range into the 16-235 gamut.  That loss of precision should not be as significant as viewable dynamic range for an offline edit, but if you planning to export your Avid sequence as your master without a separate conform, you should consider using a 10bit codec in Avid, like DNxHD 175x.  That will allow you to maintain both the original dynamic range and the bit depth, at the expense of higher storage space requirements.

Once you have a re-linked timeline of high quality 24p footage, there are still a few more steps that can be taken to cleanup the footage.  Dead pixels should be the first thing on the list to deal with.  Dead pixels can be caused by physical debris on the sensor or lens of the camera, or by an electronic malfunction with one of the photo-receptors on the CMOS sensor.  The result is the same regardless of the cause, with one of more pixels locked at a static value throughout the shot.  The simplest way to fix this is to cover the effected pixels with information from the surrounding area.  One procedural way to fix this is to duplicate the layer of footage in an AE comp, and mask out a similar section nearby and cover it.  (If you have a horizontal row of three dead pixels, mask the three pixels above them on a second layer, and then drop the top layer down one to cover the spot)  In most cases the duplicated data will be totally invisible, but be sure to QC the result.  If you are Twixtoring your footage to a different frame rate, fix the dead pixels before applying the rate change, otherwise the motion compensation process will cause the dead pixels to move around, making them much more difficult to remove in a procedural fashion.  The next step is to look for any rolling shutter artifacts, caused by the slight difference in time between when the top and bottom of the frame are sampled.  This difference in time can manifest itself in a number of interesting ways, including distortion, with the top of the frame seeming to “lead” the bottom.  It can also cause horizontal bands of brightness with quick flashes of light only being recognized by part of the sensor.  The Foundry has a plug-in called Rolling Shutter that can help reduce the image distortion caused by motion of the camera on smoother shots.  The horizontal bands have to be removed manually in a VFX process if you want to get rid of them, borrowing data from the preceding or following frames if needed.  The Canon DSLRs also exhibit some moiré and other aliasing issues due to the way they sample the low resolution video from the high resolution sensor.  The only way to really get rid of those artifacts is to selectively mask and blur the effected sections of the frame.  Lastly, if you are using Twixtor, QC the output for corrupt frames caused by the interpretation engine being unable to guess the proper motion of the moment in the shot.  If re-rendering with different settings doesn’t help, covering the bad frame with an original frame of footage from that moment usually solves the single frame issues.  Luckily the most difficult sections of footage to calculate motion compensate for, are usually segments of where using frame dropping conversion instead is undetectable, since the extreme motion should hide any stutter caused by the missing frames. (This is coming from a guy who is processing a lot of handheld combat footage)  Once these steps, as well as the rest of your visual effects work, are finished, you are ready to export and color, which should be similar to most other workflows at this point.

Canon DSLRs name each video file “MVI_####.mov” where “####” is a constantly incrementing 4 digit number.  It is easiest to identify individual cameras if each one is shooting a totally different series of numbers. (Like CamA = 1000′s, CamB=2000′s, etc.)  While there are no options in the Canon menus for manually setting the starting file number, there is way to trick the camera into starting at any number you want.  Reset the counter to zero in the menu, and then return the setting to Continuous.  Take a picture, and then use your computer to rename the JPEG file on the card to the desired number.  (In this case: 5000)  Put the card back in the camera and take another picture.  Verify that it incremented from there. (IMG_5001)  Format the card or remove it, and your camera will continue incrementing from that point. (So my next shot, will be IMG_5002, or MVI_5002 if you switch to video mode.)  You can reset it to any four digit number you want, and it will increment from there, unless you shoot with a card from another camera, without reformatting it first.  In that case, the camera will increase the file number if necessary, above any files already on the card.  The Canon 1D MarkIV has a few more options for filename prefixes, but the objective is the same, to create a uniquely identifiable series of filenames for each camera on your shoot.

Doing this will make it much easier to sort through your files, since you will be left with a separate numbered sequence of files from each camera.  I was recently on a two week shoot with twenty Canon cameras, and by setting their initial numbers 500 apart, (Since four digits gives you 10K possibilities, divided by 20) it was easy to sort the resulting files, even though due to logistics, I got some of the files many days after they were shot, and out of order.  If you set the clocks on the cameras accurately, the timestamps on the files will make it easier figure out which files are different angles of the same take.  Even with all that, I still had duplicate file names on that project, with 5600 files for that shoot, so renaming to a better convention will be important on most large projects.

Having a good naming convention for your media files can save a lot of time and trouble.  The info that needs to be contained within the file name will vary depending on the nature of the project.  For the Navy Seal movie I setup a convention that included Scene, Setup, Take, and Camera.  The next project I worked on was more of a documentary type shoot, with no script or scene numbers, so the file name was composed of Date, Camera, and an incrementing shot number.  The important things to keep in mind are: order the info intelligently for searching, keep place values consistent for sorting, (Use leading zeros) and make it unique within the first 8 characters for EDLs.  For the movie, 005A_02b.mov is: scene 5, first setup, second take, camera B.  If I want to see the other camera angles for this take, they should be nearby if the footage is sorted alphabetically.  And if I want to find other takes from that setup, I shouldn’t have to scroll too far. If during onlining I am looking for a specific clip based on the name, I go to the proper scene folder, and scroll down to the right setup, and start looking from there.  Now if you want to include descriptions in your filenames, put them after the unique identifiers up front.  This is because sorting descriptions alphabetically is relatively useless, but you can still search through them with regular OS tools, as long as the description is somewhere in the filename. (10_E_026_BigExplosion.mov)  And if your software at some point in the process truncates the name, there should still be enough info in the first 8 characters to identify the exact source file.

During the process of renaming, it is a good idea to keep a log of what you have done in case you make a mistake, and it forms a basis for content logging as well.  I do this with a simple Excel spreadsheet that records: Original Name (MVI_####), New Asset Name, Description, Comments, as well as CameraName, Date, FrameRate, or any other data that might be relevant depending on the project.  I also sort out the bad takes and delete the .THM thumbnail files during this stage. (The THM files are actually JPEGs that can be opened if you rename them, but I have never found any practical use for them)  Once I have this list of what I want the files to be renamed to, to fit the filename convention for the project, I convert the log into a renaming batch file.

To do this, save a copy of your log sheet as a “Text (Tab Delimited)” file from Excel. Then delete every column except the original name and the new asset name, and insert blank columns before, in between, and after those two columns. Fill each of those three columns with a unique character, like #, $, and %. Save your file and close Excel. Open the file in Notepad, and press CTRL+H to bring up the replace window. Replace “#([TAB]” with a command and the path to your media files. (“REN D:\ProjectName\Footage\”) Replace “[TAB]$[TAB]” with “.MOV[SPACE]” unless your log included file extensions, in which case just replace that with [SPACE]. Replace “[TAB]%” with “.MOV” unless your log included that, in which case replace it with nothing. (In WinXP the only way to insert a [TAB] into the Replace dialog box is to cut and paste one from the document, with CTRL+V.)  You should be left with a text file with a series of commands that each look something  like this:
REN D:\ProjectName\Footage\MVI_0001.MOV 057_T12A_NewClipName.MOV
Save this document and rename the file extention to .bat, and double click on it to run the batch.  This should assign new names that match your log to each of your files, and make it much easier to sort through your footage, if you designed your naming convention well.

The other thing you can do with you footage log is generate an ALE, so that if you are editing in Avid, your log notes can be linked to your clips within the program.  I will go into those details, and other tips for editing DSLR files in Avid, in my next post.

The second possible complication is brought to you by the fact that lots of HD footage is now being shot and edited at 24 progressive frames per second.   One solution is to make 24 frame progressive DVDs, which can work if you do everything perfectly.  I have yet to see this done properly, but Encore is adding more 24p support with every version.  I believe you can import and burn 24fps MPEG2 files to regular SD-DVDs in Encore, even though there are no presets for it.  I have yet to see a fool-proof flawless system for doing this, so I still add the pulldown myself for maximum compatibility and stability.  To do this, you once again need to use AE to convert a master HD export of your finished piece.  Import it into AE and leave it progressive with no interlacing interpretation, but reinterpret to 23.976fps if it is true 24p.  Insert the footage into a DV widescreen 23.976 comp and scale it, as described above.  Add to render que and use the same encoder settings as above.  In the render settings dialog, you still want to set the Field Renderer to Lower-Field-First, but you also want to add pulldown, in the dropdown menu directly below.  You will want to set that to the last option which is “WWSSW”, although honestly any of those would work fine for DVD.  You should see the output framerate change to 29.97 when you select a pulldown option, and you should be all set to render.  Hopefully this points people in the right direction in regards to getting the highest possible image quality to DVD from your high definition video projects.

Everything listed above is geared toward NTSC based production.  The first set of steps will also work for PAL, going from 1080i50 to 576i50, but your output render should be Upper-Field-First.  There is no pulldown option to go from 24p to PAL.  It can be done instead by increasing the footage framerate, but this causes the audio to need to be sped up by 4.2%.  Usually the speed change is not perceivable, but the final product will be a shorter duration than the original.  It also is not possible to generate any true PAL interlacing information from 24p footage with this method.