Optimizing for Speed

There are many factors that influence Isadora’s video processing speed, including the speed and type of your processor, amount of RAM installed on your computer, hard disk speed, the format in which the video files are stored, and several other factors. This section details several tips to help you get the very fastest frame rates from Isadora.

 Tip 1: When it comes to speed, the three most important hardware considerations are Graphics Card (GPU), the Central Processing Unit (CPU), and the Hard Drive.

Graphics Card (GPU)

In Isadora 2.0, the graphics card has become the most important consideration when it comes to speed. Most Isadora actors now rely on the graphics card to process video as do all FreeFrameGL (FFGL) plugins. The Core Image and Quartz Composer plugins on Mac OS also rely on the GPU. When you see a video stream input or output that says vid-gpu then you know that this video is being processed by your GPU. Generally speaking, any actors that use vid-gpu are going to give excellent performance, even with full resolution HD video.

If you have a desktop machine, and can install a more expensive video card, it might increase performance, but even today’s average GPU is going to provide excellent results.

CPU

It is always the case that higher CPU frequencies are better. At this time, a dual-core 2 GHz processor would be the minimum one would want to consider for serious work when working with higher resolution video. Multiple core machines like an i7 or i5 processor will make a noticeable difference.

Some older Isadora plugins (and all FreeFrame 1.0) plugins process video on the CPU. When you see a video stream input or output that says vid-cpu then you know that this video is being processed by your CPU.  For these plugins, CPU speed is the top consideration when it comes to speed. You should not expect these plugins to give good performance with high resolution images. If you must use them, try to keep your video resolution near to SD (standard definition) resolutions for best results.

Hard Drives

When it comes to hard drives, the fastest units are the newer SSD (solid-state) models. Because there is no physical head to move, access speeds, are phenomenally fast, especially when playing more than one movie. Such speed can be especially important if you are attempting to play multiple HD clips or using clips that require high throughput, like Apple Pro Res.

Second in speed would be eSata drives (internal or externally connected), followed by the external drive options of USB3, followed by FireWire 800, and finally USB 2. We do not recommend USB 2 when playing high-resolution movie clips from an external drive. When considering traditional HDD drives, a drive with a high RPM speed will improve performance. Drives with 10,000 RPM are better, with 7200 RPM being the recommended minimum. It is worth noting that many computers ship with 5400 RPM drives installed as standard, so requesting a custom build with a faster drive is recommended.

 Tip 2: Add as much RAM (Random Access Memory) to your computer as you can afford.

Modern operating systems rely on a feature called “virtual memory.” When the computer senses that some data that resides in your computers RAM (i.e. the extremely fast internal memory linked to your main processor) has not been used for a while, it will write it out to disk to make more RAM available to other applications. This is called “page swapping” and can adversely affect performance.

The best way to solve this problem is to add more RAM to your computer. As a general rule, 2 Gigabytes (GB) is a good base line. 4GB is even better. Page swapping will often take place on machines with less than 2GB of memory.

 Tip 3: Don’t run other applications at the same time as Isadora

Every running application consumes system resources. So, especially when using Isadora for a live performance, don’t run any other applications at the same time. Even if they aren’t “doing anything” they get some of the processor’s time, and thus slow down Isadora.

 Tip 4: Choosing the correct codec for recompression is crucial

High Definition (HD)

For HD resolution the Apple Pro Res codec (APR 422 or 4444) has performed best in our tests on both Mac OS and Windows, especially when running off an SSD hard drive.

When running Windows, we recommend using AVI or WMV formats, because these movies will use native Windows routines for playback, as opposed to .mov files which require Apple’s QuickTime. 

NOTE: It not recommended  for Windows users to run mp4 video files with Isadora since the mp4 format is not able to run with the more efficient DirectShow playback engine, and can be unstable with the Quicktime engine.

For both platforms, we also recommend the HAP codec. (For more information about HAP, see this page: http://vdmx.vidvox.net/blog/hap) HAP is an open source format that provides excellent performance on Mac OS, and tremendous performance on Windows.

H264 is also a good choice, but has some important limitations (see below.) If you use H264, be careful to compress it using software that allows you to control the settings to achieve a reasonable target bandwidth. Some users assume that using the highest quality is the best choice, but this often offers only a slight improvement in image quality while introducing noticeable performance issues because of the high bandwidth.

H264 should not be your first choice!

The H264 is extremely popular because it makes small files with good image quality. But it is not the best choice to use with Isadora (or any interactive video software) for several reasons.

First, the CPU overhead required to decompress the image is significantly higher than for Apple Pro Res, Photo JPEG or DV NTSC/PAL.

Second, H264 is only designed to play forward. If you are a VJ or artist who is scrubbing video or you often use the position input of the Movie Player actors, then H264 is most definitely not for you. If you play an H264 movie backward, or you scrub the video back and forth rapidly, you will see the CPU usage increase significantly. This is simply because the codec is not designed to deliver the images in reverse order.

By contrast, the Apple Pro Res, HAP, Photo JPEG and DV formats render each frame individually (i.e., the current image is not related to the previous one.) Therefore these codecs are very fast when jumping to an arbitrary location within the clip and will play just as well backwards as forwards.

You can use a number of software tools to recompress your files, but perhaps easiest is the Pro edition of Apple’s QuickTime Player. See Tip 10 for instructions on how to use QuickTime Player to compress your video.

Photo JPEG

At this point Photo JPEG should probably be considered a “legacy” format, but it still offers meaningful speed improvements when playing video from your hard drive using the CPU (seen as vid-cpu streams). While the files will not be as small as with other codecs, the Photo JPEG requires less processor power to decompress the images when compared to many other codecs.

When compressing using Photo JPEG, we suggest using a quality setting of “high” or “medium” (75% or 50%). Using a quality setting of “best” produces very large files that are not significantly better in image quality.

DV NTSC or DV PAL

For standard DV resolution, PAL is best for 720 x 576 and NTSC is best for 720 x 480. The NTSC and PAL DV codecs exhibit many of the same benefits as Apple Pro Res and Photo JPEG, and are relatively efficient to decompress.

 Tip 5: The duration of a movie has little or no impact upon performance.

We often are asked: “Can Isadora play movies that are an hour long?” The answer is “yes,” because, in fact, the duration of a clip has little to do with playback performance.

The video playback software within the operating system only reads a small amount of the video file at a time, usually about a half-second in advance of where the current playback position. Thus, whether a clip is 1 or 1000 minutes long has little impact on performance. Far more important are the factors listed in the tips above.

 Tip 6: Understand the new optimize input on the Movie Player

Isadora v1 used Apple’s QuickTime to play video on both Macintosh and Windows. However, the updated Movie Player in Isadora v2 works differently. The default playback engine is now AVFoundation on Mac OS X, and DirectShow on Windows. The advantage of using these systems is speed: you can play more high-resolution clips at the same time than you can with QuickTime.

However, AVFoundation and DirectShow do not shine when it comes to interactive control: for example, they respond poorly when “scrubbing” a video by sending values into the ‘position’ input of the Movie Player. But, since interactive control was the reason Isadora was invented, we have made sure you can still access QuickTime as a playback engine option. Even though it may offer reduced HD performance when compared to AVFoundation or DirectShow, there are numerous advantages when it comes to interactive control.

That’s where the optimize input parameter comes in. If you leave it set to performance (the default) Isadora will attempt to use AVFoundation or DirectShow if possible. But if you set the optimize input to interaction then Isadora will use QuickTime if possible.

Why do we say “if possible”? Because AVFoundation and DirectShow cannot play movie files compressed with certain video codecs. QuickTime’s Animation codec is a good example: neither AVFoundation nor DirectShow can play this format. So, even when you’ve set the optimize mode to performance, playback will revert to QuickTime if neither AVFoundation nor DirectShow can play that type of movie.

Similarly, if you set the optimize input to interaction and QuickTime cannot play the movie, it will switch to AVFoundation or DirectShow. (Currently, this would only happen if you set optimize to interaction and tried to play a WMV or AVI file, which only DirectShow can play.)

So how can you tell which system is playing your movie? The Movie Player in Isadora 2.0 has a new output called pb engine. If this output says AV, then AVFoundation is playing your movie. Similarly, DS means DirectShow, and QT means QuickTime.

 Tip 7: Stick with one frame rate for all your videos.

Isadora will happily play videos of any frame rate within the same file. But for the smoothest playback performance, you should use the same frame rate for all your videos and set the Target Frame Rate in the General tab of the Isadora Preferences to the same rate.

Furthermore, you should set the Refresh rate of your monitor or video projector to a multiple of your chosen frame rate if at all possible. (Consult the manual for your monitor or projector to learn how to do this.)

For NTSC video rendered at 29.97fps:

  • Set the Target Frame Rate to 29.97 fps
  • Set the Monitor/Video Projector refresh rate to 60 Hz.

For PAL/SECAM video rendered at 25fps:

  • Set the Target Frame Rate to 25 fps
  • Set the Monitor/Video Projector refresh rate to 50 Hz.

For videos transferred from film at 24fps or 23.978 fps:

  • Set the Target Frame Rate to 24 fps or 23.978 as appropriate.
  • Set the Monitor/Video Projector refresh rate to 50 Hz.
 Tip 8: Turn off the Video Capture when you’re not using it.

The Capture Control actor allows you to turn live video on and off from within an Isadora scene. If you are not using live video in a section of your piece, then use this actor to turn it on and off at the appropriate moment.

 Tip 9: If you use a CPU based plugin (vid-cpu) keep your video resolution as low as you can.

When processing video on the CPU, the biggest burden any real-time video software faces is the size of the image it has to process. This data has to be brought in from the hard drive or from a live camera input, moved through any effects, and sent to the graphics card for output. It’s a simple relationship: as resolution increases, the speed at which those frames can be processed decreases.

In addition, there is the CPU overhead necessitated by decoding the video from it’s compressed format to the uncompressed format that can be processed by the video effects and/or graphics card.

So, while Isadora has no limitation on the resolution of the video it can play, if you want good frame rates, it is best to keep the resolution as low as possible to achieve the best frame rates.

To give you a sense of the amount of data being processed, take a look at this table:

 

Resolution

Megabytes / Frame

Megabytes / Sec. @ 30FPS

320×240 QVGA

0.3 Mb

9.2 Mb/S

720×480 DV Video

1.3 Mb

41 Mb/S

1920×1080 HD

8.2 Mb

248 Mb/S

Thus you can expect a single frame of full res 1080p video to take 26 times longer to process than a QVGA frame and six times longer than a frame of DV. Moreover, consider the situation when you’re cross fading from one scene to the next: if you were playing two 1080p HD clips, you’d be pushing around nearly 500 Mb per second. That’s approximately 62% of the maximum bus bandwidth of a top-of-the-line 2011 Mac Book Pro. Playing a third HD clip would put you at 93% of the maximum bus bandwidth of the same machine.

Note that the final column in the table above is the decompressed bandwidth of the image; the actual bandwidth required when reading the clip from disk can be considerably lower depending on the compression scheme used.

  BONUS TIP: If you are outputting to video projector with less than HD resolution (e.g., 1024×768) there is no reason to play a full res 1080p HD video. Recompress or crop the video files to match the resolution of your output device.

  BONUS TIP: If you are letterboxing 16:9 video into a 4:3 output device (e.g., 1024×768) calculate the resolution you are actually using. For example, on a 1024×768 device, the resolution of a 16:9 image that uses the entire width of the display would be 1024 x 576. (used height = display width * 9 ÷ 16) Using a movie rendered at 1024×576 saves you 25% of the full 1024×768 bandwidth.

 Tip 10: If you use a CPU based plugin (vid-cpu) keep all video resolutions consistent

When Isadora has to combine two vid-cpu streams of different resolutions (e.g., with the Video Mixer actor), it has to scale one stream to match the resolution of the other. (Which of the two streams is rescaled is determined by the “Video Image Processing” section of the Video tab in the Preferences.) Scaling is slow, and so you want to avoid it as much as possible. If all of the video streams you are processing are the same resolution, you will get the best possible performance. Note: vid-gpu or vid-ci streams do not suffer this limitation; scaling on the GPU is “cheap.”

 Tip 11: How to use Apple’s QuickTime Player (Pro Edition) to recompress your movies.

Below are instructions on how to use the Apple’s QuickTime Player to store your movies in this format.

NOTE: While QuickTime and QuickTime Player are free, the Pro edition required to recompress movies costs $29.99 at the time of writing. You must have the Pro Edition to recompress movies using QuickTime Player.

1)    Open QuickTime Player by double clicking on its icon. From the File menu select “Open Movie” to open the movie you want to recompress.

2)    Choose File->Export or press cmd-E to call up the export dialog.

3)    After the window appears, click the Options button to show the “Movie Settings” window.

4)    Then click the Settings button.

5)    Choose your desired compression format using the pop-up menu under the heading “Compressor”.

6)    Set the slider under “Quality” to the halfway point, which should be labeled “Medium.” You can use “High” if you like – you’re movies will be somewhat larger but the quality of the image will be better.

7)    In the text box to the right of “Frame Rate” type 30, if it is not already set to 30. (Use 25 if you are compressing PAL movies.) Uncheck the other two boxes that say Key Frame and Limit Data Rate.

8)    Press the OK button.

9)    You’re back at the Movie Settings window. Click on the Size button.

10) Click the “Use Custom Size” radio button. Two text boxes marked Width and Height will appear.

11) Type your desired sizes into the Width and Height text box.

12) Press the OK button.

13) You’re back at the Movie Settings window. Press the OK button.

14) Choose a name for your movie (it should be different than the old name) and press the Save button.

15) This process can take a while, depending on how long your movie is. A progress dialog (a window with a little blue thermometer type display) will appear to show how things are going.

16) When the progress dialog disappears, your movie is done.

17) Open the new movie and play it to check that you are satisfied with the results.

 

To see the difference in size between the original movie and the newly compressed one, do the following:

1)    Bring the original movie to the front by clicking on its window.

2)    Choose Movie->Get Info to show a window that gives information about the movie.

3)    Choose “Movie” from the pop-up menu on the top left.

4)    Choose “General” from the pop-up menu on the top right.

5)    Look at the “Data Size” and “Data Rate” in that window. Take note of the values.

6)    Close the window by clicking on its close box in the top left or by choosing File->Close.

7)    Now bring the new movie to the front by clicking on its window.

8)    Follow the same procedure starting at Step 2 to show its Data Size and Data Rate.

9)    You’ll be able to see how much smaller the movie is and how its data rate has been lowered through the resizing and compression process.

 Tip 12: Windows-compatible software tools for recompressing movies (codec conversions).

Note: This is not an complete list. Please check the Isadora forum for up-to-date recommendations. WMV and AVI files are the preferred codec for HD playback on Windows in Isadora, and are able to use the newer DirectShow playback engine.

MPEG Streamclip (freeware): (Mac/Windows) Batch converts, and able to do a wide range of codec conversions. 

Microsoft Moviemaker (freeware): Allows exports to WMV format but not QKT (and not HAP). Does export MP4 in a usable H264 format. NOTE: All mp4 codec files on Windows will be read by the Quicktime engine, which is less efficient than the DirectShow playback engine and can be unstable on Windows.

Expression Encoder (Freeware): Useful for batch processing (converting multiple files at the same time), to WMV codec, but the interface is a little trickier to learn.

Batch Export Utility (Freeware): An older application, but works well with all Quicktime codecs and HAP.

AviDemux (freeware): Simple to use, edit and re-encode AVI files.

Handbrake (freeware)

Sony Vegas Movie Studio($):  Low cost commercial software that supports many output formats with full control of dimensions etc.

Adobe Photoshop ($): Commercial imaging software. The latest versions allow conversion of video files to Quicktime and other formats.

 

 

 

Comments are closed.