isovideo Legato: GPU file based 4k/UHD motion compensated frame rate conversion

The purpose of frame rate converters is to make video look smooth without duplicating or dropping frames (causing judder), and without objectionable visual artifacts. This is achieved by converting an input frame rate to match a displayed or rendered frame rate using Motion Compensated Frame Interpolation (MCFI) with dense motion fields. Legato handles frame sizes up to 4k or even 8k. It is an engineering challenge to make high quality MCFI work in real-time on 1080p60 video in a server/LAN/file-based environment.

Principal Features:

Legato is our real time motion-compensated frame rate conversion technology. It's main features are:

conversion from film rates (24FPS) to other frame-rates with high quality (Mobile 1080pNTSC24 to 1080p25).
superior temporal conversion allows material to fit a time slot without introducing judder due to frame dropping/duplication. Conversion can either be applied globally, or or to localized section specified by timecode.
time-code support, including on-video display and pipeline timing communication between isovideo tools.
Legato has very robust scene-change detection, preventing spurious motion artifacts at scene changes (Aspen 1080pNTSC30 to 1080pNTSC60).
repeated input frames are detected and dropped for smoother motion, while preserving audio/video delay.
imperceptible spatial distortion from scenes with complex occlusion (Park joy 1080p50 to ntsc60p).
resilient handling of camera flashes, fade-ins/outs, cross-fades (Controlled burn 1080i60 to 1080pNTSC24).
without the use of expensive high-speed cameras and facilities, Legato can provide cost-effective slow-mo (high upsample ratios) effects, particularly for some sports events.
Legato has precise timing. If you specify exact output : input ratios as integers, Legato will reconstruct output frames at exactly the expected points in time with respect to the input sequence, ensuring no drift in relative audio/video delay.
Legato has a minimal frame pipeline, so by default, the first input frame appears on the output, thereby simplifying the timing with respect to other parallel signals, including audio.
Legato requires no adjustments that depend on scene content in order to get good results. All benchmarks and sequences on our viewing page were made with the same Legato settings, with the obvious exceptions of file names, frame rate conversion ratios, and display frame rates. Most competitor's FRC algorithms require time-consuming experimentation with settings, depending on the source material.
performs 16-bit I/O for each Y,Cb,Cr component, and 32-bit internal video processing for superior cascadability and reduced noise.

Accurate Simulated Motion-blur

Legato provides an additional sophisticated, accurate simulated motion-blur capability to reduce judder on playback, when downconverting from 50, 60, 120 frames/sec to lower rates such as 24, 30, 48.

manually controlled motion blur can be applied using an output shutter angle setting,
simulated, accurate motion-blur for playback judder reduction, while saving an average of 15% H264/HEVC compressed bandwidth without sacrificing visual quality.

The smooth video from Legato is appreciated on a computer monitor or a big-screen Internet TV. Video quality has become a key commercial issue for content providers, since home displays have grown larger, and consumers expect more. Most consumer display systems do not have anywhere near the compute power to perform real time frame rate conversion, so they are only able to drop or duplicate frames when streaming video from other regions.

Judder Reduction Examples

Note: When adding simulated motion blur for FRC from 50p to 24p, some fine lines/details may appear less sharp than in the original 50p when viewed frame-by-frame. This same motion blurring would have occured on a real 24p camera.

Cinema Application - Low Frame-Rate Conversion

Viarte (Legato) frame rate conversion was used successfully for footage conversion (converting from 1980x1080p29.97 to 1080p23.976) for independent director/producer Karina Epperlein's latest feature length documentary "Finding the Gold Within". The film will be world premiered at the Mill Valley Film Festival on October 3 and 4, 2014. To learn more about the film and the filmmaker, and to see the trailer here. The following three clips are excerpts from the film - courtesy of Karina Epperlein.

High Performance

Legato obtains its high performance using NVIDIA CUDA technology running on Graphics or GPU cards. Fast processing allows a single processor system to perform more than one conversion at a time, or perform more than one type of conversion (e.g. FRC, deinterlacing, inverse telecine, scaling and de-noise, etc.) on the same input stream, or to dramatically speed up processing file-to-file.

The Legato Frame Rate Conversion Technology