Viarte/Legato: GPU File based MC Frame Rate Conversion
Sharp - Smooth - Superior Video Quality

Simulated Motion-blur for Effective Judder Reduction

What is Video Judder?

Judder & Motion Aliasing

Judder is perceived uneven or jerky video playback arising from movement of objects or edges or detail from one frame to the next. We evolved to see objects with smooth motion in the real world, so judder is simply a perceived artifact of uneven motion due to scene sampling. How each person perceives judder varies, but it is related to:

Note that modern, large, high-brightness displays will make judder issues much more objectionable, so the trend is for judder problems to get worse unless addressed properly.

The perception of judder is reduced at higher frame rates because the motion of objects is reduced between frames. In principle, highly detailed, high contrast, with lots of detail, and moving fast close to your eyes may require very high frame rates (maybe several hundred frames/sec) to be perceived without judder. However, the higher the frame rate, the less likely that random, uncontrolled scenes sampled by a camera will contain perceived judder.

Motion aliasing is not an issue when you view the world around you - the eye doesn't have a "sampling frequency" as such. However, lighting of objects modulated by AC power frequencies (usually at 50/60Hz, or double these frequencies for full-wave rectification) may induce a perception of judder in direct viewing by imposing contrast sampling that is strong enough to override the normally smooth perception of motion. Video frame rates and monitor refresh rates impose a similar effect to illumination flicker.

The famous wagon-wheel effect is a sampling effect due to the fact that our vision system also tends to match a wheel spoke to its nearest similar-looking spoke in the next frame, and not some random one. This effectively constrains the degree of motion, thereby actually reducing judder. However, if the nearest matching spoke is the wrong one, the wheel's rotation doesn't match the vehicle's motion. Regular sampling can cause smooth motion in one direction to translate to a perception of smooth motion in the other - a similar effect to spectral aliasing. However, motion aliasing can also occur as an instantaneous matching effect from frame to frame - it is not strictly a frequency domain phenomenon. Higher frame rates reduce the likelihood of motion aliasing in a random or uncontrolled scene by reducing the scene-to-scene motion and thereby reducing the likelihood of matching to the wrong spoke.

When viewing a monitor, judder is due to the the render rate (the number of new frames per second to be sent to a display) and the display refresh rate, and also due to the relationship between these values. Perceived judder (but not motion aliasing) can often be reduced down to any fundamental judder due to the display refresh rate by using Motion-Compensated Frame-Rate Conversion.

In low frame-rate film, directors and photographers have tight control of what goes in the camera - so they can reduce the effects of 1, 2 and 3 above by changing the scene and its lighting, and by reducing extreme camera motions. However, to deal with daylight action scenes, they usually resort to adding motion blur by increasing camera shutter angles or exposure times, thereby reducing the effect of 4. See our Frame Rate Options for Shooting "Future-Proof" Video for more on this subject.

Blur has some unfortunate effects - the eye may track a blurred object, and see it as blurred. Also motion blur is extremely difficult to remove, once added, so frame-rate conversion to a higher frame rate may just produce overly blurred objects with smoother motion. This is because the eye has a tendency to view sampled scenes as having a blur commensurate with the exposure time, sharp edge detail, and motion per frame. The motion per frame reduces with frame-rate conversion, but the perceived exposure time and sharp edge detail remain constant. If the edge detail of moving objects is too blurred for the motion and exposure time, it just appears as blurry.

Input Frame Rate Judder

One source of perceived judder is based on the rendering rate. For source material sampled at a higher frame-rate (but still below the display refresh rate), time per exposed frame decreases, and smoothness improves. For example, 60Hz rendering always looks better than 24Hz film rate on any synchronized display device (i.e. where refresh rate is an integer multiple of the render rate). Some people like the artistic "film look" of 24 frames per second, but for most of us, a frame rate of at least 50Hz is recommended. Not much is gained beyond 60Hz except in very fast motion situations, and where the image takes a large portion of the field of view.

Presentation Judder

This second primary cause of judder is due to switching between sets of repeated frames in order to achieve an average rate between them for the chosen display refresh rate. The two adjacent repeat lengths are given by

      Repeat_Lo = Floor(<refresh_Hz> / <render_frames/sec>)

where Floor(x) is the largest integer less than or equal to x, and

      Repeat_Hi = Ceiling(<refresh_Hz> / <render_frames/sec>)

where Ceiling(x) is the smallest integer greater than or equal to x. The effect of presenting "Repeat_Hi" frames in succession may have dramatically worse perceived judder than presenting "Repeat_Lo" frames, but it also depends on the extent of sharp moving edges in the sequence, and the subtended angle of viewing.

For example, 24 frames/second on a 60Hz refresh monitor is an average of 60/24=2.5 frames of presentation per monitor refresh, which requires an alternating 2,3 frame-repeat presentation cadence to get that average. The 3-frame repeat is essentially equivalent to a 3/60 = 50 millisecond (20Hz) frame-rate judder which can be very annoying without application of motion blur. In contrast, the 2/60 = 33 millisecond (30Hz) frame-rate judder is usually much less objectionable. The overall presentation judder is a weighted average of the two frame-presentation-time judder experiences.

See also our practical judder solutions and our proprietary superior quality motion-compensated frame rate conversion Legato.

Judder Introduced by Processing

In practise, judder is often inadvertently introduced when attempting to adapt material from one frame-rate to another by simply dropping or duplicating frames. This practice has been frequently used to avoid large objectionable conversion artifacts resulting from inferior frame rate conversion (FRC) in order to ensure each frame looks perfect. However, it introduces undesirable motion judder during normal playback, and should be avoided if possible. In the past, most FRC systems introduce artifacts that can be seen, even at normal playback speeds, and they have also used frame blending which can produce large "ghosting effects" which are extremely objectionable when viewed frame-by-frame, and sometimes even in normal playback.

Furthermore, frame blending really does not significantly reduce judder especially for panning scenes. The most advanced FRC uses motion estimation (ME) to improve the quality of interpolated frames, however, not all ME technologies are created equal, some ME introduces edge boiling effects or other errors such as break-ups, etc. for various fast motion scenes. Note that for extreme motion situations, the appearance of an interpolated frame may not be well defined, although the human vision system is quite adept at judging when the motion flow seems wrong. The optimum balanced solution is to achieve both smooth motion, while minimising conversion artifacts and generating "good looking" frames when viewed individually. Viarte (Legato) achieves smooth motion, with no objectionable frames when viewed at normal speed. Viarte typically generates over 99.5% perfect-looking frames when viewed individually, while artifacts are limited to minimal and localised regions. Most competing systems can barely average 80% good-looking frames, often with large-area ghosting or break-up.

Dropping frames is particularly difficult to repair (e.g from material shot at 30p down to 24p) as it not only introduces judder, but is irreversibly damaging, and cannot truly be undone as (1) it is difficult to reliably locate where frames were actually dropped, and (2) it is more difficult to repair using frame-interpolate due to the very large frame-to-frame motion that may occur. As far as interpolation is concerned, frame dropping is equivalent to halving the frame rate around each missing frame, so 30p motion looks like 15p.

Judder Reduction Examples

Note: When adding simulated motion blur for FRC from 50p to 24p, some fine lines/details may appear less sharp than in the original 50p when viewed frame-by-frame. This same motion blurring would have occured on a real 24p camera.

Excerpts from the Film: "Finding The Gold Within"

Viarte frame rate conversion was used successfully for its entire footage conversion (converting from 1980x1080p29.97 to 1080p23.976) for independent director/producer Karina Epperlein's latest feature length documentary "Finding the Gold Within". The film was world premiered at the Mill Valley Film Festival on October 3 and 4, 2014. To learn more about the film and the filmmaker, and to see the trailer, go to Goldthefilm. Excerpts courtesy of Karina Epperlein.