Judder is perceived uneven or jerky video playback arising from movement of objects or edges or detail from one frame to the next. We evolved to see objects with smooth motion in the real world, so judder is simply a perceived artifact of uneven motion due to scene sampling. How each person perceives judder varies, but it is related to:
Note that modern, large, high-brightness displays will make judder issues
much more objectionable, so the trend is for judder problems to get worse
unless addressed properly.
Judder perception is reduced at higher frame rates because the
motion of objects is reduced between frames. In principle, a nearby image
sequence of fast moving, high detail, high brightness, high contrast
frames may require very high frame rates (possibly hundreds of
frames/sec) to appear smooth. Furthermore, the higher the frame rate, the
less likely that random, uncontrolled scenes sampled by a camera will
be perceived as having judder.
Motion aliasing is not an issue when you view the world around you -
the eye doesn't have a "sampling frequency" as such. However, lighting
of objects modulated by AC power frequencies (usually at 50/60Hz, or
double these frequencies for full-wave rectification) may induce a
perception of judder in direct viewing by imposing contrast sampling
that is strong enough to override the normally smooth perception of
motion. Video frame rates and monitor refresh rates impose a similar
effect to illumination flicker.
The famous wagon-wheel effect is a sampling effect due to the fact that
our vision system also tends to match a wheel spoke to its nearest
similar-looking spoke in the next frame, and not some random one. This
effectively constrains the degree of motion, thereby actually reducing
judder. However, if the nearest matching spoke is the wrong one, the
wheel's rotation doesn't match the vehicle's motion. Regular sampling
can cause smooth motion in one direction to translate to a perception
of smooth motion in the other - a similar effect to spectral aliasing.
However, motion aliasing can also occur as an instantaneous matching
effect from frame to frame - it is not strictly a frequency domain
phenomenon. Higher frame rates reduce the likelihood of motion aliasing
in a random or uncontrolled scene by reducing the scene-to-scene motion
and thereby reducing the likelihood of an uncontrolled rotation speed
causing matching to the wrong spoke.
When viewing a monitor, judder is due to the the number of new frames per
second in the source material, and the display refresh rate, and also due
to the relationship between these values. Perceived judder (but not motion
aliasing) can often be reduced down to any fundamental judder due to the
display refresh rate by using
Motion-Compensated Frame-Rate Conversion.
In low frame-rate film, directors and photographers have tight control
of what goes in the camera - so they can reduce the effects of 1, 2 and
3 above by changing the scene and its lighting, and by reducing extreme
camera motions. However, to deal with daylight action scenes, they
usually resort to adding motion blur by increasing camera shutter
angles or exposure times, thereby reducing the effect of 4.
See our
Frame Rate Options for Shooting "Future-Proof" Video
for more on this subject.
Blur has
some unfortunate effects - the eye may track a blurred object, and see
it as blurred. Also motion blur is extremely difficult to remove, once
added, so frame-rate conversion to a higher frame rate may just produce
overly blurred objects with smoother motion. This is because the eye
has a tendency to view sampled scenes as having a blur commensurate
with the exposure time, sharp edge detail, and motion per frame. The
motion per frame reduces with frame-rate conversion, but the perceived
exposure time and sharp edge detail remain constant. If the edge detail
of moving objects is too blurred for the motion and exposure time, it
just appears as blurry.
One source of perceived judder is based on the rendering rate. For source material sampled at a higher frame-rate (but still below the display refresh rate), time per exposed frame decreases, and smoothness improves. For example, 60Hz rendering always looks better than 24Hz film rate on any synchronized display device (i.e. where refresh rate is an integer multiple of the render rate). Some people like the artistic "film look" of 24 frames per second, but for most of us, a frame rate of at least 50Hz is recommended. Not much is gained beyond 60Hz except in very fast motion situations, and where the image takes a large portion of the field of view.
This second primary cause of judder is due to switching between sets of
repeated frames in order to achieve an average rate between them for the
chosen display refresh rate.
The two adjacent repeat lengths are given by
Repeat_Lo = Floor(<refresh_Hz> / <render_frames/sec>)
where Floor(x) is the largest integer less than or equal to
x, and
Repeat_Hi = Ceiling(<refresh_Hz> / <render_frames/sec>)
where Ceiling(x) is the smallest integer greater than or equal to
x.
The effect of presenting "Repeat_Hi" frames in succession may have
dramatically worse perceived judder than presenting "Repeat_Lo" frames,
but it also depends on
the extent of sharp moving edges in the sequence, and the subtended
angle of viewing.
For example, 24 frames/second on a 60Hz refresh monitor is an average
of 60/24=2.5 frames of presentation per monitor refresh, which requires
an alternating 2,3 frame-repeat presentation cadence to get that
average. The 3-frame repeat is essentially equivalent to a 3/60 sec (20Hz) = 50 millisec
frame-rate judder which can be very annoying without application of
motion blur. In contrast, the 2/60 = 33 millisecond (30Hz) frame-rate judder is usually
much less objectionable. The overall presentation judder is a weighted average of
the two frame-presentation-time judder experiences.
See also our practical judder solutions and
our proprietary superior quality motion-compensated frame rate conversion Legato.
In practise, judder is often inadvertently introduced when attempting to adapt material
from one frame-rate to another by simply dropping or duplicating frames. This practice has been
frequently used to avoid large objectionable conversion artifacts resulting from inferior frame rate conversion (FRC)
in order to ensure each frame looks perfect.
However, it introduces undesirable motion judder during normal playback,
and should be avoided if possible.
In the past, most FRC systems introduce artifacts that
can be seen, even at normal playback speeds, and they have also used frame blending
which can produce large "ghosting effects" which are extremely objectionable when viewed
frame-by-frame, and sometimes even in normal playback.
Furthermore, frame blending really does not
significantly reduce judder especially for panning scenes. The most advanced FRC uses motion estimation (ME)
to improve the quality of interpolated frames, however, not all ME technologies are created equal,
and some ME introduces edge boiling effects or other errors such as break-ups, etc. for various fast motion scenes.
Note that for extreme motion situations, the appearance of an interpolated frame may not be well defined, although the
human vision system is quite adept at judging when the motion flow seems wrong. The optimum balanced solution is
to achieve both smooth motion, while minimising conversion artifacts and generating "good looking" frames when
viewed individually.
Viarte (Legato) achieves smooth motion,
with no objectionable frames when viewed at normal speed. Viarte typically generates over 99.5% perfect-looking frames
when viewed individually, while artifacts are limited to minimal and localised regions. Most competing systems can
barely average 80% good-looking frames, often with large-area ghosting or break-up.
Dropping frames is particularly difficult to repair (e.g from material shot at 30p down to 24p)
as it not only introduces judder, but is irreversibly damaging, and cannot truly be undone as
(1) it is difficult to reliably locate where frames were actually dropped, and
(2) it is more difficult to repair using frame-interpolate due to the very large
frame-to-frame motion that may occur. As far as interpolation is concerned, frame dropping is
equivalent to halving the frame rate around each missing frame, so 30p motion looks like 15p.
Note: When adding simulated motion blur for FRC from 50p to 24p, some fine lines/details may appear less sharp than in the original 50p when viewed frame-by-frame. This same motion blurring would have occured on a real 24p camera.
Viarte frame rate conversion was used successfully for its entire footage conversion (converting from 1980x1080p29.97 to 1080p23.976) for independent director/producer Karina Epperlein's latest feature length documentary "Finding the Gold Within". The film was world premiered at the Mill Valley Film Festival on October 3 and 4, 2014. To learn more about the film and the filmmaker, and to see the trailer, go to Goldthefilm. Excerpts courtesy of Karina Epperlein.
Share:
