Visual Effects Quality Control Practices
Quote
"One unfortunate reality in film production is that the typical artist display is not as high fidelity as the eventual theatrical viewing environment. Most often the theatrical audience will see more color detail than the original artist."
Jeremy Selan, Sony Pictures Imageworks, Cinematic Color VES, 2012
The first rule of VFX is do no harm: the work must not reduce the plate's technical quality. Define and test the imaging pipeline before production, then document the expected result for every vendor and artist.
The facility technical director normally owns this work. On smaller productions, assign it to the VFX supervisor or lead artist. Escalate pipeline problems as soon as they appear.
Image Processing Practices
Scaling Algorithms
Scaling remaps pixels. Filter choice controls interpolation, sharpness, aliasing, and ringing.
Impulse scaling, also called nearest-neighbor, is prone to producing noticeable aliasing and distortion, but is computationally inexpensive.
Better, smoother scaling algorithms result in softer edges with fewer scaling artifacts, but are more computationally intensive.
Many scaling algorithms introduce a degree of sharpening to produce apparent resolution gain when positively scaling an image. These are negative-lobed filters, including Rifman and Lanczos. The filter kernel has regions of negative weight. Those negative weights are what produce ringing, the halo of overshoot on the bright side of a high-contrast edge and undershoot on the dark side.
Negative filter lobes cause ringing. Negative pixels are the resulting undershoot. Clamping removes the undershoot, not the cause.
Scaling in scene-linear is physically correct. Resampling averages light, and light is linear. Averaging code values in a non-linear encoding is not the same operation and does not conserve energy. The difficulty is that linear encoding is also what makes the ringing so visible. A specular highlight sitting at 60.0 next to a shadow at 0.02 gives the filter an enormous local contrast to overshoot on, so the halo that would be mild in a display encoding becomes severe, and the undershoot swings well below zero.
There are several ways to manage this, and the right one depends on the shot:
- Use a filter without negative lobes where the softness is acceptable: no negative weights, no ringing.
- Scale as close to 1:1 as possible, and resample as few times as possible. Every resample is another opportunity. A chain of transforms that concatenates is one resample instead of four.
- Round-trip through log around the resize. This bounds the overshoot by compressing the range the filter sees. It is a genuine mitigation, but not a free one: filtering log values no longer averages light, so it trades physical correctness for artifact suppression. It suits alpha-free images at close to 1:1 best, and is least appropriate where energy accuracy matters.
Compositing applications do not manage this automatically. Nuke filters in the current working
space. A log round trip must be explicit. Its Transform and Tracker clamp control discards negative
intermediate values but leaves highlight overshoot intact.1
When a scene-linear resize produces severe ringing, test a log round trip around the resize. This is common for large camera-original-to-VFX-pull resizes with strong highlights. It trades energy accuracy for artifact control, so use it only when the ringing is more damaging.
The same principles apply to titles and graphics compositing in a digital intermediate.
Concatenated Image Transforms
Every resample can soften an image. Concatenate successive transforms so the application performs one net resample. Color corrections, filters, distortions, merges, and other non-concatenating nodes between transforms can break the chain.
In the example below, Transform1 is scaling by a factor of 0.10 and Transform2 by 10.0: a
net-unity scaling transform, since the two factors multiply to 1.0. In the example on the left the
transforms are concatenated before the ColorCorrect node, resulting in a color corrected image
of the original, identity-transformed image. On the right, the ColorCorrect node bisects the
transforms and breaks the concatenation. The result is an image with substantial loss of
sharpness.
Colour correction is only the most common way to break a chain. Any node that cannot concatenate will do it: a filter, a blur, a distortion or lens-warp node, a merge against an element that was itself transformed. If sharpness is being lost across a chain that ought to be an identity, look for whatever sits between the transforms rather than assuming the transforms are at fault.
Transfer Functions
It is common for compositing to be performed in a working space other than the original plate or delivery-specified encoding(s). Strict color management throughout the compositing process is necessary to ensure that the final delivery is in the desired color space and that no loss of dynamic range has occurred. Compositing is often best performed in a scene-linear working space where math operations function predictably and linearly, rather than in a logarithmic or gamma-referred working environment.
Compositing packages like Foundry's Nuke are designed around compositing in a linear working space by applying floating-point transfer functions to pre-convert material from camera-native logarithmic encoding to scene-linear and then back to a chosen destination encoding on render. This all works because the transfer functions are mathematically reversible, are not "s-shaped" display-referred transforms, and are performed in high precision 32-bit float.
Other compositing packages2 coming from different pedigrees have different philosophies about desired working spaces and color management practices. Conducting end-to-end workflow tests in pre-production is paramount to identifying and resolving potential issues and deciding on a workflow that yields the best results.
Quality Control Practices
QC every shot before delivery. Check the complete frame for dropouts, matte edges, color shifts, formatting, naming, and frame-number errors. Compare against the plate and prior version, then review the sequence for continuity. Automated checks catch missing or malformed frames. Visual inspection catches compositing errors.
Many composited images look great when viewed in their target viewing space on the display they were composited on. However, those shots will ultimately undergo significant color grading and be viewed on consumer displays under less than ideal conditions.3 Under those conditions, masks with unfeathered edges, differences in noise and grain patterns, and changes in shadow density and black points become significantly more apparent.
Gamma Checking
Raise and lower the viewport gamma aggressively to reveal matte lines and mismatches in noise, grain, or black level.
Exposure Checking
Additionally, linear increases in exposure can provide artists with a sense of the real dynamic range of their image. Do artificial (composited or CGI) highlights clip or read at the same level as comparable highlights in the original plate? Do artificial shadows match the black point (color and value) of comparable shadows in the plate? How do these values compare within the shot and to other shots nearby?
Difference Checking
Performing a high-gain mathematical difference operation between a visual effects render and an original plate can reveal a variety of compositing issues that may not be visible to the naked eye, including matte lines, noise/grain patterns, tracking issues, and highlight/shadow compression or clamping.
-
Foundry, Nuke documentation: the Transform node's controls are translate/rotate/scale/ skew/center/invert/filter/clamp/black-outside/motionblur, with no colour-space knob. And "when using filters that employ sharpening, such as Rifman and Lanczos, you may see a haloing effect. Some nodes, such as Transform and Tracker, include a clamp control to correct this problem." ↩
-
Adobe After Effects is one common example. ↩
-
Sports mode, anyone? ↩