VFX artists often need to composite the CG objects over a live action film plate. To convince the audience that the CG objects are real and belong to the scene, holdouts are used to collect the CG objects' shadow and reflection that match very closely to the shadow and reflection in the live action plate.
Holdout objects are used in place of real world object to catch light interactions from 3D objects. In RenderMan, holdout objects are specified by enabling the trace holdout attribute.
Integrators and AOVs
Holdouts are an illumination feature implemented at the integrator level. They are able to catch a variety of integrator-specific illumination effects such as diffuse and specular reflections, transmissions, and caustics. Available integrator that support holdouts are: PxrPathTracer, PxrVCM, and PxrUPBP.
LPE workflow remain unchanged for holdout geometry.
Because holdouts workflow involves multiple AOVs, rendering with adapt all enabled can reduce noise.
By default, light sources do not illuminate holdout objects directly. This prevents double contribution between 3d lights used to illuminate non-holdout geometry and the illumination already captured on the plate image.
To create a holdout light (i.e. a light source that directly illuminates holdout objects), create a light using "__illumholdout" as the light group suffix. This tells integrators to consider this light as a holdout light source when illuminating holdout geometry.
Alpha, Shadows, and Compositing
Holdout contributions can be composited over a film plate using an alpha and shadow pass as the masking term. The shadow pass can be output using the PxrShadowFilter plug-in. The occluded and unoccluded input AOVs should be rendered using the holdouts LPE prefix so that they only contain the shadowing term for holdout geometry.
Compositing can be interactively previewed in RenderMan by using PxrImagePlaneFilter SampleFilter plug-in. This will composite the film plate with the beauty render, using the alpha channel and holdout shadow AOV to mask the plate.