In the real world, light refracts and reflects when it hits the boundary between two dielectric (i.e. transparent) media which have a different index of refraction (IOR). For example, in the following glass of liquid, light refracts at the various boundaries between the air, glass, liquid and ice.
Creating such a scene in a renderer is typically accomplished by modelling the glass and water as closed meshes (filled with constant IOR material) which are either flush, intersect, or have an air-gap. Or in some renderers, one has to explicitly model each interface where the IOR changes, flagged with the IOR on each side. Manually breaking the scene up like this into separate interfaces where the IOR jumps is very inconvenient. Also, modelling geometry as flush or air-gapped produces various artefacts.
So the approach taken in Arnold (and many other renderers) is to use nested dielectrics, which means the scene is modelled by making the dielectric objects overlap, and in the areas of space where they overlap, the material is resolved into one of the overlapping dielectrics via a priority system. This correctly tracks how the IOR changes as the rays refract leading to realistic renders.
To clarify these notions, let's use a glass of water as an example. This involves four transparent media (that is the glass, the water, the ice, and the surrounding air), which are all dielectrics defined by an index of refraction (or IOR). Of course, other non-dielectric media may exist in the scene, such as the metal table on which the glass sits.
At the boundaries between the dielectric media, the IOR in general jumps from one value to another as the ray transmits through the interface. On rendering this object the ray reflects or transmits at each dielectric boundary, and the Fresnel factor and refraction direction at each surface is determined by the ratio of the IORs on each side of the boundary.
So how do we set this up in a renderer? First, a couple of approaches which Arnold doesn't use, to compare.
Explicit IOR jumps
One approach as mentioned above is to explicitly model the various interfaces, specifying for each the IORs on the exterior and interior side (as depicted below). This would involve breaking up the object into separate meshes for each interface, or tagging faces somehow. While this is a correct approach which some renderers take, it's obviously very inconvenient for an artist to have to do this, especially if the geometry is complex and/or animated.
Air gapping or flush interfaces
Another approach is to model closed meshes with well-defined IORs and put air-gaps between them so they don't touch or overlap, as below. This is perhaps slightly more convenient than breaking up the model by interface but unfortunately isn't physically correct as it generates inter-reflections in the air gaps. Another bad approach is to model the surfaces as exactly flush, which introduces numerical imprecision issues.
The approach that Arnold takes, which is also used by many other renderers, is based on the 2002 paper "Simple Nested Dielectrics in Ray Traced Images", by Schmidt & Budge. In their approach (which is usually referred to as nested dielectrics), dielectrics are modelled as closed surfaces which are allowed to overlap, but then we must specify which of the overlapping ones exists in a given region by assigning priorities. That is we assign each dielectric medium an integer priority, and then in an overlapping area, the highest priority medium is the only one which is assumed to exist, as depicted in the diagram below. (Note that air doesn't have a priority since we can just treat it as the absence of a medium). Light bounces through the scene reflecting and refracting correctly at the surviving interfaces after priority resolution, and the IOR of the medium is correctly "tracked" as the ray propagates.
This is now both physically correct and relatively easy to set up. Note that this means that for example, the boundary of the water proxy mesh inside the glass is "inactive" in the sense that there is no actual IOR jump across this boundary. Schmidt & Budge refer to these boundaries as "false interfaces". These low priority false interfaces are effectively cutaway, similar to a Boolean operation. No shading occurs at these interfaces, light simply passes through them without interacting. Light reflects and transmits only at the real interfaces which are not cut away.
In Arnold, dielectric media (glass, water etc.) are created via the standard_surface shader, specified mostly under transmission, though note that the IOR of the dielectric is specified by the specular_IOR parameter. The priority is thus naturally a parameter of the standard_surface shader, i.e. dielectric_priority. As described above, when dielectric objects overlap, the higher priority objects override lower priority. The surviving highest priority media in a given region of space then specify the dielectric properties in that region (i.e. the IOR as well as the volumetric scattering properties).
The most basic effect of the priorities is illustrated by these overlapping glass spheres with an interior scattering medium:
In the case of a glass of whisky, we expand the whisky mesh to overlap the surrounding glass and give the whisky a lower priority than glass. The whisky boundary within the glass then functions merely as a "proxy" mesh which indicates the presence of the lower priority whisky. We also give the whisky a lower priority than the ice, so the ice displaces the whisky. For example with the priorities shown, this defines the correct IOR at every point in space.
Specifying the dielectric medium
As mentioned, low priority "false" interfaces are cut away and effectively light passes through them undisturbed. Consequently, any shading parameters such as roughness take effect only at the real interfaces. However, not all shading parameters are completely ignored on the false interfaces – the shader parameters which define the interior of the dielectric medium are still taken into account as rays enter the dielectric medium (as the standard_surface shader also allows specification of an internal scattering medium "embedded" in the dielectric). These are the following:
Setting these parameters on the water in the example above means that the interior of the water will be assigned these properties, even though the water interface inside the glass is false.
Parameters of the standard_surface shader which do not control the interior properties of the dielectric medium will be ignored if applied to the false interfaces of low priority objects. For example, in the glass of orange juice example if we set zero transmission_depth (which is the default), the transmission_color functions then only as a surface tint, which as it occurs on the false boundary of the juice inside the glass, is ignored in the interior "bulk" of the juice leading to an incorrect look:
Rendering realistic reflections and refractions in complicated geometry may require a high ray depth due to the large number of internal reflections which can occur.