Parallax Occlusion Mapping Deferred Decals in UE4 (Technical Solution)

One of the core techniques in my project, was to implement a parallax occlusion mapping (POM)-based method of rendering deferred decals so that they could populate the main environment and thus adding detail at a very low cost. This is a totally non-destructive workflow that can be seen in the upcoming space simulator 'Star Citizen'.

The main advantage of this method of adding detail lies in the modular and non-destructive nature of the process. It works by taking a series of small decals (with low resolution maps generated from them) and duplicating and applying them to the surface of the main mesh. The mesh decal shader will only affect the deferred buffers such as normal, base colour and roughness, allowing for non-destructive changes to be made to the underlying material.

By doing this, you can add screws, panels, panelling lines and other details that can be changed at any given time without baking. To change the appearance of the detail, simply change the texture of the decal without any need for baking. This is another advantage to the technique, as you would get really fine crisp details whereas a purely baked method is limited to the resolution of the base texture.

To make this work, I had to extract the world-space vectors of the deferred decal actor, feed them into the POM material function and convert them to local space; all whilst being completely dynamic. UE4 by default derives the Z-axis needed to calculate the depth from the face normal of the geometry; however for deferred decals it makes the world Z-axis constant, which results in rendering issues when rotating the decal actor.

I first created a blueprint that generates a dynamic material instance of the decal material in a construction script. I then extracted the blueprint actor's rotation and fed the respective XYZ values into parameters in the POM function. I then passed these through an "Inverse Transform Matrix" function, which transforms the vectors from world-space into local space. In the decal material I provided constants for the XYZ vectors, which tells UE4 the direction in which the vectors are facing.

The blueprint:

The modified POM function:

The function in a decal material:

A problem with this method is that it cannot be applied to mesh decals, due to the way UE4 handles rendering of mesh decals as opposed to projected deferred decal actors, meaning it wouldn't be used for screws or bolts or any other small details. Another downside is that it doesn't handle pixel depth offset which is used for conforming decals to other POM surfaces. Nevertheless, this solution is perfect for applying hard surface details at a negligible cost and with a lot of flexibility.

Here are some examples of the decals working in engine (affecting normals, roughness, base colour and metalness):