tekBlueprint

To take a break, I decided to revisit some old stuff. It might all be outdated now, but still. A small but interesting challenge involving implementing “fog of war” in 3D. Inside are the implementation details and a demo project in THREE.js.

Info

The source code is available in the GitHub repository. – Проект.

Introduction

A long time ago, I needed to create a global map for navigating the world between points of interest, where the unexplored part was obscured by fog. The difficulty was that the map was 3D, and fog of war needed to be implemented taking into account elevation and lighting. However, there were no proper implementation examples. Everything was either half-functional or not quite as needed. I wanted to try to put together a more or less complete implementation. I’m definitely not a professional or even an amateur, but I gave it a try anyway :)

Implementation requirements

While 2D is more or less clear – there’s a texture + animated fog mask, all in one plane. In 3D, however, volume appears, which needs to be taken into account when drawing the fog. What was needed:

1. Volume – this is three-dimensional fog with the ability to set height. With volumetric waves that slowly roll, taking into account lighting and casting shadows.
2. The fog’s boundaries need to be smooth, not sharp, but tapering smoothly to the ground. Additionally, a multi-layered effect is needed, with the lower part of the fog being more transparent and able to be a different color.
3. The explored area should not be simply square, but rather have an arbitrary shape and be located anywhere on the map.
4. Tall objects should be hidden, even if they are above the fog level (seems logical).
5. If possible, the implementation should not be too demanding on GPU performance.

Implementation methods

Among those I know of:

1. Using Screen Space. The effect is applied to the screen during post-processing. This is convenient because it doesn’t require any additional hiding and takes care of object overlap on the map.
2. Using Volumetric Fog with Volumetric Textures.
3. Using fake volume through 3D meshes with transparent textures and vertex deformation to simulate fog movement.

The first option is probably the simplest and most common implementation method. It doesn’t require a lot of computing resources and can hide anything. It does have its drawbacks, such as limited ability to interact with scene objects, since they aren’t “physical objects,” after all. There are also some rendering artifacts.

The second option is relatively GPU-intensive, but it allows for quick and easy volumetric fog creation. It also looks quite good and realistic. It does have some issues. For example, with shadow rendering, there are flickering artifacts, and when the camera is rotated, fog can be seen reappearing in different places. There are ways to combat this, but not all solutions are implemented. Examples of such solutions can easily be found in engine marketplaces and online. An example is Expand Your World With Volumetric Effects

The last option is more difficult to implement, as it requires more work to simulate fog, but it eliminates artifacts and reduces GPU load (if done correctly). Implementing it with sprites is unsuitable, as they are essentially 2D textures with different approaches to positioning in 3D space. Therefore, we will use the second approach, using meshes. How it is implemented:

• Create a plane the same size as the map. It is necessary to specify a sufficient number of segments (vertices) for it, as they will shift in height. If there are not enough, the result will look too angular.
• Create a mask, a regular texture (offscreen), which will be used as a “memory” for open areas on the map. It is used in fragment shaders to determine the visibility of mesh vertices.
• In the mask, the explored area is added with a gradient. That is, the boundaries should smoothly change from one color (closed) (black) to another (open) (white). - To create a smooth transition to the ground surface, the height is calculated in the vertex shader using a mask (gradient – height).
• The border transparency in the fragment shader is calculated based on the current height data from the vertex shader.
• An additional texture or two with noise are applied to the mesh for greater realism. Here you can get creative with multiple layers, movement, transparency, and so on.
• The visibility of objects is determined by the mask when it changes.

Example – UE4 VFX For Games: Brute Force Volumetric Gases - Ground Fog & Vapor

Implementation of the second option

The implementation example will be in WebGL via Three.js, as this makes it quicker and easier to demonstrate as a demo in a browser. I’ll skip the description of parameter binding for the generated geometry and its UI, as it’s not essential for understanding the process.

Creating a mesh and textures

A regular plane the size of the scene and the required number of segments is used as a base.

const geometry = new THREE.PlaneGeometry(this.worldWidth, this.worldHeight, this.segments, this.segments);

Segment calculation can be done simply, depending on the scene size. This is necessary to preserve the required detail when deforming the mesh.

// Calculate mesh segments based on world size - more segments for larger worlds
// Using 256 as base scale factor for finer geometry and smoother edges
this.segments = Math.max(256, (Math.max(this.worldWidth, this.worldHeight) / 192) * 256);

Next, we need to create the material that will be used for the mesh. The uniforms parameter is used to pass data to the shader. In this case, this will be data about fog physics, textures, masks, etc. It’s important to remember to specify transparent: true and depthWrite: true, as we will use transparency and depth to correctly display the fog. The side: THREE.DoubleSide parameter is required so that the shader correctly displays fog from all sides.

// Create shader material using the fog vertex and fragment shaders
this.material = new THREE.ShaderMaterial({
    vertexShader: FOG_VERTEX_SHADER,
    fragmentShader: FOG_FRAGMENT_SHADER,
    uniforms: this.uniforms,
    side: THREE.DoubleSide,
    transparent: true,
    depthWrite: true,
});

export interface ThreeUniforms {
    uTime: THREE.IUniform<number>;
    uSpeed: THREE.IUniform<number>;
    uScale: THREE.IUniform<number>;
    uAmplitude: THREE.IUniform<number>;
    uColor: THREE.IUniform<THREE.Color>;
    uEmission: THREE.IUniform<number>;
    uRadius: THREE.IUniform<number>;
    uWallWidth: THREE.IUniform<number>;
    uFogHeight: THREE.IUniform<number>;
    uHistoryTexture: THREE.IUniform<THREE.CanvasTexture | null>;
    uFogTexture: THREE.IUniform<THREE.CanvasTexture | null>;
    uTextureNeedsUpdate: THREE.IUniform<boolean>;
}

We combine everything into a mesh.

this.mesh = new THREE.Mesh(geometry, this.material);

Additionally, we create a uFogTexture texture (noise-texture.ts) to create a noise effect on the fog surface.

// Generate a noise texture for the initial fog appearance
const noiseCanvas = new NoiseTexture().generate(512, 512);
// Link the noise texture for visual variation in the fog appearance
this.uniforms.uFogTexture.value = new THREE.CanvasTexture(noiseCanvas);

Mask for the explored area

The mask will be drawn on a regular canvas, which is then used as a texture in the shader.

this._texture = new THREE.CanvasTexture(this.canvas);
this._texture.minFilter = THREE.LinearFilter;
this._texture.magFilter = THREE.LinearFilter;

In this example, rendering is performed on mouse hover, using a brush-like effect with a circle size and gradient (fog-mask-texture.ts, reveal method), but other methods are also possible. For example, only when the player moves, revealing the map in segments as the player moves, and so on. It depends on the game design.

We bind to the shader parameters:

// Link the mask texture to the shader's history texture for state persistence
this.uniforms.uHistoryTexture.value = this.mask.texture;

Shaders

I won’t provide the shaders in full, as the noise generation part isn’t particularly interesting and can be found in the source code (fog-shader.ts).

Vertex shader (FOG_VERTEX_SHADER). This is where the main geometry deformation logic is executed to simulate fog waves and a smooth descent to the ground along the mask.

// Uniform inputs from JavaScript side
    uniform float uTime;               // Animation time
    uniform float uSpeed;              // Speed of fog movement
    uniform float uScale;              // Scale of the noise pattern
    uniform float uAmplitude;          // Height amplitude of fog waves
    uniform float uRadius;             // Radius of fog blob
    uniform float uFogHeight;          // Base height of fog bottom
    uniform sampler2D uHistoryTexture; // History texture for persistence/transition

    // Varying outputs passed to fragment shader
    varying vec2 vUv;                  // Texture coordinates
    varying float vElevation;          // Calculated elevation from noise
    varying vec3 vWorldPosition;       // World-space position
    varying vec3 vNormal;              // Surface normal for lighting
    varying float vDropFactor;         // Current blend factor (history value)

	...

    void main() {
        // Pass texture coordinates to fragment shader
        vUv = uv;

        // Read history texture for persistence effect
        float history = texture2D(uHistoryTexture, vUv).r;
        vDropFactor = history;

        // Calculate noise at current position with time-based movement
        vec3 noisePos = vec3(position.x * uScale, position.y * uScale, uTime * uSpeed);
        float noiseVal = fbm(noisePos);

        // Add high-frequency detail noise for texture
        float detail = snoise(vec3(position.x * uScale * 3.0, position.y * uScale * 3.0, uTime * uSpeed * 1.5));
        vElevation = noiseVal + (detail * 0.2);

        // Calculate new position with elevation based on noise
        vec3 newPos = position;
        newPos.z += vElevation * uAmplitude;

        // Define fog shape boundaries
        float topZ = vElevation * uAmplitude;
        float bottomZ = -uFogHeight + vElevation + 0.05;

        // Interpolate between top and bottom based on history/drop factor
        newPos.z = mix(topZ, bottomZ, vDropFactor);

        // Calculate surface normals from noise gradient with larger offset for smoother edges
        float offset = 0.5;
        float hR = fbm(vec3((position.x + offset) * uScale, position.y * uScale, uTime * uSpeed));
        float hL = fbm(vec3((position.x - offset) * uScale, position.y * uScale, uTime * uSpeed));
        float hU = fbm(vec3(position.x * uScale, (position.y + offset) * uScale, uTime * uSpeed));
        float hD = fbm(vec3(position.x * uScale, (position.y - offset) * uScale, uTime * uSpeed));

        vec3 tangent = normalize(vec3(offset * 2.0, 0.0, (hR - hL) * uAmplitude));
        vec3 bitangent = normalize(vec3(0.0, offset * 2.0, (hU - hD) * uAmplitude));
        vec3 noiseNormal = normalize(cross(tangent, bitangent));

        // Calculate wall-style normals from history texture gradient with smoother steps
        vec2 offsetUV = vec2(1.0/256.0, 0.0);
        float hX = texture2D(uHistoryTexture, vUv + offsetUV.xy).r - texture2D(uHistoryTexture, vUv - offsetUV.xy).r;
        float hY = texture2D(uHistoryTexture, vUv + offsetUV.yx).r - texture2D(uHistoryTexture, vUv - offsetUV.yx).r;

        vec3 wallNormal = normalize(vec3(-hX, -hY, 0.3));

        // Blend between procedural noise normals and wall normals based on drop factor
        vNormal = normalize(mix(noiseNormal, wallNormal, vDropFactor));

        // Transform to world space
        vec4 modelPosition = modelMatrix * vec4(newPos, 1.0);
        vWorldPosition = modelPosition.xyz;

        // Final clip space transformation
        gl_Position = projectionMatrix * viewMatrix * modelPosition;
    }

The most important parameter here is the uHistoryTexture parameter, which contains a texture with explored and unexplored areas of the map (more on that below). It uses the texture’s red channel, which contains values from 0 to 1.0 – this is the height of the current geometry vertex to the ground plane.

Next comes the fragment shader (FOG_FRAGMENT_SHADER). Here, we calculate the color, glow, and transparency based on the vDropFactor, which we got from the vertex shader. The higher the vDropFactor, the closer the vertex is to the ground, and the more transparent it should be, up to complete invisibility if the value is greater than 0.995.

    // Uniform inputs
    uniform vec3 uColor;               // Base fog color
    uniform float uEmission;           // Emissive intensity
    uniform sampler2D uFogTexture;     // Background texture/gradient

    // Varying inputs from vertex shader
    varying vec2 vUv;
    varying float vElevation;
    varying vec3 vWorldPosition;
    varying vec3 vNormal;
    varying float vDropFactor;

    void main() {
        // Discard fully dropped/merged pixels (no rendering)
        if(vDropFactor > 0.995) {
            discard;
        }

        // Calculate lighting
        vec3 normal = normalize(vNormal);
        vec3 lightDir = normalize(vec3(1.0, 1.0, 1.0));
        float diff = max(dot(normal, lightDir), 0.0);
        float ambient = 0.3;  // Base ambient light

        // Get fog background color from texture
        vec3 fogColor = texture2D(uFogTexture, vUv).rgb;

        // Blend between fog background and base color based on drop factor
        vec3 finalColor = mix(fogColor, uColor, vDropFactor);

        // Apply lighting to the blended color
        vec3 litColor = finalColor * (diff + ambient);

        // Add emission (glow effect)
        litColor += finalColor * uEmission;

        // Calculate height-based transparency: closer to ground more transparent
        float transparency = smoothstep(0.0, 0.6, 1.0 - vDropFactor) ;

        // Output final color with transparency based on height
        gl_FragColor = vec4(litColor,  transparency);
    }

Hiding invisible objects

The map may contain tall objects that shouldn’t be visible. Raising the fog level too high can create an unsightly appearance (like a steep wall). Alternatively, use an existing mask of explored areas to check if an object is visible.

In my case, I use a mask check on mouse movement (fog-scene.ts – updateObjectsVisibility). I simply look at the pixel value in the mask, and if its value is greater than 0.6, meaning it’s explored and already at the edge of visibility, the object is displayed.

Performance optimization and improvements

In this example, some parts are not quite optimally implemented to simplify the code. However, even in this form, it still delivers 60 FPS on the integrated version, which is already quite old. Here are some quick thoughts:

1. The most obvious is to replace noise generation in the vertex shader with a pre-generated texture. This is usually done, as the goal is to minimize the number of executed instructions.
2. Next, you can use the good old LOD. As the camera zooms out, replace the fog plane with a simpler one with fewer segments (vertices). This will reduce the number of shader calls.
3. If this isn’t enough, you can split the plane into independent mesh parts. Then, you can use culling of the invisible parts. Only those within the camera’s field of view will be rendered. The shader will need to use world coordinates for texture blending (or noise generation), rather than local coordinates, to avoid seams at plane boundaries.
4. Use multithreaded processing to asynchronously update fog masks. This will reduce the load on the main rendering thread.
5. Use different shaders (with refinement) for different levels of detail. You can also use different textures.

There are probably more, but I don’t remember any more.

Possible improvements include:

1. In the example, the surface is flat, but you could create surface-based fog so it rises smoothly on high ground or sinks in low ground. A pre-computed heightmap could be added, for example, to the blue channel of the mask.
2. Create additional effects for smoke or fog plumes above the surface, using something like Niagara or other effects.
3. Animate the smooth fading of the fog during exploration. Again, through a separate channel of the mask.

Demo

Conclusion

Obviously, the final version still requires some refinement and optimization, but this is more about the implementation ideas. GLSL itself is portable (well, almost), so it’s relatively easy to replicate the example in the desired engine and finish it. However, with UE, you’ll have to port it to materials.