Diffuse Roughness support #16253

MiiBond · 2025-03-05T22:03:04Z

This PR replaces #16183

This PR implements the base_diffuse_roughness parameter from the OpenPBR specification.

The diffuse roughness is implemented for analytic lights, realtime-filtered IBL, prefiltered IBL and spherical harmonics.
I've added a flag to a material to choose between Lambert, Burley and the new Energy Conserving Oren-Nayar (EON) model used by OpenPBR. The default is OpenPBR's EON model.

The previous behaviour was to use Burley diffuse for analytical lights and Lambert for IBL. Also, previously, specular roughness was applied to diffuse roughness for analytical lights while IBL didn't use it at all (because it was simply Lambertian). So, the new default slightly changes existing projects that used analytical lights but I question how noticeable that will be.

Analytical Light:
https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#MXACV7#3

Realtime IBL:
https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#MXACV7#5

The diffuse roughness models are heavily dependent on the light direction and view direction and are therefore difficult to handle with a prefiltered IBL. I came up with two methods for approximating roughness with prefiltered IBL's. The first, if we prefiltered using CDF, we generate a dominant light direction to use in the BRDF calculations. It works reasonably well.

Prefiltered IBL with CDF:
https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#MXACV7#9

The second approach, if you don't use CDF, is by approximating roughness by bending the surface normal towards the camera to add some of the retro-reflective behaviour that you get with EON. Because of this, you'll notice that Burley and EON are identical and the shadow terminator appears to move as diffuse roughness increases. In practice, however, this example uses an extreme IBL with a bright sunlight. With other IBL's, the effect tends to be more convincing.

Prefiltered IBL without CDF
https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#MXACV7#10

The default IBL lighting in Sandbox uses spherical harmonics so we need to approximate diffuse roughness with this as well. I'm using the same bent normal technique as with prefiltered IBL without CDF.

Spherical Harmonics IBL
https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#MXACV7#11

bjsplat · 2025-03-05T22:04:28Z

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

bjsplat · 2025-03-05T22:15:54Z

Snapshot stored with reference name:
refs/pull/16253/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/16253/merge
https://gui.babylonjs.com/?snapshot=refs/pull/16253/merge
https://nme.babylonjs.com/?snapshot=refs/pull/16253/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#BCU1XR#0

MiiBond · 2025-03-05T22:24:13Z

packages/dev/core/src/Materials/PBR/pbrBaseMaterial.ts

@@ -2321,6 +2374,7 @@ export abstract class PBRBaseMaterial extends PushMaterial {
                }

                ubo.updateFloat("baseWeight", this._baseWeight);
+                ubo.updateFloat("baseDiffuseRoughness", this._baseDiffuseRoughness ?? this._roughness ?? 1);


I put this logic here to fallback on the old behaviour if baseDiffuseRoughness wasn't used. However, this was inconsistent between IBL and analytic lights so I suggest we just remove this old behaviour. What do you think?

I've removed the fallback to the old behaviour now. How do you guys feel about this?
The roughness previously only had an effect on diffuse for analytical lights.

Would by default the rendering be impacted with the new code ? We had a few remarks/issues regarding the latest fixes in the PBR with ppl willing to use back the old behavior. If the impact is tiny it is ok but if it is more, we need at least a fallback flag

Yes, there is a change to the default behaviour. It only affects rough surfaces that use analytical lights (the default behaviour of IBL is unaffected). Previously, the surface roughness was used as input to the Burley diffuse model but this was only ever used for analytical lights (i.e. direct lighting). IBL didn't use Burley and so diffuse irradiance from IBL was unaffected by roughness. This inconsistency is one of the reasons I think we should change the default behaviour.

Here's an example of the most extreme case where the change would be most noticeable. The sphere on the right has roughness 1.0.
Before the change:

After the change:

Most cases won't be nearly this noticeable, of course. Diffuse roughness (for Burley or EON) is most noticeable when the view is aligned with the light direction, like in this example.

Another point to note is that, if we change the default diffuse roughness model to EON, rather than Burley, the default look of these cases is going to change anyway.

I'm seeing that in a lot of these visualization test failures.

bjsplat · 2025-03-05T23:16:24Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-05T23:16:27Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-06T02:23:23Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-06T02:23:25Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-06T22:37:17Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-06T22:37:18Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-07T05:45:56Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-07T05:45:59Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-07T07:18:52Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-07T07:18:52Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

virtualzavie

This is only a partial review with a couple of comments.

virtualzavie · 2025-03-07T14:32:01Z

packages/tools/tests/test/visualization/ReferenceImages/openpbr-base-diffuse-roughness.png

Are you sure everything is correct with the bottom row? It seems to have way too much energy.
Burley is not energy conserving, but I don't think it should be this bright.

Yeah, I thought the same thing but it seems as though this retro-reflectivity is expected when viewing directly along the direction of the light. When viewing from the side, it gets a lot darker.

However, it is still slightly brighter than it should be and this is due to me post-multiplying the albedo rather than passing the albedo into the EON computation. For analytical lights and realtime filtering of the IBL, we can do EON correctly but I also want this to work for prefiltered IBL's so I want a solution where we can post-multiply the albedo. I'm working on improving this...

virtualzavie · 2025-03-07T14:32:24Z

...ols/tests/test/visualization/ReferenceImages/openpbr-base-diffuse-roughness-realtime-ibl.png

Why is this render black? Does it not include your IBL implementation yet?

Because I didn't check the output before I pushed my changes :)

Thanks for pointing this out. It's fixed now.

virtualzavie · 2025-03-07T14:33:06Z

packages/tools/tests/test/visualization/config.json

+            "title": "OpenPBR Base Diffuse Roughness Realtime IBL",
+            "playgroundId": "#MXACV7#5",
+            "referenceImage": "openpbr_base_diffuse_roughness_realtime_ibl.png",
+            "excludedEngines": ["webgl1"]


Why is webgl1 excluded?

Good point. The CDF generator doesn't work in WebGL 1.0 but that shouldn't prevent this test from running. I've tested in WebGL 1.0 now and fixed a whole bunch of shader issues.
I think I can make the CDF generator work in WebGL 1.0 as well (as part of a future PR). I just have to get rid of all the texelFetch calls since that function doesn't exist in WebGL 1. Originally, the CDFGenerator was only for IBL Shadows which can't work in WebGL 1 because it doesn't support 3D textures.

virtualzavie

Continued review.

virtualzavie · 2025-03-07T14:38:20Z

packages/dev/core/src/Shaders/ShadersInclude/hdrFilteringFunctions.fx

+                        vec3 diffuseRoughnessTerm = vec3(1.0);
+                        #if BASE_DIFFUSE_ROUGHNESS_MODEL == 1
+                            diffuseRoughnessTerm = vec3(diffuseBRDF_Burley(NoL, NoV, VoH, diffuseRoughness) * PI);
+                        #elif BASE_DIFFUSE_ROUGHNESS_MODEL == 2
+                            diffuseRoughnessTerm = diffuseBRDF_EON(vec3(1.0), diffuseRoughness, NoL, NoV) * PI;
+                        #endif


I'm a bit surprised there's a ×π factor for Burley and EON but not for Lambert. The result looks correct for Lambert though, so when does it happen? Is there a discrepency regarding when it's factored in?

Both diffuseBRDF_Burley and diffuseBRDF_EON contain an explicit 1.0 / PI.

It sounds like the engine assumes the ×π factor cancels out for Lambert, but keeps it explicit in the other two...

virtualzavie · 2025-03-07T14:42:36Z

packages/dev/core/src/Shaders/ShadersInclude/pbrBlockReflection.fx

@@ -264,6 +266,7 @@
            #else
                vec3 irradianceVector = vec3(reflectionMatrix * vec4(normalW, 0)).xyz;
            #endif
+            vec3 irradianceView = vec3(reflectionMatrix * vec4(viewDirectionW, 0)).xyz;


Wouldn't it be preferable to do:

vec3 irradianceView = mat3(reflectionMatrix) * viewDirectionW;

virtualzavie · 2025-03-07T14:44:30Z

packages/dev/core/src/Shaders/ShadersInclude/pbrDirectLightingFunctions.fx

+    float diffuseTerm = 1.0 / PI;
+    #if BASE_DIFFUSE_ROUGHNESS_MODEL == 1
+        diffuseTerm = diffuseBRDF_Burley(info.NdotL, info.NdotV, info.VdotH, info.diffuseRoughness);
+    #elif BASE_DIFFUSE_ROUGHNESS_MODEL == 2
+        diffuseTerm = diffuseBRDF_EON(vec3(1.0), info.diffuseRoughness, info.NdotL, info.NdotV).x;
+    #endif


Why is the scale ×1/π ; ×1 ; ×1 here, whereas it was ×1 ; ×π ; ×π in the other implementation?

This one is for direct lighting from an analytical light. The other was for the IBL filtering and I think the PI is already accounted for in the PDF.

I see. It makes sense, although that's not very consistent.

virtualzavie · 2025-03-07T14:47:12Z

packages/dev/core/src/Shaders/ShadersInclude/pbrDirectLightingFunctions.fx

+        float diffuseTerm = 1.0 / PI;
+        #if BASE_DIFFUSE_ROUGHNESS_MODEL == 1
+            diffuseTerm = diffuseBRDF_Burley(NdotL, info.NdotV, info.VdotH, info.diffuseRoughness);
+        #elif BASE_DIFFUSE_ROUGHNESS_MODEL == 2
+            diffuseTerm = diffuseBRDF_EON(vec3(1.0), info.diffuseRoughness, NdotL, info.NdotV).x;
+        #endif


Same remark here.

virtualzavie · 2025-03-07T14:47:36Z

packages/dev/core/src/ShadersWGSL/ShadersInclude/pbrDirectLightingFunctions.fx

+    var diffuseTerm: f32 = 1.0 / PI;
+    #if BASE_DIFFUSE_ROUGHNESS_MODEL == 1
+        diffuseTerm = diffuseBRDF_Burley(info.NdotL, info.NdotV, info.VdotH, info.diffuseRoughness);
+    #elif BASE_DIFFUSE_ROUGHNESS_MODEL == 2
+        diffuseTerm = diffuseBRDF_EON(vec3(1.0), info.diffuseRoughness, info.NdotL, info.NdotV).x;
+    #endif


Same remark here.

virtualzavie · 2025-03-07T14:47:47Z

packages/dev/core/src/ShadersWGSL/ShadersInclude/pbrDirectLightingFunctions.fx

+        var diffuseTerm: f32 = 1.0 / PI;
+        #if BASE_DIFFUSE_ROUGHNESS_MODEL == 1
+            diffuseTerm = diffuseBRDF_Burley(NdotL, info.NdotV, info.VdotH, info.diffuseRoughness);
+        #elif BASE_DIFFUSE_ROUGHNESS_MODEL == 2
+            diffuseTerm = diffuseBRDF_EON(vec3(1.0), info.diffuseRoughness, NdotL, info.NdotV).x;
+        #endif


bjsplat · 2025-03-10T18:31:47Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-10T18:59:43Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-10T21:35:08Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-10T21:47:31Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-12T00:28:31Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-12T00:42:06Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-12T01:45:08Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-12T02:26:03Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-12T19:14:55Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-12T19:26:27Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-17T22:00:09Z

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

bjsplat · 2025-03-17T22:16:28Z

Snapshot stored with reference name:
refs/pull/16253/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/16253/merge
https://gui.babylonjs.com/?snapshot=refs/pull/16253/merge
https://nme.babylonjs.com/?snapshot=refs/pull/16253/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/16253/merge#BCU1XR#0

bjsplat · 2025-03-17T22:30:01Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-17T22:41:28Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-18T18:03:06Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-18T18:12:22Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-18T18:56:52Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-18T19:18:01Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-18T19:43:51Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-18T20:33:19Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-19T16:35:49Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-19T17:22:23Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

bjsplat · 2025-03-19T18:20:51Z

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgpuplaywright/index.html

bjsplat · 2025-03-19T18:23:51Z

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/16253/merge/testResults/webgl2playwright/index.html

MiiBond marked this pull request as draft March 5, 2025 22:03

MiiBond commented Mar 5, 2025

View reviewed changes

MiiBond force-pushed the openpbr/diffuse_roughness branch from 3580a82 to 55f34a0 Compare March 7, 2025 04:29

virtualzavie reviewed Mar 7, 2025

View reviewed changes

MiiBond force-pushed the openpbr/diffuse_roughness branch from 203a6cf to 5ee9410 Compare March 11, 2025 23:57

MiiBond marked this pull request as ready for review March 12, 2025 18:25

MiiBond force-pushed the openpbr/diffuse_roughness branch from 6b0f7a4 to 2f1cf0e Compare March 12, 2025 18:37

Mike Bond added 10 commits March 17, 2025 14:54

Remove albedo being passed through to irradiance calc

8bf707e

Fix for EON calculation

14bca04

Add diffuse roughness vis tests

56ea8d4

Update vis test reference

4ee3c76

Fix baseDiffuseRoughness in WebGL 1.0

6d6c1a4

Add diffuse roughness for prefiltered IBL

1424766

Fix for Burley IBL calculation

2345b65

Update vis tests

b737406

Add diffuse roughness prefiltered IBL approximation

e9a21f0

Add shaders

8195f90

MiiBond force-pushed the openpbr/diffuse_roughness branch from 4bbbf90 to 8195f90 Compare March 17, 2025 21:55

Fix import error

ecc3c49

Add support for diffuse roughness with SH

770431c

Fix vis tests

a73a42a

Explicitly define eye position in pbr vert

7d5b7dd

Define diffuse roughness in pbr vert

da10941

Diffuse Roughness support #16253

Are you sure you want to change the base?

Diffuse Roughness support #16253

Conversation

MiiBond commented Mar 5, 2025 • edited Loading

bjsplat commented Mar 5, 2025

bjsplat commented Mar 5, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjsplat commented Mar 5, 2025

bjsplat commented Mar 5, 2025

bjsplat commented Mar 6, 2025

bjsplat commented Mar 6, 2025

bjsplat commented Mar 6, 2025

bjsplat commented Mar 6, 2025

bjsplat commented Mar 7, 2025

bjsplat commented Mar 7, 2025

bjsplat commented Mar 7, 2025

bjsplat commented Mar 7, 2025

virtualzavie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

virtualzavie Mar 7, 2025 • edited Loading

Choose a reason for hiding this comment

MiiBond Mar 10, 2025 • edited Loading

Choose a reason for hiding this comment

virtualzavie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjsplat commented Mar 10, 2025

bjsplat commented Mar 10, 2025

bjsplat commented Mar 10, 2025

bjsplat commented Mar 10, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 12, 2025

bjsplat commented Mar 17, 2025

bjsplat commented Mar 17, 2025

bjsplat commented Mar 17, 2025

bjsplat commented Mar 17, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 18, 2025

bjsplat commented Mar 19, 2025

bjsplat commented Mar 19, 2025

bjsplat commented Mar 19, 2025

bjsplat commented Mar 19, 2025

MiiBond commented Mar 5, 2025 •

edited

Loading

virtualzavie Mar 7, 2025 •

edited

Loading

MiiBond Mar 10, 2025 •

edited

Loading