Tuesday, August 13, 2019

Further Pathtracing and Lessons Learned


Since the last article, I've made some substantial changes. I've added screenspace settings to the build, added multiple rays per pixel, randomly generated the scene at runtime, added movement controls, added fast movement compensation with adaptive temporal accumulation for VR, and finally I added a light Gaussian blur effect to reduce noise.


Dynamically changing the temporal accumulation in addition to improving the reprojection warping allows the user to get a clean image while still remaining temporally stable and responsive to faster movement.

It's clear that simply increasing the rays per pixel has little effect on image quality while absolutely demolishing any semblance of performance. The resulting image remains noisy and generally unsatisfactory.
Temporal accumulation does a substantially better job while not impacting performance and only introducing a somewhat obnoxious temporal artifact that is mitigated by higher framerates and movement compensation. Temporal accumulation alone however still leaves a fine grain noise with pixel sized bright specks and black spots.
Finally, the softer Gaussian blur helps remove those pixel specks and spots and results in a fairly clean image better than nearly anything short of intelligent hardware accelerated denoising.

As a side experiment I tried building to UWP and using holographic remoting to run the project on the Hololens. Performance was great even at 4x resolution due to the low 720p resolution of the Hololens displays but wifi speeds & reliability as well as compression artifacts were problematic.

Simple high raycount per pixel
Lower rays/pixel and no Gaussian blur but high temporal accumulation

With light Gaussian blur, lower rays/pixel, and heavy temporal accumulation


Things that worked:
  • Separate rendering for each eye for XR
  • Moving Unity GameObjects and meshes each frame
  • Gaussian blur to reduce noise
  • Dynamically changing RenderTexture resolution
  • Distorting temporal accumulation based on 3DOF head rotation
  • Changing temporal accumulation rate based on head movement rate
  • Hololens support with Holographic remoting
Things that didn't:
  • Foveated rendering with Graphics.CopyTexture()
  • OpenGL ES 3.1 compute shaders for Android
  • MultiGPU parallel compute shaders
Things that might be possible but aren't easy:
  • RT core acceleration
  • MultiGPU support with RT cores and/or shaders
  • AI tensor core or CPU intelligent realtime denoising
  • Dynamic rate of rays per pixel (based on pixel coordinates)
  • Foveated rendering with that rays/pixel value
  • Complex meshes without stranglehold bandwidth and calculation bottlenecks