RetroRGB Interview on Vint
I recently had an interview with Bob from RetroRGB about Vint, 24 FPS video, and display temporals.
Check it out here:
https://retrorgb.com/interview-with-vint-creator-william-sokol-erhard.html
Showing posts with label rtx. Show all posts
Showing posts with label rtx. Show all posts
Monday, January 12, 2026
Friday, January 24, 2025
My New Temporal Upscaling App, Vint, is Now on Steam
Vint can now be downloaded from itch.io: https://willse.itch.io/vint
I'm happy to announce that the application I've been working on for many months now is ready to release very soon. Vint is focused on improving the temporal fidelity of any video you throw at it. Vint supports the highest quality realtime video interpolation available and allows unique features and control over the configuration. The app also supports many different types of Black Frame Insertion (BFI) strobing to add motion clarity without any interpolation.
Vint uniquely enables CRT scanout beam emulation on all video sources. CRT emulation is based on https://blurbusters.com/crt
Vint can play video from nearly any video format and most web sources. It can even use UVC video sources from capture cards or webcams and upscale on the fly. Vint allows for offline transcoding as well in case you want higher quality, want to watch on a weaker device, or you are making a video and you want to make a slow-mo video from a 30FPS recording.
I write a little more about it here: https://www.willse.me/vint
The release and further information about new features will be coming in the next few weeks.
Let me know what features you want to see.
Tuesday, August 13, 2019
Further Pathtracing and Lessons Learned
Since the last article, I've made some substantial changes. I've added screenspace settings to the build, added multiple rays per pixel, randomly generated the scene at runtime, added movement controls, added fast movement compensation with adaptive temporal accumulation for VR, and finally I added a light Gaussian blur effect to reduce noise.
Dynamically changing the temporal accumulation in addition to improving the reprojection warping allows the user to get a clean image while still remaining temporally stable and responsive to faster movement.
It's clear that simply increasing the rays per pixel has little effect on image quality while absolutely demolishing any semblance of performance. The resulting image remains noisy and generally unsatisfactory.
Temporal accumulation does a substantially better job while not impacting performance and only introducing a somewhat obnoxious temporal artifact that is mitigated by higher framerates and movement compensation. Temporal accumulation alone however still leaves a fine grain noise with pixel sized bright specks and black spots.
Finally, the softer Gaussian blur helps remove those pixel specks and spots and results in a fairly clean image better than nearly anything short of intelligent hardware accelerated denoising.
As a side experiment I tried building to UWP and using holographic remoting to run the project on the Hololens. Performance was great even at 4x resolution due to the low 720p resolution of the Hololens displays but wifi speeds & reliability as well as compression artifacts were problematic.
 |
| Simple high raycount per pixel |
 |
| Lower rays/pixel and no Gaussian blur but high temporal accumulation |
 |
| With light Gaussian blur, lower rays/pixel, and heavy temporal accumulation |
Things that worked:
- Separate rendering for each eye for XR
- Moving Unity GameObjects and meshes each frame
- Gaussian blur to reduce noise
- Dynamically changing RenderTexture resolution
- Distorting temporal accumulation based on 3DOF head rotation
- Changing temporal accumulation rate based on head movement rate
- Hololens support with Holographic remoting
- Foveated rendering with Graphics.CopyTexture()
- OpenGL ES 3.1 compute shaders for Android
- MultiGPU parallel compute shaders
- RT core acceleration
- MultiGPU support with RT cores and/or shaders
- AI tensor core or CPU intelligent realtime denoising
- Dynamic rate of rays per pixel (based on pixel coordinates)
- Foveated rendering with that rays/pixel value
- Complex meshes without stranglehold bandwidth and calculation bottlenecks
Friday, June 21, 2019
How to Pathtrace in VR
Before I start, I want to thank David Kuri of the VW Virtual Engineering Lab in Germany for his example project and tutorial that I've based my modifications off of.
The following is his tutorial:
http://blog.three-eyed-games.com/2018/05/03/gpu-ray-tracing-in-unity-part-1/
This link is to the BitBucket Git repo:
https://bitbucket.org/Daerst/gpu-ray-tracing-in-unity/src/Tutorial_Pt3/
My project can be found at https://github.com/Will-VW/Pathtracing
To start, this is a project I worked on as an experiment while part of the Volkswagen Virtual Engineering Lab in the Bay Area.
While searching for interesting raytracing implementations to challenge our 2080 TI, we were coming up fairly short. We found a couple lackluster and simple DirectX projects, poor Unity and Unreal support, and marginally impressive demos on rails. That is until we found our colleague's project using simple compute shaders in regular Unity.
Obviously we forgo RTX and DXR acceleration and for the moment we do not have denoising or proper mesh support but we gain near unlimited flexibility and control.
As far as we are aware (as of June 2019), there are no raytracing or pathtracing applications, demos, games, roadmaps, plans, or anything else with any sort of VR support or connection. We decided to change that.
I took David's existing project that worked in "simple" single camera pancake views and modified it until I could get it to work properly with SteamVR.
Temporal accumulation was the biggest issue and due to the fact that both right and left eyes were from different perspectives and the constant VR camera movement it was effectively impossible to have anything more than a single frame of accumulation in the headset. I first used instances to split the left and right eye textures so accumulation would be separated. I then used a shader to account for head rotation and adjust the accumulated frames accordingly to attempt to line up with the newest perspective.
Both efforts were effective. The former far more so than the latter which still suffered from artifacts due to the distortion at high FOVs like VR headsets have.
Next I targeted a dynamic resolution so that the number of rays projected each frame could be controlled to suit our needs. I changed the RenderTexture resolution appropriately and allowed the program to render based off of the VR headset resolution rather than the Unity mirrored view resolution.
Later, I attempted foveated rendering to improve performance which remained lackluster. I learned how expensive the Graphics.Blit() and Graphics.CopyTexture() methods are in bandwidth and found that while foveated rendering was possible, it hurt performance more than it helped.
I then added in support for moving around mesh objects that were imported at runtime based on the GameObject Transform. This allowed me to put in a moving and rotating block that moved around the scene. I also learned of the inefficiencies of the existing mesh pathtracing technique and began to explore more implicit surface calculations.
I looked at SLI support to parallelize the pathtracing but found that compute shaders do not support multi-GPU and CUDA/OpenCL options were unviable for our purposes.
Thus we managed to get a fully pathtraced program running at full 90FPS in a Vive Pro at native resolution and 1 ray per pixel with perfectly accurate reflections, lighting, ambient occlusion, and shadows. Albeit with a lot of noise, no RTX acceleration, limited mesh usage due to performance, no standard materials or shaders, and no effective foveated rendering.
Subscribe to:
Posts (Atom)