docRight
download 
Report Transcription
Right
Virtual Reality Features of NVIDIA GPUs Mark Kilgard, July 27, 2016 Mark Kilgard My Background Principal System Software Engineer OpenGL driver implementation & API evolution Other interests Programmable shading languages, original Cg GPU shading language Web, vector graphics & path rendering, GPU rendering for 2D graphics Also known for creating OpenGL Utility Toolkit (GLUT) 2 GPU Challenges for Virtual Reality Rendering 1. Must render unique view for each eye Two scene renders per frame 2. Wide field-of-view rendered More pixels, distributed towards periphery 3. HMD lens magnifies rendered scene Non-linear inverse lens warp Allows the HMD lens magnification to provide convincing sense of presence 3 Magnification Effect of HMD Lens Graph paper viewed & magnified through HMD lens 4 Head Mounted Display Lens Effect Barrel Distorted Images on HMD Screen Magnified Overlapping Wide Field-of-view Binocular View 5 Focus on Virtual Reality Features of Pascal GPUs Virtual Reality rendering features in Pascal GPUs GeForce 1080/1070/1060 NEW Quadro P6000 & P5000 As exposed in standard 3D APIs 6 Background: NVIDA GPU Architecture Road Map What is Pascal mentioned on last slide? Our interest NVIDIA GPU architectures of interest: Pascal 7 Conventional 3D Single-eye Scene 8 Conventional 3D Single-eye Scene 9 Wide Stereo Views of Same Scene Left Right 10 Left Right 11 Right Left 12 Left & Right Views Are Similar but Different Left and Right eye views have different occlusion While left & right eye views look similar, must be draw independently Really only different in a small translation in X NVIDIA Single Pass Stereo supports generating left & right eye positions GPU automatically makes left & right eye versions of each primitive Then renders each view to a different framebuffer slice Exposed in OpenGL by NV_stereo_view_rendering extension Exposed for Direct3D 11 by NVAPI SinglePassStereoMode feature 13 Image Difference of Two Views − Left eye view + 0.5 = Right eye view Clamped difference image 14 Left & Right Views Rendered Efficiently in One Pass 15 Lens Matched Shading Left Right 16 Visualizing Lens Matched Shading 17 Lens Matched Shading Left Right 18 Visualization of Lens Matched Shading Rendering 19 Still Needs Non-linear Warp Non-linear image Lens Matched Shading rendering Warped image suitable for HMD display 20 Still Needs Non-linear Warp Non-linear image Lens Matched Shading rendering BUT substantially less pixel motion and distortion than conventional VR re-warping Warped image suitable for HMD display 21 Example HMD Post-rendering Warp 22 Lens Matched Shading with Window Rectangle Testing 23 Lens Matched Shading with Window Rectangle Testing Nothing in black corners is shaded or even rasterized Yellow lines show overlaid 8 inclusive window rectangles Same 8 window rectangles “shared” by each view’s texture array layer 24 Lens Matched Shading with Window Rectangle Testing 25 What Didn’t Have to be Rendered 17% of left view pixels 17% of right view pixels 26 Window Rectangle Testing During Rasterization New multi-vendor OpenGL extension EXT_window_rectangles Window Rectangles Test Provides a set of window-space rectangles, 8 for NVIDIA GPUs Rasterizer either includes or excludes rasterization from rectangles NEW stage Discards at speed-of-light As if free to rasterize to irregular rectangular regions OpenGL’s fragment processing pipeline 27 Warped Lens Matched Scene Warped version of Lens Matched Shading to match HMD lens 28 Warped Lens Matched Shading 29 Visualizing Warp Window Rectangles 30 What Didn’t Have to be Warped 17% of left view pixels 17% of right view pixels 31 VR Rendering Pipeline in Images LMS Left Eye View Warped Left Eye View LMS Right Eye View Warped Right Eye View Scene LMS = Lens Matched Shading 32 VR Rendering Pipeline in Images LMS Left Eye View Warped Left Eye View Pascal does all this efficiently in a single rendering pass! 8 viewports, 1 pass Displayed within HMD Scene LMS Right Eye View Warped Right Eye View Single Rendering Pass Drawn with Single Triangle Single Pass Stereo + Lens Matched Shading + Window Rectangle Testing Fragment Shader Warping Window Rectangle Testing Perception to user is linear rendering HMD lens “undoes” warping to provide a perceived wide field-of-view 33 Traditional 3D Rendering Pipeline uses pinhole camera projection, rasterizing to uniform rectilinear pixel grid App Processing Transform 3D View Rasterize 3D View Display Frame Downsample monitor directly displays uniform rectilinear pixel grid if multisampled 34 Virtual Reality Rendering Pipeline Rendering Left & Right Eye Views warps uniform rectilinear grid based on HMD’s device profile Transform Left View Rasterize Left View Downsample HMD Warp Display Frame to HMD App Processing Transform Right View Rasterize Right View Downsample HMD Warp second view rendering 35 Virtual Reality Rendering Pipeline With Lens Matched Shading less pixel resampling effort & error operating on LMS rasterized images Transform Left View Lens Matched Shading Rasterize Left View Lens Matched Shading Rasterize Right View Downsample HMD Warp Display Frame to HMD App Processing Transform Right View Downsample HMD Warp replicates primitive to 4 projected quadrants 36 Virtual Reality Rendering Pipeline With Single Pass Stereo App Processing Transform Left & Right View Rasterize Left View Downsample HMD Warp Display Frame to HMD Rasterize Right View Downsample HMD Warp one transform instance, outputs left & right position 37 Virtual Reality Rendering Pipeline Single Pass Stereo + Lens Matched Shading use window rectangle testing to bound lens region, during both rasterization & warp App Processing Transform Left & Right View + Lens Matched Shading Rasterize Left View Lens Matched Shading Rasterize Right View Downsample HMD Warp Display Frame to HMD Downsample HMD Warp Advantages 1. Single transform for both views 2. Closer match between rendering & lens images 3. Fewer overall pixels for similar quality 38 Pascal GPU Features for VR Unique to NVIDIA Pascal renders left & right eye Better matches rendering to HMD lens In single rendering pass Single pass Halves CPU load for rendering Four separately projected quadrants Better concurrent texture locality Less rendering, less resampling error OpenGL: NV_stereo_view_rendering OpenGL: NV_clip_space_w_scaling Direct3D: NVAPI SinglePassStereoMode Direct3D: NVAPI ModifiedWSupport 39 Other NVIDIA Virtual Reality Initiatives 40 More Information VRWORKS 2.0 Software Development Kit Available NOW Pascal OpenGL extensions for Virtual Reality Specification now public In official OpenGL.org extension registry OpenGL Extension Wrangler 2.0 (GLEW) support Example code in VRWORKS 2.0 SDK VRWORKS also documents NVAPI usage For Direct3D 11 Includes Examples code 41
