Metal for VR

WWDC 2018

VR on macOS

• Support for HTC Vive Pro
  - Two 1440x1600 AMOLED displays, at 615 ppi
  - 78% resolution increase
  - 37% ppi increase
  - Stereo cameras for MR
• SteamVR Tracking System 2.0

• Valve SteamVR runtime
• Valve OpenVR framework

VR Application

Rendering overview

Different rendering patterns depending on MSAA

Different rendering patterns unified ( new )

• New Metal texture type
• MTLTextureType2DMultisampleArray
• Separate control of 
  - Rendering space
  - Views count
  - Anti-aliasing mode
• Single draw, render, and resolve pass

2D multisample array textures ( new )

Application creates 2D MS Array texture for rendering VR content

Cross-Process Texture Sharing ( new )

IOSurfaces vs shareable Metal textures

• Ability to share Metal textures between processes
• Shareable Metal textures can have complex structure
• Shareable texture can be used only in scope of single GPU
• Enable advanced VR use cases, but are not limited to them

Standard vs sharable texture creation ( new )

Application creates 2D Array texture that can be shared with Compositor

Passing texture to compositor ( new )

Passing frame for presentation, using shareable Metal texture

Passing across process boundary

Compositor optimizations

Frame Pacing

• Single vs Multi threaded application

• Splitting command buffer encoding

• CPU + GPU optimization

Advanced Frame Pacing

• GPU workload examples

• Pose independent work 😡

• Multi-GPU workload distribution 😆

Multi-GPU Workload Distribution

• MTLEvent
  - Mechanism of synchronizing GPU workloads
  - Synchronizes across Command Queues
• MTLSharedEvent
  - Extends Event
  - Synchronizes across GPU’s
  - Can be shared between process

Increasing Resolution

Optimizing for Vive Pro

Reducing Fill Rate ⚠️

• Clipping invisible pixels
• Multi-resolution shading
• Monoscopic far-filed rendering
• Stereo shading reprojection

Clipping invisible pixels

Use StreamVr stencil mask to clip invisible pixels

Multi-resolution shading

• Contribution of edge and corner regions

• Rendering edge and corner regions at reduced resolution

• Upscaling rendered regions to final resolution for submission