Metal for VR
Metal for VR
WWDC 2018
VR on macOS
- Support for HTC Vive Pro
- Two 1440x1600 AMOLED displays, at 615 ppi
- 78% resolution increase
- 37% ppi increase
- Stereo cameras for MR - SteamVR Tracking System 2.0
![](https://cdn-images-1.medium.com/max/800/1*s82MERyAjuHmm4Nag5NPQw.png)
- Valve SteamVR runtime
- Valve OpenVR framework
![](https://cdn-images-1.medium.com/max/800/1*L5Q645fA1PJz9JSK6lo62g.png)
VR Application
Rendering overview
![](https://cdn-images-1.medium.com/max/800/1*ZiWko69FNXb4bdaOTfmXAQ.png)
Different rendering patterns depending on MSAA
![](https://cdn-images-1.medium.com/max/800/1*Iru0ZHTICr2IVG1PaFIojA.png)
Different rendering patterns unified ( new )
- New Metal texture type
MTLTextureType2DMultisampleArray
- Separate control of
- Rendering space
- Views count
- Anti-aliasing mode - Single draw, render, and resolve pass
![](https://cdn-images-1.medium.com/max/800/1*0BwPWS4PquZqwlN-d7Krcg.png)
2D multisample array textures ( new )
Application creates 2D MS Array texture for rendering VR content
![](https://cdn-images-1.medium.com/max/800/1*iPFqj-ySQpyQRWcaBeeyCQ.png)
![](https://cdn-images-1.medium.com/max/800/1*gpdfjrfy3cKz91xQOvYvFw.gif)
Cross-Process Texture Sharing ( new )
IOSurfaces vs shareable Metal textures
- Ability to share Metal textures between processes
- Shareable Metal textures can have complex structure
- Shareable texture can be used only in scope of single GPU
- Enable advanced VR use cases, but are not limited to them
Standard vs sharable texture creation ( new )
Application creates 2D Array texture that can be shared with Compositor
![](https://cdn-images-1.medium.com/max/800/1*rHARVCKb0M1IkcBEL23COw.png)
Passing texture to compositor ( new )
Passing frame for presentation, using shareable Metal texture
![](https://cdn-images-1.medium.com/max/800/1*9nca-zqONXlVhrb2NGDzew.png)
![](https://cdn-images-1.medium.com/max/800/1*RM8TK2H7UA3wCmVuEENM-A.png)
Passing across process boundary
![](https://cdn-images-1.medium.com/max/800/1*a93yhxyE4ar6sJ1XyxOktA.png)
![](https://cdn-images-1.medium.com/max/800/1*BpEV0TX742Za-8aycfuE3w.gif)
Compositor optimizations
![](https://cdn-images-1.medium.com/max/800/1*E0WvCw35pYnlrRA4lMDRcA.gif)
Frame Pacing
- Single vs Multi threaded application
![](https://cdn-images-1.medium.com/max/800/1*OzB2BsiU3WYrWT-55DB5Pg.png)
![](https://cdn-images-1.medium.com/max/800/1*0cfxjo1Z2C-0MYOPomr3tA.png)
- Splitting command buffer encoding
![](https://cdn-images-1.medium.com/max/800/1*-Pnva0t1U9Jj1ezlB10tMA.png)
![](https://cdn-images-1.medium.com/max/800/1*g2Sq87WY7G1tSR-yJgUOkA.png)
- CPU + GPU optimization
![](https://cdn-images-1.medium.com/max/800/1*SwgwdDgNHorY6-_hYonUNw.png)
Advanced Frame Pacing
- GPU workload examples
![](https://cdn-images-1.medium.com/max/800/1*oBZbohBZ-sj9vhCSiWb6mw.png)
- Pose independent work 😡
![](https://cdn-images-1.medium.com/max/800/1*lwDTPKoafg7ssAcjQVtvtw.png)
- Multi-GPU workload distribution 😆
![](https://cdn-images-1.medium.com/max/800/1*xNrF4noSSx_94g8-cQ0Bjw.png)
Multi-GPU Workload Distribution
- MTLEvent
- Mechanism of synchronizing GPU workloads
- Synchronizes across Command Queues - MTLSharedEvent
- Extends Event
- Synchronizes across GPU’s
- Can be shared between process
![](https://cdn-images-1.medium.com/max/800/1*JhRPaqVfeK2lVQ68W6aMqA.gif)
![](https://cdn-images-1.medium.com/max/800/1*5cTSX2J5NXwiywQTyeq_aQ.png)
Increasing Resolution
Optimizing for Vive Pro
![](https://cdn-images-1.medium.com/max/800/1*CYkpBA8mKXP4NW0eeUp7vA.png)
Reducing Fill Rate ⚠️
- Clipping invisible pixels
- Multi-resolution shading
- Monoscopic far-filed rendering
- Stereo shading reprojection
Clipping invisible pixels
Use StreamVr stencil mask to clip invisible pixels
![](https://cdn-images-1.medium.com/max/800/1*EoRb7Hrk0Xh3SXyisui4zQ.png)
![](https://cdn-images-1.medium.com/max/800/1*8eTJY_mX-yBPaBOGCpPIdw.png)
Multi-resolution shading
- Contribution of edge and corner regions
![](https://cdn-images-1.medium.com/max/800/1*eNgxZn9hoQFKb2kCPJVLrg.png)
- Rendering edge and corner regions at reduced resolution
![](https://cdn-images-1.medium.com/max/800/1*gtgX67FYIwUfD9vvbZ9YLA.png)
- Upscaling rendered regions to final resolution for submission
![](https://cdn-images-1.medium.com/max/800/1*LwGkxmNs5o6vg9nVUUZQHA.png)
![](https://cdn-images-1.medium.com/max/800/1*9AbRrlQNznYkrj1fs08_5Q.png)