Metal for OpenGL Developers
Metal for OpenGL Developers
WWDC 2018
![](https://cdn-images-1.medium.com/max/800/1*9C5ZJ8l8UDoQu3T3hXgNqQ.png)
- These legacy APIs are deprecated
- Still available in iOS 12, macOS 10.14 and tvOS 12
- Begin transitioning to Metal
Choosing an Approach
- High-level Apple frameworks
- SpriteKit, SceneKit, and Core Image - 3rd-party engine
- Unity, Unreal, Lumberyard, etc
- Update to lastest version
Challenges with OpenGL
- OpenGL designed more than 25 years ago
- Core architecture reflects the origin of 3D graphics
- Extensions retrofitted some GPU feature - Fundamental design choices based on past principles
- GPU pipeline has changed
- Multithreaded operation not considered
- Asynchronous processing, not core
Design Goals for Metal
- Efficient GPU interaction
- Low CPU overhead
- Multithreaded execution
- Predictable operation
- Resource and synchronization control - Approachable to OpenGL developers
- Built for modern and Apple-design GPUs
Key Conceptual Differences
- Expensive operations less frequent
- Expensive CPU operations performed less often
- More GPU command generation during object creation
- Less needed when rendering - Modern GPU pipeline
- Reflects the modern GPU architectures
- Closer match yields less costly translation to GPU commands
- State grouped more efficiently - Multithreaded execution
- Designed for multithreaded execution
- Clear rules for multithreaded usage
- Cross thread object usability - Execution model
- True interaction between software and GPU
- Predictable operation allows efficient designs
- Thinner stack between application and GPU
![](https://cdn-images-1.medium.com/max/800/1*1NQqC7q6sfEvjUB47atrPA.png)
![](https://cdn-images-1.medium.com/max/800/1*1Qyc4QWwtk4dfxNlsNmEwQ.gif)
![](https://cdn-images-1.medium.com/max/800/1*P6q4ag1xEw7vYpvjboV2Hw.gif)
Command Encoders
- Render Command Encoder
- Blit Command Encoder
- Compute Command Encoder
Render Command Encoder
- Commands for render pass
- Encodes a series of render commands
- Also called a Render Pass
- Set render object for the graphics pipeline ( Buffer, texture, shaders )
- Issue draw commands ( draw primitives, draw index primitives, instanced draws ) - Render targets
- Associated with a set of render targets ( Textures for rendering )
- Specify a set of render targets upon creation
- All draw commands directed to these for lifetime of encoder
- New render targets need a new encoder
- Clear delineation between sets of render targets
Render Object
- Textures
- Buffers
- Samplers
- Render pipeline states
- Depth stencil states
Render Object creation
- Create from a device
- Usable only on the device - Object state set at creation
- Descriptor object specifies properties for render object - State set at creation fixed for the lifetime of the object
- Image data of textures and values in buffers can change - Metal compiles objects into GPU state once
- Never needs to check for changes and recompile - Multithreaded usage more efficient
- Metal does not need to protect state from changes on other threads
Metal Porting
![](https://cdn-images-1.medium.com/max/800/1*VkmwM6c1KX6F7YLydKXhaA.png)
Metal Shading Language
- Based on C++
- Classes, templates, structs, enums, namespaces - Built-in types for vectors and matrices
- Built-in functions and operators
- Built-in classes for textures and samplers
![](https://cdn-images-1.medium.com/max/800/1*e_7FKFAjHJOojqSVtPeuvg.png)
SIMD Type Library
Types for shader development
- Vector and matrix types
- Usable with Metal shading language and application code
![](https://cdn-images-1.medium.com/max/800/1*fe5YQvFU9RBRQRw9ekmIXg.png)
Shader Compilation
Build with Xcode
- Xcode compiles shaders into a Metal library (.metallib)
- Front-end compilation to binary intermediate representation
- Avoids parsing time on customer systems
- By default, all shaders built into default.metallib
- Placed in app bundle for run time retrieval
Runtime Shader Compilation ⚠️
- Also can build shaders from source at runtime
- Significant disadvantages
- Full shader compilation occurs at runtime
- Compilation errors less obvious
- No header sharing between application and runtime built shaders - Build time compilation recommended
Devices
- A device represents one GPU
- Create render objects
- Texture, buffers, pipelines - macOS multiple devices may be available
- Default device suitable for most applications
![](https://cdn-images-1.medium.com/max/800/1*KLIw94YjAvgt-DCmm4DLOw.png)
![](https://cdn-images-1.medium.com/max/800/1*tXNQp91DrDNafxV-5uQUjw.png)
Command Queues
- Queue created from a device
- Queue execute command buffers in order
- Create queue at initialization - Typically one queue sufficient
Texture
![](https://cdn-images-1.medium.com/max/800/1*0FWV9RjVrDu5ow0u17zOaw.png)
Storage Modes
![](https://cdn-images-1.medium.com/max/800/1*HSMKrN1TbLuSNvYsbao5lA.png)
![](https://cdn-images-1.medium.com/max/800/1*T2qCX7Fp1mnDy8y4BGNDhw.png)
![](https://cdn-images-1.medium.com/max/800/1*Skjz5OXRVN-qaJLVdGbeTA.png)
![](https://cdn-images-1.medium.com/max/800/1*KaO24-l4-g1bagKn8GAvPA.png)
![](https://cdn-images-1.medium.com/max/800/1*xnCH2H7xd7SkuiGvpodKBw.png)
Texture Differences ⚠️
- Sampler state never part of texture
- Wrap modes, filtering, min/max LOD - Texture image data not flipped
- OpenGL uses bottom-left origin, Metal uses top-left origin - Metal does not perform format conversion
Buffers
- Metal uses buffers for vertices, indices, and all uniform data
- OpenGL’s vertex, element, and uniform buffer are similar
- Easier to port apps that have adopted these
![](https://cdn-images-1.medium.com/max/800/1*SIC8n1BHrT4fTHmhs_4NLg.png)
![](https://cdn-images-1.medium.com/max/800/1*8wtHHNi9E2uPwULxEe2M2A.png)
Notes About Buffer Data ⚠️
!!! Pay attention to alignment !!!
![](https://cdn-images-1.medium.com/max/800/1*FzkjV2EZnxgn-YdIF8gnbA.png)
- SIMD libraries vector and matrix types follow same rules as Metal shaders
- Special packed vector types available to shaders
-packed_float3
consumes 12 bytes
-packed_half3
consumes 6 bytes - Cannot directly operate on packed types
- Cast to non-packed type required
Storage Modes for Porting ⚠️
- Use most convient storage modes
- Easier access to data - On iOS
- Create all textures and buffers withMTLStorageModeShared
- On macOS
- Create all textures withMTLStorageModeManged
- Make judicious use ofMTLStorageModeShared
for buffers
Separate GPU only data from CPU accessible data
MetalKit
Texture and buffer utilities
- Texture Loading
- Textures from KTX, PVR, JPG, PNG, TIFF, etc - Model Loading
- Vertex buffers from USD, OBJ, Alembic, etc
PipeLines
![](https://cdn-images-1.medium.com/max/800/1*3v1at5zuXqfnsJoMh-Su8A.png)
![](https://cdn-images-1.medium.com/max/800/1*fSCkI0rwFmv_jACA0ij1Hw.png)
Pipeline Differences
![](https://cdn-images-1.medium.com/max/800/1*JB3E92FgaTIQEMVmOZeLwQ.png)
Pipeline Building
- Create at initialization
- Full compilation key advantage of state grouping
- Choose a canonical vertex layout for meshes
- Use a limited set of render target formats - Lazy creation at draw time ⚠️
- Store pipeline state objects in a dictionary using descriptor as key
- Construct descriptor at draw time with current state
- Retrieve existing pipeline from dictionary OR build new pipeline
Create Render Objects at Initialization
- Object creation expensive
- Pipelines require backend compilation
- Buffers and textures need allocations - Once created, much faster usage during rendering
Command Buffers
- Explicit control over command buffer submission
- Start with one command buffer per frame
- Optionally split a frame into multiple command buffers to
- Submit early and get the GPU started
- Build commands on multiple threads - Completion handler invoked when execution is finished
![](https://cdn-images-1.medium.com/max/800/1*-2oOaLho4ED5s08_6JAAFA.png)
![](https://cdn-images-1.medium.com/max/800/1*RbOzj6q-k9JOEiZXwFmEpw.png)
![](https://cdn-images-1.medium.com/max/800/1*IV1RzpvYVB0Gz4o7tteVFg.png)
Resource Updates
- Resources are explicitly managed in Metal
- No implicit synchronization like OpenGL
- Allows for fine gained synchronization
- Application has complete control
- Best model dependent on usage
- Triple buffering recommended
![](https://cdn-images-1.medium.com/max/800/1*cmRMPRJKCJQt9BqLyfTWAg.png)
Problem
![](https://cdn-images-1.medium.com/max/800/1*eEcyRp9d5Ou3SMsTz4mjJg.gif)
Temporary Solution ⚠️
Synchronous wait after every frame
![](https://cdn-images-1.medium.com/max/800/1*98g9hclkBSAVsARNn6hirQ.png)
Triple Buffering
Shared buffer pool 👍
![](https://cdn-images-1.medium.com/max/800/1*Bs-Uhv2Di7ukpfLLxKzIrg.gif)
![](https://cdn-images-1.medium.com/max/800/1*a9nQku3G6SxhsuA1H7XZoA.png)
Render Encoders
![](https://cdn-images-1.medium.com/max/800/1*AkXjrWnuj3QOCXquMjJ_EQ.png)
Render Pass Descriptor
![](https://cdn-images-1.medium.com/max/800/1*f6ZmDQ-I05KK77qFUuanMA.png)
Render Pass Descriptor
![](https://cdn-images-1.medium.com/max/800/1*f6ZmDQ-I05KK77qFUuanMA.png)
Render Pass Setup
![](https://cdn-images-1.medium.com/max/800/1*pwnBjCtyhDDfxlpWJJbUTg.png)
ender Pass Load and Store Actions
![](https://cdn-images-1.medium.com/max/800/1*rblBLRxgfrMxF4HsGosd-g.png)
![](https://cdn-images-1.medium.com/max/800/1*ypz_myGDMaa_sKy-0PQtsg.png)
![](https://cdn-images-1.medium.com/max/800/1*1Xd-UaW_6349ekGQskIY1A.png)
Rendering with OpenGL
![](https://cdn-images-1.medium.com/max/800/1*wS8tCa2Ae958IFJ4JpJouQ.png)
Rendering with Metal
![](https://cdn-images-1.medium.com/max/800/1*nVVfP3r1IqVSWzH3Xwx2bg.png)
Display
Drawables and presentation
- Drawables — Textures for on screen display
- Each frame MTKView provides
- Drawable texture
- Render pass descriptor setup with the drawable - Render to drawables like any other texture
- Present drawable when down rendering
![](https://cdn-images-1.medium.com/max/800/1*Su4_3R3-BhsNzL3E39dggg.png)
![](https://cdn-images-1.medium.com/max/800/1*q9J7JvZdgikz8VrSwnQESA.png)
Incrementally Porting ⚠️
- Create shared Metal/OpenGL textures using IOSurface or CVPixelBuffer
- Render to texture on 1 API and read in the other - Can enabled mixed Metal/OpenGL applications
- Sample Link
Multithreading
Metal is designed to facilitate Multithreading
- Consider multithreading if application is CPU bound
- Encode multiple command buffers simultaneously
- Split single render pass using MTLParallelCommandEncoder
Staying on the GPU
- Metal natively supports compute
- Performance benefits
- Reduces CPU utilization
- Reduces GPU-CPU synchronization points
- Free’s data bandwidth to the GPU - New algorithms possible
- Particle systems, physics, object culling
Developer Tools
Debug and optimize your applications
- Xcode contains an advances set of GPU tools
- Enable Metal’s API vaildation layer
- On by default when target run from Xcode
![](https://cdn-images-1.medium.com/max/800/1*kpHrqyhP65FZqAda-rKU-w.png)
![](https://cdn-images-1.medium.com/max/800/1*9SwaPv6TjRNgdxeWu3-2ag.png)
![](https://cdn-images-1.medium.com/max/800/1*Cr3FlW32Ukw-5N1fR4zTeA.png)
![](https://cdn-images-1.medium.com/max/800/1*f6Nj2ylbw8ov7I5J31JmhA.png)
![](https://cdn-images-1.medium.com/max/800/1*3r_vIfTOBVijHXQQpLV92w.png)