Integrating Apps and Content with AR Quick Look
Integrating Apps and Content with AR Quick Look
WWDC 2018
What Is AR Quick Look?
- Preview 3D content in AR
- System-wide
- Uses Quick Look technology
- Available on iOS 12
![](https://cdn-images-1.medium.com/max/800/1*sSQ3QDVE3yFA-xY41AvZnw.gif)
![](https://cdn-images-1.medium.com/max/800/1*XvAuV0Gujmfrv2ufKakXkA.png)
Developer Benefits
- No need to create your own
- Really easy to integrate
- Built-in AR Setup
- No understanding of AR technology required
‘usdz’ FIle
- New File format for distributing 3D models
- Packages a model and its textures into a single file
- Based on Pixar’s open-source USD format
- Supported on iOS and macOS
- used_converter tool in Xcode 1
AR Quick Look Integration
![](https://cdn-images-1.medium.com/max/800/1*YlYAWt2vkW6NTnRl8ojNDw.png)
Quick Look Preview Flow
![](https://cdn-images-1.medium.com/max/800/1*DpbQOI-09AXtsPQe5BHmXw.png)
Previewing ‘usdz’ Objects Using Quick Look
![](https://cdn-images-1.medium.com/max/800/1*cwm-MLLhqZef01mpgmRt0Q.png)
![](https://cdn-images-1.medium.com/max/800/1*iNWvkIv0hrS2FDnIJ2lI1A.png)
![](https://cdn-images-1.medium.com/max/800/1*Yzy3vrir2U5VebHziyLOuw.gif)
![](https://cdn-images-1.medium.com/max/800/1*pH1yQleO_B9qaBAjLNvSQg.png)
Previewing ‘usdz’ content in Safari
HTML markup
![](https://cdn-images-1.medium.com/max/800/1*HeC4yY9fAN93Ug8qJa2lXA.gif)
Automatic badging
![](https://cdn-images-1.medium.com/max/800/1*59yHqlqAdJ8cUN73OtSwFw.png)
Provide custom thumbnail image
![](https://cdn-images-1.medium.com/max/800/1*J9-REtRndejQEv_hWipXGg.png)
Supports drag and drop
![](https://cdn-images-1.medium.com/max/800/1*zXypqGKZRXFUFCeJYYKB8w.gif)
Supports long-press
![](https://cdn-images-1.medium.com/max/800/1*DabEPl3QFgLp-haPUXHAeQ.gif)
HTML Markup for Previewing ‘usdz’ Objects
<img> element
![](https://cdn-images-1.medium.com/max/800/1*xLBjyqx11x8c1L_V2zMs_A.png)
<picture> element
![](https://cdn-images-1.medium.com/max/800/1*gkArnn3UWFFjoZeOe2t_gA.png)
MIME type
![](https://cdn-images-1.medium.com/max/800/1*4CPh3LW4avc8_WPY7GRtDw.png)
Creating 3D Models for AR Quick Look
Placement
- Place objects facing towards the camera (facing + z)
- Base of object should sit on the ground plane (y=0)
- Pivot point should be at the origin (x,y,z = 0)
![](https://cdn-images-1.medium.com/max/800/1*MUfMMLUB6EHXIkOlUKM58w.gif)
Physical Size
![](https://cdn-images-1.medium.com/max/800/1*qDfWzRrR2RL-RBkFxcz87Q.gif)
Animation
- Provide an “idle” animation to add life to the object
- Animations always loop
- Animations can be a mix of skeletal animation and transform animation
![](https://cdn-images-1.medium.com/max/800/1*8XmiveunvTtQN_unsvUr4w.gif)
Animation Tips
- Choose animations that enhance AR immersiveness
- Don’t animate objects away from the origin
- Keep a consistent bounding box throughout an animation
- Prefer animations that make sense at a static location
- Or create animations as self-contained scenes
![](https://cdn-images-1.medium.com/max/800/1*B2JC_wyxApWnHcKUCMjAZw.gif)
Contact Shadow
- AR Quick Look provides a contact shadow for you
- Can turn the shadow on / off
- Can apply ambient lighting conditions - Don’t bake a contact shadow into the model
- First animation frame is used for shadow creation
Model Appearance
Albedo (base color)
![](https://cdn-images-1.medium.com/max/800/1*YYX2JHTLfddmOyirGA8kmA.png)
Metallic (conductor or insulator)
![](https://cdn-images-1.medium.com/max/800/1*b8hmuPfHsZYSYoGIjgyAWw.png)
Roughness (rough or shiny)
![](https://cdn-images-1.medium.com/max/800/1*klyBb2ED2ALxvhD0Hvy2MA.png)
Normal (surface details)
![](https://cdn-images-1.medium.com/max/800/1*6k9Zbqnud7tcau0KP18EAg.png)
Ambient occlusion (internal shadows)
![](https://cdn-images-1.medium.com/max/800/1*xPDnaCVhUlhTdM-6kctaQw.png)
Emissive (emits light)
![](https://cdn-images-1.medium.com/max/800/1*H2CY6l3vwmiSkt5rROGGiw.png)
Transparency
- Use a separate material for transparent and non-transparent parts of the model
- Provide an albedo texture with transparency in its alpha channel
- Use transparency for see-through areas, not for cutouts
![](https://cdn-images-1.medium.com/max/800/1*00X9DtvjZeJHiAMsy3v8mQ.png)
Texture Formats
!! Texture should be square powers of 2 (2048, 1024, 512 …) !!
- Albedo (RGB/RGBA)
- Metallic (Grayscale)
- Roughness (Grayscale)
- Normal (RGB)
- Ambient occlusion (Grayscale)
- Emissive (RGB)
Supporting All Devices
- System-wide extension that shares system memory with applications
- Optimize and test your models for high-memory devices
- iPhone 7+, iPhone 8+, iPhone X
- iPad Pro 12.9' - AR Quick Look will dynamically downsample textures for other devices when needed
- Meshes and animations are not modified
Size Limits
- Many factors affect a model’s memory requirements
- Mesh, animations complexity, Texture size & count - As a guide, for a model with a single PBR material
- 100k polygons
- One set of 2048 x 2048 PBR textures
- Ten seconds of animation - Always test your model on a real device
Optimizing Models
- Freeze transforms and merge adjacent vertices
- If possible, use a single material/texture set for the entire model
- Don’t include textures you don’t need
- Spend your texture budget on ares that add most value and realism
- Remember that pixels have a physical size in AR
- Balance texture size and quality against download size
‘usdz’ Converter
- Command line tool to convert 3D models to .usdz format
- Ships with Xcode 10
- Maps PBR textures to meshes and submeshes
- Input format
- OBJ file
- Single-frame Alembic (ABC) file
- USD file (.usda, .usdc)
![](https://cdn-images-1.medium.com/max/800/1*td_9xZh7a39Yhmhl5h9ocw.png)
Anatomy of a ‘usdz’ File
- .usdz file is an uncompressed zip archive
- First file in the archive is a .usdc file (model, animation, material)
- Remaining files (if needed) are image files
![](https://cdn-images-1.medium.com/max/800/1*q1v6siWKWtbbMBbxGUQLeA.png)
Sample
![](https://cdn-images-1.medium.com/max/800/1*kZe5NlPFKDnIAnximlTfvw.png)