Integrating Apps and Content with AR Quick Look
Integrating Apps and Content with AR Quick Look
WWDC 2018
What Is AR Quick Look?
- Preview 3D content in AR
- System-wide
- Uses Quick Look technology
- Available on iOS 12
Developer Benefits
- No need to create your own
- Really easy to integrate
- Built-in AR Setup
- No understanding of AR technology required
‘usdz’ FIle
- New File format for distributing 3D models
- Packages a model and its textures into a single file
- Based on Pixar’s open-source USD format
- Supported on iOS and macOS
- used_converter tool in Xcode 1
AR Quick Look Integration
Quick Look Preview Flow
Previewing ‘usdz’ Objects Using Quick Look
Previewing ‘usdz’ content in Safari
HTML markup
Automatic badging
Provide custom thumbnail image
Supports drag and drop
Supports long-press
HTML Markup for Previewing ‘usdz’ Objects
<img> element
<picture> element
MIME type
Creating 3D Models for AR Quick Look
Placement
- Place objects facing towards the camera (facing + z)
- Base of object should sit on the ground plane (y=0)
- Pivot point should be at the origin (x,y,z = 0)
Physical Size
Animation
- Provide an “idle” animation to add life to the object
- Animations always loop
- Animations can be a mix of skeletal animation and transform animation
Animation Tips
- Choose animations that enhance AR immersiveness
- Don’t animate objects away from the origin
- Keep a consistent bounding box throughout an animation
- Prefer animations that make sense at a static location
- Or create animations as self-contained scenes
Contact Shadow
- AR Quick Look provides a contact shadow for you
- Can turn the shadow on / off
- Can apply ambient lighting conditions - Don’t bake a contact shadow into the model
- First animation frame is used for shadow creation
Model Appearance
Albedo (base color)
Metallic (conductor or insulator)
Roughness (rough or shiny)
Normal (surface details)
Ambient occlusion (internal shadows)
Emissive (emits light)
Transparency
- Use a separate material for transparent and non-transparent parts of the model
- Provide an albedo texture with transparency in its alpha channel
- Use transparency for see-through areas, not for cutouts
Texture Formats
!! Texture should be square powers of 2 (2048, 1024, 512 …) !!
- Albedo (RGB/RGBA)
- Metallic (Grayscale)
- Roughness (Grayscale)
- Normal (RGB)
- Ambient occlusion (Grayscale)
- Emissive (RGB)
Supporting All Devices
- System-wide extension that shares system memory with applications
- Optimize and test your models for high-memory devices
- iPhone 7+, iPhone 8+, iPhone X
- iPad Pro 12.9' - AR Quick Look will dynamically downsample textures for other devices when needed
- Meshes and animations are not modified
Size Limits
- Many factors affect a model’s memory requirements
- Mesh, animations complexity, Texture size & count - As a guide, for a model with a single PBR material
- 100k polygons
- One set of 2048 x 2048 PBR textures
- Ten seconds of animation - Always test your model on a real device
Optimizing Models
- Freeze transforms and merge adjacent vertices
- If possible, use a single material/texture set for the entire model
- Don’t include textures you don’t need
- Spend your texture budget on ares that add most value and realism
- Remember that pixels have a physical size in AR
- Balance texture size and quality against download size
‘usdz’ Converter
- Command line tool to convert 3D models to .usdz format
- Ships with Xcode 10
- Maps PBR textures to meshes and submeshes
- Input format
- OBJ file
- Single-frame Alembic (ABC) file
- USD file (.usda, .usdc)
Anatomy of a ‘usdz’ File
- .usdz file is an uncompressed zip archive
- First file in the archive is a .usdc file (model, animation, material)
- Remaining files (if needed) are image files
Sample
