Understanding ARKit Tracking and Detection

WWDC 2018

Orientation Tracking

• Tracks orientation only (3 DoF)
• Spherical virtual environment
• Augmentation of far objects
• Not suited for physical world augmentation from different views

World Tracking

Inertial Odometry

Visual Odometry

Visual Inertial Odometry

Visual Inertial Odometry

World Tracking API

World Tracking Quality

• Uninterrupted sensor data
• Textured environments
• Static scenes

Tracking State

World Tracking

• Tracks orientation and postion (6 DoF)
• Augmentation into your physical world
• World Map
• Guide user to achieve best tracking quality
• Runs on your device only
• Sample ( Building Your First AR Experience )

Plane Detection

Saving and Loading Maps

Saving and Loading World Map

• Acquire a good World Map
• Share the World Map
• Relocalized to World Map

World Map Data

• Internal tracking data
  - Map of 3D feature points
  - Local appearance
• List of named anchors
• Serializable

Acquiring a Good World Map

• Dense feature points on map
• Static environment
• Multiple points of view
• World mapping status

Guide the User

Share the World Map

Relocalize to World Map

• Before Relocalization
  - Tracking state is Limited with reason Relocalizing
  - World origin is first camera
• After Relocalization
  - Tracking state is Normal
  - World origin is initial world map
  - Only minor changes in environment allowed

Image Tracking

Localize the image

Dense tracking

Image Tracking API

Adding Images as Assets

• Create AR Resource Group
• Drag images to be detected
• Set physical dimension for images

Setting Physical Dimension for Images

• Physical image size must be known
• Allows content to be in physical dimension
• Consistent with world tracking data

Good Images to Track

• High texture
• High local constrast
• Well distributed histogram
• No repetitive structures

Reference Image Quality in Xcode

Reference Image Quality — Tips

Crop image to its core content

Use multiple AR Resource Groups

• Allow many more images to be detected
• Max 25 images per group reccomended
• Switch between groups programmatically

Image Tracking Configurations

Getting Results

Object Detection

How to Acquire an Object?

• Similar representation as a world map
• Use “Scanning and Detecting 3D Objects”
• Detection Quality affected by Scanning Quality

Object Scanning

Good Objects to Track

• Rigid objects
• Texture rich
• No Reflective
• No Transparent

Object Detection API

Get Results

Object Detction vs World Map Relocalization