Measuring and Optimizing HLS Performance

WWDC 2018

Best User Experience for HLS

• Content authoring → playlist authoring, media formats, codecs
• Application design → using the AVFoundation API
• Delivery → server configuration and performance
• Understand and quantify user experience in conjunction with your changes
• Measure to find the optimal configuration

What Is This Session About?

• Establish a common language for streaming Quality of Service
• Objectively measure the streaming QoS
• Identify and solve problems that impair Qos
• Get master playlists right

Anatomy of an HLS Playback session

Quantifying the User Experience

Key Performance Indicator (KPIs) for HLS

Startup Time

How much time did users spend waiting for playback to start?

Stall Rate

• How often was playback interrupted?

• How long was playback interrupted?

AVPlayerItemAccessLog

Provides a history of the playback session

Time-Weighted Indicated Bitrate

What was the overall media quality?

Playback Failure Percentage

How often did the stream end due to an error?

AVPlayerItemErrorLog

Conveys failures with varying degrees of user impact

The User Experience

Key performance indicators for HLS

• KPIs are not always comparable between device models and usage scenarios
• Gather context information along with your streaming metrics
• Partition your playback sessions in classes that make sense for your application

Sample

• Using AVFoundation to Play and Persist HTTP Live Streams

What can Delay Start of playback?

How Do I Reduce Startup Time?

• Multiple time-consuming operations
• Measure time cost
  - Time spent between API calls and Player/ PlayerItem status change notifications
  - AVPlayerItemAccessLogEvent.startupTime 
  - Time from starting media download to the first isPlaybackLikelyToKeepUp == true

Reduce Startup Time — AVAsset

Reduce Startup Time — AVAsset

• Create and inspect AVAsset before user decides to play
• Prefetch decryption keys
  - Adopt AVContentKeySession!

Reduce Startup Time — AVPlayer and AVPlayerItem

• Set up AVPlayerItem before buffering
• Set AVPlayer rate before setting player item

Reduce Startup Time — AVPlayerItem Transition

• Use AVQueuePlayer to play multiple items
• Enqueue second AVPlayerItem well in advance

What Determines Network Buffering Time?

• Choice of variant
• Content bitrate
• Playlist target duration
• Network bandwidth

Network Buffering Time — Typical Case

Network Buffering Time — Download Slower Than 1x

Network Buffering Time — Initial Variant Choice

Reducing Network Buffering Time

• Wise choice of initial variant
  - Lower content bitrate means shorter buffering time for startup
  - But it’s a trade off of initial video quality 🤔
• Be mindful of all media format
  - Consider initial variant for each flavor of media format
  - HDR / SDR, HEVC / H.264, Stereo / DD / DD+

Why Does the Playback Stall ?

• Stalls can happen !
• How to investigate stalls
• How to avoid stalls

Stall Investigation

• Listen to stall notifications

• Check AVPlayerItem status

• AVPlayerItemErrorLog
• AVPlayerItemAccessLog

Reducing Stalls — Dealing With Variable Networks

Provide a full set of bit rate tiers

• Each codec combination needs its own set of tiers

Reducing Stalls — Dealing With Variable Networks

Your content server and CDN must

• Deliver media playlists, segments, and keys without delay
• Update live playlists at least every target duration
• Synchronize discontinuity sequence numbers between playlists
• Indicate server-side failure clearly

How to Investigate Errors

• Error and access logs
• Error properties
• Media validation

Streaming Errors — AVPlayerItemErrorLog

• Can be triggered by problems with the network or content format
• May be nonfatal
• Details are in the error comments

Streaming Errors — AVPlayer, AVPlayerItem

• Observe AVPlayerItem.status and AVPlayerItem.error
• AVPlayerItem errors end playback, remove item from player queue

Streaming Errors — HDCP

• Current item requires external protection
• Device does not meet protection level
• User observes video loss
• Have a variant which doesn’t require HDCP protection
• App UI should reflect property value changes

Streaming Error — Media Validation

• Media stream validator
• Available on developer website

Master PlayLists

How do I make sure the player chooses the right stream?

Attributes For Variants

EXT-X-STREAM-INF tag

Why don’t I see images in fast-forward or in the Apple TV scrubber bar ?

What’s an I-frame?

I-frame Video Segments

Consist of a single frame

High Density I-frames

Evenly spaced, closer together, better user experience

How do I support multiple languages?

What’s the difference between DEFAULT and AUTOSELECT ? 🤔

• AUTOSELECT
  - This rendition can be automatically selected
  - Generally you want this set to YES
  - Set to NO for things like a commentary track
• DEFAULT
  - Player chooses this rendition unless the user chooses something else
  - Only one member of a group can be DEFAULT = YES
  - Must have AUTOSELECT = YES
  - Has nothing to do with the default variant

How do I support multichannel audio?

How do I support several audio bit rates ?

How do I want HEVC or Dolby Vision ?

How do I handle subtitles ?