Tag Archives: ABR

Adaptive Bitrate (ABR) Playback in HTML 5 and Azure Media Services

Posted on by

I highly recommend to read these following posts before continuing on this.

Once your media is ready to be streamed we need some mechanism for the ABR playback in the browser without any plugins. Because HTML 5 video tag does not perform ABR and it just performs progressive download. (This is explained in the first link above).

MPEG DASH is the protocol / standard to stream ABR playback using HTML 5.

YouTube adapted this in January 2015.

Creating a DASH player is a custom implementation since W3C defines it in the MSE (Media Source Extensions) as a standard.

Dash.js – https://github.com/Dash-Industry-Forum/dash.js
is an open source project which helps to build the DASH player. The source contains samples ranging from simple streaming to complex samples like captioning and ad insertion.

Azure Media Player (AMP)

Azure Media Player is an implementation of a DASH player adhering to MSE and EME (Encrypted Media Extensions). AMP is also provides fallback mechanism to play the media using Flash / Silverlight when DASH is not supported by the browser. It has a limitation of it will only playback the media streamed from Azure Media Services, but for the intended purpose it serves very well.

Read documentation of AMP here. http://amp.azure.net/libs/amp/latest/docs/

You can check a sample online AMP here. http://amsplayer.azurewebsites.net/azuremediaplayer.html

Simple implementation of AMP using DASH without any fallback mechanisms would look very similar to the below code snippet.

Advertisement

Delivering media content using Azure Media Services – Part 2

Posted on by

Preparing the video for the HTTP Adaptive Streaming using AMS

I assume you have read the Part 1 of this series and you have the understanding about the HTTP Adaptive Streaming and why we need to support it in HTML 5.

HTTP adaptive streaming allows the browsers to request video fragments in varying quality based on the varying bandwidth. In order to serve video fragments in varying qualities, the media should be encoded in different qualities and kept ready to be streamed. For example if you have a video in 4K quality, you have to encode it in different qualities like 1080p, 720p and 480p likewise. This eventually will produce individual files in different qualities.

Note: Typically upscale encoding process do not produce quality videos, meaning that if your original video is in 2K quality and encoding it in lower qualities would produce best outputs but encoding it in the 4K quality would not produce clear 4K most of the times.

So in the process of encoding the video files in different qualities, the first step is uploading the source file. This is known as Ingestion in the media work flow.

Assuming that you have AMS up and running in you Azure account, the following code snippet would upload your video file to the AMS. When creating the AMS you have to create a new / associate an existing storage account with that. This storage account will be used to store the source files and the encoding output files (these files are known as assets).

First install the following AMS NuGet packages

var mediaServicesAccountName = “<your AMS account name>”;

var mediaServicesKey = “<AMS key>”;

var credentials = new
MediaServicesCredentials(mediaServicesAccountName, mediaServicesKey);

var context = new
CloudMediaContext(credentials);

var ingestAsset = context.Assets.CreateFromFile(“minions.mp4”, AssetCreationOptions.None);

After the upload we should encode the file with one of the available HTTP Adaptive Streaming encoding types. The encoding job will produce different files in different qualities for the same source. In the date of this writing AMS supports 1080p as the maximum ABR encoding quality.

Following lines of code submit the encoding job to the AMS Encoder. You can scale the number of encoders and the type based on your demand like any other services in Azure. When the encoding is completed AMS will put the output files in the storage account under the specified name (adaptive minions MP4).

var job = context.Jobs.CreateWithSingleTask(MediaProcessorNames.AzureMediaEncoder,


MediaEncoderTaskPresetStrings.H264AdaptiveBitrateMP4Set1080p, ingestAsset, “adaptive minions MP4”,


AssetCreationOptions.None);

job.Submit();

job = job.StartExecutionProgressTask(p =>

{


Console.WriteLine(p.GetOverallProgress());

},


CancellationToken.None).Result;

Once the job is finished, we can retrieve the encoded asset, apply polices to it and create locators. Locators could have conditions on the access permission and the availability. When we create locators, assets will be published.

var encodedAsset = job.OutputMediaAssets.First();

var policy = context.AssetDeliveryPolicies.Create(“Minion Policy”,


AssetDeliveryPolicyType.NoDynamicEncryption, AssetDeliveryProtocol.Dash, null);

encodedAsset.DeliveryPolicies.Add(policy);

context.Locators.Create(LocatorType.OnDemandOrigin, encodedAsset, AccessPermissions.Read, TimeSpan.FromDays(7));

var dashUri = encodedAsset.GetMpegDashUri();

We can retrieve the URLs for different streaming protocols like HLS, Smooth and DASH. Since our topic of discussion focuses on delivering HTTP adaptive streaming over HTML 5, we will focus on constructing the DASH URI for the asset.

This is the manifest URI, which should be used in HTTP adaptive streaming playback.

In the Part 1 of this series, I have explained what HTML 5 video tag does and the browser is not aware of requesting varying fragments based on varying bandwidths. In order to do this we need a browser which supports DASH.

There’re several implementations of the DASH player, we will be using Azure Media Player for the playback since it has fallbacks for the other technologies if DASH isn’t supported.

Note: AMS should be configured with at least one streaming end point in order to enable the HTTP Adaptive Streaming.

Delivering media content using Azure Media Services – Part 1

Posted on by

Introduction

Media is a big business in the Internet, and it is the highest bandwidth consuming content available in WWW. This post will give you detailed information about delivering media through the Internet, what are the concepts and technologies associated with that and how Azure Media Services (AMS) can be used to deliver media.

Before talking about what AMS is and how to use it in delivering media content through the HTTP, let me explain some basics of the media (here the topic of discussion is videos but the concepts are very analogous to audio as well).

Video formats are just containers for media content. It packages the video content, audio content and other extras like the poster, header and much more. There are many video container formats available, the *.mp4, *.ogv, *.flv and *.webm are few well known container formats.

CODECS are used to package the media content in those containers. The term CODEC is used to describe both the encoding and decoding. So for any playback the player should know two things.

  • How to open / read the specific media container.
  • How to decode the media content using the CODEC used to encode it.

H.264, Theora and VP8 are few well known video codecs.

I highly recommend this post in order to get more information about media containers and codecs.

The real world experience

Talking about video creation, it is very easy for all of us to create a video even with 4K quality using smartphones. We record the video using our devices, edit them and share it.

From that point onwards what happens to that content and how it is handled by the sites like YouTube is the core part of this discussion.

Before the introduction of HTML 5 video tag, we had to install a third party plugin (mostly the Flash Player) in order to view the videos. This is because before the HTML 5, browsers did not know how to perform the above mentioned two tasks to play a video. (They didn’t know how to open the container and they didn’t know how to decode the content).

When delivering media to highly diversified audience (different operating systems, different browsers and different devices) we should concentrate we should concentrate on the format (container and codec) of the media. Prior to HTML 5, we had no choice other than installing the plugins for our browsers.

These plugins were smart enough to do the following main tasks.

  • They can read and decode the media
  • They could detect the bandwidth variations and request the media fragments in the right quality which is known as adaptive streaming / adaptive bitrate streaming (ABR)

The second task is very important and that made the streaming a fruitful experience in low bandwidth conditions as well. I’m pretty sure everyone has experienced this in YouTube. (Don’t tell me that you haven’t used YouTube).

In order to provide this experience not only the client players (plugins) need special capabilities, the streaming servers should also encode the video in different qualities as well.

HTML 5

HTML 5 ripped off most plugins from the browsers. It also has the video tag which can play the video. Browsers do not need a third party plugin to playback videos / audios. Put your video files in a static directory in your web application and open it using the <video> tag. Things works perfectly well.

This will work perfectly fine as long as your browser supports *.mp4 / H.264 (all modern browsers do support this).

This is how browser handles the request.

You can notice, before hitting the play there are few requests to the media content in order to render the initial frame. But the content request is made after hitting the play (as auto play is not enabled in the above HTML snippet) and it is in the Pending state because the browser has made the request to the HTTP server (localhost in this case) and the response is not complete.

But your browser starts to play the video before the response is completed, because browsers are capable of rendering the content. When you seek the progress bar you have to wait till content to be loaded to that point from the start. Because your browser just send a HTTP request for the video file and it simply downloading it, the browser do not know anything about how to request based on varying bandwidth and how to request in the middle of the download. It is just a start to end download. This is known as Progressive Download.

Progressive download can give us the streaming experience though technically it is not streaming because the browsers know how to render / playback the content what is available to it.

HTML 5 video / audio tags do this. They render / playback the content.

In this case we do not have a client which can detect varying bandwidth. We cannot request video chunks (known as fragments) in different qualities since we have only one video file. In the progressive download you will get an uninterrupted video playback only if your bandwidth is higher than the bitrate of the video.

The above diagram shows the flow. It is very similar how you deliver any other static content like images and scripts.

So it is obvious using the static content delivery we do not get ABR which is very important in the modern world since we have multiple devices and varying bandwidth.

So we require a technology in HTML 5 that can perform the ABR streaming. This requires changes in both the client and streaming side. Part 2 of this series will explain how to do Adaptive HTTP Streaming and ABR play back in HTML 5 without plugins.