Introduction

Media is a big business in the Internet, and it is the highest bandwidth consuming content available in WWW. This post will give you detailed information about delivering media through the Internet, what are the concepts and technologies associated with that and how Azure Media Services (AMS) can be used to deliver media.

Before talking about what AMS is and how to use it in delivering media content through the HTTP, let me explain some basics of the media (here the topic of discussion is videos but the concepts are very analogous to audio as well).

Video formats are just containers for media content. It packages the video content, audio content and other extras like the poster, header and much more. There are many video container formats available, the *.mp4, *.ogv, *.flv and *.webm are few well known container formats.

CODECS are used to package the media content in those containers. The term CODEC is used to describe both the encoding and decoding. So for any playback the player should know two things.

• How to open / read the specific media container.
• How to decode the media content using the CODEC used to encode it.

H.264, Theora and VP8 are few well known video codecs.

I highly recommend this post in order to get more information about media containers and codecs.

The real world experience

Talking about video creation, it is very easy for all of us to create a video even with 4K quality using smartphones. We record the video using our devices, edit them and share it.

From that point onwards what happens to that content and how it is handled by the sites like YouTube is the core part of this discussion.

Before the introduction of HTML 5 video tag, we had to install a third party plugin (mostly the Flash Player) in order to view the videos. This is because before the HTML 5, browsers did not know how to perform the above mentioned two tasks to play a video. (They didn’t know how to open the container and they didn’t know how to decode the content).

When delivering media to highly diversified audience (different operating systems, different browsers and different devices) we should concentrate we should concentrate on the format (container and codec) of the media. Prior to HTML 5, we had no choice other than installing the plugins for our browsers.

These plugins were smart enough to do the following main tasks.

• They can read and decode the media
• They could detect the bandwidth variations and request the media fragments in the right quality which is known as adaptive streaming / adaptive bitrate streaming (ABR)

The second task is very important and that made the streaming a fruitful experience in low bandwidth conditions as well. I’m pretty sure everyone has experienced this in YouTube. (Don’t tell me that you haven’t used YouTube).

In order to provide this experience not only the client players (plugins) need special capabilities, the streaming servers should also encode the video in different qualities as well.

HTML 5

HTML 5 ripped off most plugins from the browsers. It also has the video tag which can play the video. Browsers do not need a third party plugin to playback videos / audios. Put your video files in a static directory in your web application and open it using the <video> tag. Things works perfectly well.

This will work perfectly fine as long as your browser supports *.mp4 / H.264 (all modern browsers do support this).

This is how browser handles the request.

You can notice, before hitting the play there are few requests to the media content in order to render the initial frame. But the content request is made after hitting the play (as auto play is not enabled in the above HTML snippet) and it is in the Pending state because the browser has made the request to the HTTP server (localhost in this case) and the response is not complete.

But your browser starts to play the video before the response is completed, because browsers are capable of rendering the content. When you seek the progress bar you have to wait till content to be loaded to that point from the start. Because your browser just send a HTTP request for the video file and it simply downloading it, the browser do not know anything about how to request based on varying bandwidth and how to request in the middle of the download. It is just a start to end download. This is known as Progressive Download.

Progressive download can give us the streaming experience though technically it is not streaming because the browsers know how to render / playback the content what is available to it.

HTML 5 video / audio tags do this. They render / playback the content.

In this case we do not have a client which can detect varying bandwidth. We cannot request video chunks (known as fragments) in different qualities since we have only one video file. In the progressive download you will get an uninterrupted video playback only if your bandwidth is higher than the bitrate of the video.

The above diagram shows the flow. It is very similar how you deliver any other static content like images and scripts.

So it is obvious using the static content delivery we do not get ABR which is very important in the modern world since we have multiple devices and varying bandwidth.

So we require a technology in HTML 5 that can perform the ABR streaming. This requires changes in both the client and streaming side. Part 2 of this series will explain how to do Adaptive HTTP Streaming and ABR play back in HTML 5 without plugins.