Playing secure Akamai HLS streams

Jun 30, 2014 at 11:29 PM
I'm trying to play some "secured" HLS streams hosted by Akamai. I am able to initially authenticate, and I can point the player to the initial m3u8 file, e.g:

http://iviewhls-i.akamaihd.net/i/SMIL/kids_peppa_03_20_hi.smil/master.m3u8?hdnea=st=1404170005~exp=1404177205~acl=/*~hmac=06262b6819a7da4fa67fc34f6ded6ab63ec800d1c20cb59b2383ff009908c8a2

However attempts by the player to download the subsequent files referenced in this first document fail with a 403 error. I'm not positive this is the problem, but I noticed that the first response returned some Akamai cookies:

Set-Cookie: hdntl=exp=1404257006~acl=%2f*~data=hdntl~hmac=c02bbab1efcdce848768d15e279716ef63ea67ee78db477461223ed6f4542e32; path=/; domain=iviewhls-i.akamaihd.net;
Set-Cookie: _alid_=rQqLCPI3rnotb7ezdXYlUA==; path=/i//SMIL/kids_peppa_03_20_hi.smil/; domain=iviewhls-i.akamaihd.net

I think these need to be sent on subsequent requests, but the player framework isn't doing that.
I do however know that it is possible to play these streams as they work fine in the eval SDK from 3ivx.

Has anyone looked into playing these kinds of secured streams before?
Coordinator
Jul 1, 2014 at 11:03 AM
If you need cookies and you are using the HttpClient-based reader (the default), then the HttpClients constructor needs a non-null cookie container (note the "s" at the end of that class name; sorry about the poor class name). Cookies are disabled by default. SimulatedPlayer's Program.cs has an example of this usage:
    var cookies = new CookieContainer();

    var userAgent = HttpSettings.Parameters.UserAgentFactory("SimulatedPlayer", "1.0");

    using (var httpClients = new HttpClients(userAgent: userAgent, cookieContainer: cookies))
       ...
If that doesn't help, you might try setting up Fiddler and then playing the stream with both your own app and with an example that works. Usually there will be an obvious difference in the cookies or the referrer (the user agent or some kind of URL token might also cause trouble).
Jul 1, 2014 at 11:18 AM
Thanks Henric - that solved it!
Jul 2, 2014 at 12:12 PM
Actually, the above code worked fine with the Win8.1 framework, but the same code is not working properly on Windows Phone 8. It compiles fine, however the cookies are not being sent on subsequent requests. Is a different technique required for handling cookies on Windows Phone, or is it possible that there's a bug in the framework?

Bonus question: is it possible to use this in a Universal app that targets both Windows Phone and Windows 8.1? :-)

thanks again
Tom
Coordinator
Jul 2, 2014 at 1:52 PM
Where did you make the change for WP8? I think you need to make it in WP8's StreamingMediaPlugin.cs. Since this code predates the use of a DI container, the IHttpClients instance is still handled manually; it should be handled by the DI container. (I'll get to it...)

As for using it with a universal app, take a look at 835cf0683f86. See the HlsView.Win81, HlsView.WP81, and HlsView.WinRT.Shared projects for an example.
Jul 2, 2014 at 9:52 PM
Thanks Henric. I didn't make any changes to the code; I was using the approach demonstrated in the HlsView.WP8 project.
After researching it more I think the problem is due to a bug in Windows Phone as described here. It will probably be possible to work around this but it will be messy (not your problem though!).

I wasn't aware the Win8.1 version would work unchanged in WP8.1, but if it does then I'll give that a go as hopefully that same bug won't exist in the WinRT stack.

Tom
Coordinator
Jul 2, 2014 at 11:53 PM
If you need to, you can do the HTTP requests without going through HttpClient/HttpWebRequest/IXMLHTTPRequest2. I have a port of Mono's HTTP stack here: SM.Mono.Net. It runs on top of StreamSocket, so it bypasses the Microsoft HTTP stack entirely, including the cache brain damage, unmanaged memory consumption problems, and cookie issues. It is rough, hasn't been tested much, and I made some changes to make it TPL based and the locking strategy used by the Mono code is a bit difficult to follow so that my mucking could well have created problems. It shouldn't be too much work to adapt HttpWebRequestWebReaderManager to replace Microsoft's HttpWebRequest with the Mono port.

One might consider extracting the code that creates the HTTP headers and the code the parses the HTTP response and using those directly on top of a StreamSocket. The ServicePointManager adds a great deal of complexity and isn't really needed here (one could let the IWebReaderManager handle HTTP pipelining and the per-domain request limits aren't relevant for this application).
Jul 3, 2014 at 1:28 PM
I found another approach that is slightly less drastic, although it required changing the HttpClients code. I found an article here that provides a nice workaround to the WP8 bug. However this relies on a new HttpClientHandler to be used with the HttpClient. I made a change to the HttpClients code allowing a custom HttpClientHandler to be injected in lieu of the default (see code below). Is there any chance you could put this in the solution?

Also, I see that the code in the repository is diverging a fair bit from the 1.2.2 release - are you planning a new "official" release anytime soon?

thanks again!
public HttpClients(Uri referrer = null, ProductInfoHeaderValue userAgent = null, ICredentials credentials = null, CookieContainer cookieContainer = null, HttpClientHandler httpClientHandler = null)
        {
            _referrer = referrer;
            _userAgent = userAgent;
            _credentials = credentials;
            _cookieContainer = cookieContainer;
            _httpClientHandler = httpClientHandler;
        }

protected virtual HttpClientHandler CreateClientHandler()
        {
            var httpClientHandler = _httpClientHandler;
            if (httpClientHandler == null)
            {
                httpClientHandler = new HttpClientHandler
                                    {
                                        AutomaticDecompression = DecompressionMethods.GZip
                                    };
                if (null != _credentials)
                    httpClientHandler.Credentials = _credentials;

                if (null != _cookieContainer)
                    httpClientHandler.CookieContainer = _cookieContainer;
                else
                    httpClientHandler.UseCookies = false;
            }
Coordinator
Jul 3, 2014 at 2:52 PM
You may want to subclass HttpClients instead. It is registered as a singleton, so the above code will share the same handler for all HttpClient instances that HttpClients creates. I don't know what shared state there might be or what might happen when the handler instance is disposed. Alternately, one could pass a handler factory instead of a handler instance.

The code has changed quite a bit since 1.2.2, but I'm suspicious of one of those changes: a1400cfefa74. The change was needed, but I think I may have broken something in the process. For example, see the discussion here: Correct way of disposing the MediaElement. I spent two days trying to get it to blow up, but it never wanted to hang when I was looking for it. Since the startup/shutdown code is evil and that was about half the time I was expecting to spend replacing it ("flushing it down the toilet," may be more descriptive), I thought I'd tackle it by doing what I had been intending to do anyway, which is to rework how the pipeline is put together. That should have some other benefits as well, like handling #EXT-X-DISCONTINUITY tags and missing/partial segments. It would also make implementing bitrate switching simpler. Tearing my hair out trying to fix a bug in some overly complicated startup/shutdown code that I'm intending to replace anyway seemed like a poor use of my time. (Perhaps my scalp is impairing my judgment?)

Since then I haven't had any large, undistracted blocks of time, so I've been fixing a few minor things and finishing some changes related to the DI container (e.g., I'll be pushing some changes to remove GlobalPlatformServices and to let the DI container handle IHttpClients shortly).

I'll see about changing how the HttpClients instance is initialized. If the DI container would construct the HttpClientHandler instances for HttpClients, then I think your problem would be solved. At least, it would let you specify an HttpClientHandler subclass simply by registering it with the DI container.

I may also add a plugin for Player Framework on WP8.1/Win8.1. I'm playing with some code now and it doesn't look to be too complicated. The main drawback with the current attempt is that it doesn't actually work.

The first go at support for a universal app with Player Framework support might be worth a 1.3.1-alpha...
Coordinator
Jul 3, 2014 at 6:20 PM
tomhollander wrote:
I found another approach that is slightly less drastic, although it required changing the HttpClients code. I found an article here that provides a nice workaround to the WP8 bug. However this relies on a new HttpClientHandler to be used with the HttpClient. I made a change to the HttpClients code allowing a custom HttpClientHandler to be injected in lieu of the default (see code below). Is there any chance you could put this in the solution?
Could you see if 6f8537c25074 works for you? I didn't actually test overriding HttpClientHandler...

Thanks.
Jul 8, 2014 at 5:02 AM
Thanks Henric! At first I tried to follow your original suggestion of subclassing HttpClients. The technique seemed to work fine, however for some reason the 1.2.2 version of the framework refused to play the stream, and returned a MF_MEDIA_ENGINE_ERR_SRC_NOT_SUPPORTED error.

So I grabbed the latest codebase and injected a new factory as suggested:
 _mediaStreamFacade.SetParameter(() => new CookieEnabledHttpClientHandler(_cookieContainer, cookieCollection));
This seems to do the trick! I'll do a bit of testing but as far as I can tell your latest build is working fine, at least for the bits that I'm using in my app.

One other small thing I noticed was that when I finally got it working on the phone, the video was so choppy that it wasn't watchable. It turns out that the default position sampler interval of 75 milliseconds that you use in your samples is too short for low-powered devices. I changed that to 1000 at it works fine. You may want to change the samples to save others the hassle of troubleshooting this problem.

Tom
Coordinator
Jul 8, 2014 at 9:25 AM
You're welcome.

Thanks for letting me know that the HttpClientHandler override works.

Was there an HRESULT to go along with that MF_MEDIA_ENGINE_ERR_SRC_NOT_SUPPORTED? I scanned through the log since phonesm-1.2.2, but nothing jumped out at me as a likely reason for why the current code would resolve your problem.

What kind of device were you testing on when you saw the choppy video? What was the video's bitrate? Did you see the problem on both release and debug builds? Having problems from doing something less than 14 times per second (every 75ms) suggests that the "something" is taking far longer than expected. I'd like to make sure I understand what is going wrong before I change the sampling interval. Some bug may be chewing up CPU cycles or blocking on the UI thread.

BTW, if you prefer Player Framework, the changes I just pushed switches WP7 and WP8 to the simpler PF plugin used for WP8.1/Win8.1. The older plugin is still there, but not used by the "SamplePlayer" apps (renamed to MediaElementWrapperStreamingMediaPlugin). This was one of the changes I had in mind when I got rid of IMediaElementManager.
Jul 8, 2014 at 10:37 AM
Thanks Henric. Probably it's not so much how often the event was firing, but what I'm doing in the handler.

In my case I was updating the position of a slider control, even when it was hidden. I've changed the code so that it only updates the slider when it's visible (which should be rarely) and performance seems to have improved further.

I'll take a look at the Player Framework... presumably it's got some optimisations in this area.

Tom
Nov 3, 2015 at 9:07 AM
Edited Nov 3, 2015 at 11:46 AM
Hello Henric,

I have an issue. I need make to work Akamai HLS video and I have http tokens. I init cookie container but it gives me these errors :
MediaElement State: Opening
HttpClientWebReaderManager.DetectContentTypeAsync() url ext "http://areenahdworld-vh.akamaihd.net/i/world/8d/8d8e4ac9be3c26a53efa5e2f2226770f_,151552,360448,599040,1000448,.mp4.csmil/master.m3u8?hdnea=st=1446553156~exp=1446596356~acl=/i/world/8d/8d8e4ac9be3c26a53efa5e2f2226770f_*~hmac=234b0e355c5ce1b532a0cf29576dfae568641ad367b907dce1a93b2beaddb412" type M3U8 (application/vnd.apple.mpegurl)
The thread 0xfc has exited with code 0 (0x0).
PlaylistSegmentManager.ReadSubList(03/11/2015 14:42:51 +02:00)
'HlsView.WP81.exe' (CoreCLR: .): Loaded 'C:\windows\system32\SYSTEM.DIAGNOSTICS.DEBUG.NI.DLL'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
The thread 0x40c has exited with code 0 (0x0).
'HlsView.WP81.exe' (CoreCLR: .): Loaded 'C:\windows\system32\System.Net.Primitives.ni.DLL'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'HlsView.WP81.exe' (CoreCLR: .): Loaded 'C:\windows\system32\System.Net.Requests.ni.DLL'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Exception thrown: 'System.Net.Http.HttpRequestException' in System.Net.Http.Phone.ni.DLL
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
Exception thrown: 'System.Net.Http.HttpRequestException' in SM.Media.ni.DLL
The thread 0x10c has exited with code 0 (0x0).
The thread 0x4b4 has exited with code 0 (0x0).
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
Exception thrown: 'System.Net.Http.HttpRequestException' in mscorlib.ni.dll
PlaylistSegmentManager.ReadSubList() failed: Response status code does not indicate success: 403 (Forbidden).
PlaylistSegmentManager.ReadSubList(): retrying update in 00:00:05.0830000
PlaylistSegmentManager.StartAsync() no segments found

Part of the code:
 var userAgent = ApplicationInformation.CreateUserAgent();
        var cookies = new CookieContainer();

        _httpClientsParameters = new HttpClientsParameters {
            CookieContainer = cookies,
            UserAgent = userAgent
        };
What could be a problem?
Coordinator
Dec 12, 2015 at 12:56 PM
Did you figure this out? (An illness in the family has been keeping me from dealing with much of anything else.)

Are you sure the 403 is due to a missing cookie? Sometimes they care about other headers (e.g., "Referer:").

The general answer for these things is usually to take a look at the traffic with Fiddler. The HttpConnectionWebReader can be useful for this since it provides a simple way of hard-coding Fiddler as a proxy.