Why Netflix is about to get twice as good

As you probably know, Netflix is doing pretty well when it comes to the streaming content market. It recently passed the 93m subscriber mark, it’s got upcoming movies starring Brad Pitt and Will Smith in the works, and of course there are new series of House of Cards and Stranger Things on the way.

As a result, the company is set to spend US$6bn on content this year alone. But the company is also set to spend US$1bn on the science and technology of delivering all that content.

We met with Todd Yellin, Vice President of Product Innovation at Netflix, who explained that it’s all to do with keeping subscribers: the better the streaming experience, the more content people will watch, and the more likely they are to keep subscribing.

The first part of that initiative was to deliver Netflix’s physical servers to internet service providers themselves. This allows Netflix’s content to be as close to the end consumer as possible, meaning the content is delivered as quickly and efficiently as possible.

While Netflix holds a vast library of content, it can still hold it all on one of these server racks in ten different formats depending on the end user’s connection speed. (Netflix also asked us to put a famous movie quote on one of these server racks to give one of their engineers a chuckle when it makes it to an ISP.)

One of the challenges Netflix has is that more and more people are consuming content on their mobile.

Television viewing still accounts for two-thirds of Netflix content consumption right now, but there are three countries where more content is consumed on mobile than any other type of device: Japan, South Korea and India.

For that reason, Netflix is getting more and more serious about mobile. Currently, content is encoded at 10 different levels, and which version is served depends on the end user’s bandwidth. There’s also a setting within the Netflix app to limit the amount of bandwidth used, which is particularly useful for those people who have mobile data usage caps.

The next step, according to Yellin, is to make sure people get an even better experience when they’re viewing over a mobile network or a restricted Wi-Fi connection (such as at a hotel). The answer is more advanced encoding techniques, and adaptive encoding that can utilise less data for less detailed visual scenes.

The first part of the puzzle is just getting more efficient at encoding, meaning you can do more with the limited bandwidth that can often apply to users on mobile networks. As anyone who’s watched the HBO series Silicon Valley will know, compression algorithms are kind of the holy grail of multimedia delivery companies: the more you can compress, the less data you use. And as data costs money, both from the end user’s perspective and the internet service provider’s, everyone’s kept happy.

Yellin showed us a demo of Stranger Things on two identical mobile devices, and asked me to pick which one I thought was better. The one on the left was much worse – all blocky and full of artefacts, while the one on the right was a much cleaner picture. Yellin revealed that they were both being encoded at 100kbps (a very low bitrate), but that the one on the right was using a new and improved codec that will be released by Netflix in the coming months.

It’s difficult to tell from these images, but the version on the right is much cleaner. It’s based on Google’s open-source VP9 codec, which allows for a more detailed picture at the same bitrates used in previous encoding techniques.

However, VP9 also allows Netflix to be as efficient at lower bitrates as it currently is at higher ones. For example, we were shown two trailers for the upcoming Marvel’s Iron Fist, and were asked to spot the difference. This time it was much more difficult to tell – they both looked free of artefacts and we’d be happy watching them on-the-go.

The reveal was that one clip was encoded at 555kbps in the current codec, while the other was encoded at 277kbps in the new VP9 codec. The result is that Netflix is able to produce the same picture quality at half the bitrate.

To help the scientists behind the encoding, Netflix commissioned Université de Nantes in France and University of Southern California in the US to gather feedback from hundreds of people on where they could notice the compression. This data was then used to feed into the algorithms designed by their in-house scientists.

Another advancement made by Netflix recently has been adaptive compression. According to Yellin, the company used to encode everything in the same way, whether it was a real-world drama such as House of Cards, or a more visually basic animation like Bojack Horseman. They figured they should start encoding differently, because the blocks of colour and simple details in Bojack didn’t need the same level of information as the frequently darker scenes in Cards.

This technique didn’t work quite so well with another animation though, Barbie: Life in the Dreamhouse. What do you mean you’ve not seen it? Anyway, for the most part it was also made up of simple colours and details, but there was an incredibly detailed credits sequence in each episode that required a more intelligent approach. As a result, Netflix is intelligently and dynamically encoding each piece of content, giving more information over to scenes that need it, and less to scenes that don’t. The company is now even using machine learning based on the Nantes and SoCal feedback to compress shots no-one’s seen.

According to Yellin, it’s more difficult to compress things on the audio side, as the human ear is much more sensitive than the eye when it comes to compression. However, the current 56kpbs rate should be able to be compressed to 40kbps at some point in the future.

Netflix is also investing in HDR content, as HDR-enabled devices are coming out this year in the form of the LG G6 and the Sony Xperia XZ Premium. This will mean more content coming from Netflix in the form of its own shows, as well as certain shows being ‘regraded’ into HDR where the creators see fit. Stranger Things will be one show being reissued in HDR, so look out for that if you have a HDR-enabled phone or display. And Netflix CEO Reed Hastings has confirmed that both Dolby Vision and HDR 10 standards will be supported.

One thing that won’t be happening is Netflix’s arrival on Nintendo Switch. Yellin was keen to point out that the company has access to every major device before its release, such as the aforementioned phones. However, there’s no work being done on a Nintendo Switch app, which sadly is in keeping with what we’ve seen of the console’s complete focus on games at this stage.

For more on Netflix, be sure to check out our list of the best shows to watch on Netflix right now, and expect to see the new codec being rolled out in the coming months.