The big squeeze

The proliferation of video-capable devices and streaming services, combined with the advent of UHD TV means that the pressure is on to find more efficient ways of forcing video down bandwidth-restricted pipes. However, with multiple video compression formats emerging to challenge existing dominant technologies, is the scene set for a period of confusion and fragmentation? Stuart Thomson investigates.

It is a truth universally acknowledged that a video service provider in possession of an expanding customer base must be in want of a more efficient way to deliver content.

No matter how rapid the expansion of available bandwidth, the volume of video traffic will expand to fill it. The dramatic and ongoing growth in IP video consumption, and ever-growing expectations among consumers about the quality of that video, therefore means that advances in video compression remain a key objective for the video industry.

It is not only about pure volume. The rapid increase in non-linear and mobile video consumption, combined with the advent of 4K UHD TV, adds to the pressure on the internet and adds costs to traditional broadcast delivery respectively.

Most industry experts also believe that broadcast will account for an ever-declining proportion of global video delivery in the future, meaning that bandwidth-saving technology for unicast delivery will be even more in demand.

The most commonly used compression standard for digital video distribution applications, AVC/H.264, has now been around for a long time. Ongoing improvements to the technology – beyond what was expected when the standard was first created – have given AVC a longevity that would have surprised its creators.

The putative replacement standard for H.264 is HEVC/H.265, which was standardised as long ago as 2012 – six years ago. While millions of HEVC-capable devices such as set-top boxes have been deployed, the standard has not been as widely adopted by broadcasters and service providers as some expected, primarily due to industry concerns about costly licensing terms. While some of the issues around licensing have now been resolved – notably the fee for streaming applications was recently cancelled – the presence of three separate licensing bodies has contributed to this confusion and uncertainty, which remains a fact of life for those looking to adopt the technology.

“We had expected to see more activity in the pay TV space, but cable operators have not had the bandwidth to simulcast [using different formats] and IPTV operators have not jumped on it,” says Richard Peske, vice-president, video compression and advertising products at technology company Arris, which in the compression space specialises in live video applications.

While HEVC-capable set-tops have been available for the last few years – including boxes from Arris – Peske says that few IPTV operators have made the switch. For cable, he says, upgrading to HEVC would likely be an accompaniment to launching an IPTV service once they have access to sufficient DOCSIS bandwidth as part of the longer-term shift away from DVB-C – something that has not happened yet.

However, the main reason that HEVC has been of relatively limited use so far is the licensing issue, he says.

“The patent pools confusion – with three different pools and other [companies] not in any – have not helped the situation, but things are slowly getting cleared up,” says Peske, who cites Apple’s endorsement of HEVC as a possible catalyst for change.

Potential challenge

Other codecs have now emerged that could potentially challenge the primacy of HEVC for next-generation video applications. The key technology in this respect is AV1, an open-source codec developed by the Alliance for Open Media (AOM), a group of powerful companies that includes Google, Mozilla, Netflix, Microsoft, Amazon and Apple.

Many claims have been made on behalf of AV1, and the technology has generated considerable buzz, particularly since the AOM’s release of the AV1 bitstream specification at the end of March along with a reference software-based encoder and decoder. However, there are major issues that are likely to hinder adoption – particularly the processing power required on the encoder side compared with HEVC.

Arris’s Peske says that his company has not seen much interest in AV1 to date for live broadcasting, though he concedes that there is “more interest in VOD”. AV1, he says, is “so much more computationally intensive” – in other words requires much greater processing horse-power – than HEVC that it only makes sense for file-based encoding. He adds that one of the supposed key advantages of the technology over HEVC – that it is royalty-free – is less important than is sometimes claimed because of the need on the part of users for indemnification against possible infringement of patents that have not yet come to light.

If its supporters are to be believed, however, AV1 should certainly be considered a viable alternative to HEVC. The AOM claims bitrate savings of 30% compared with the open-source x265 implementation of HEVC for file transcoding.

Thierry Fautier, Harmonic

Some industry players nevertheless challenge these claims. Technology provider Harmonic earlier this year teamed up with the Institute of Research and Technology B<>COM to test AV1 and found that HEVC still outperforms AV1 on subjective testing. “In summary, compression efficiency currently seems to be equal between the two codecs for file-based applications,” says Thierry Fautier (left), VP of video strategy at Harmonic.

Tony Jones, Ericsson

Tony Jones (right), principal technologist, media solutions at Ericsson, is of a similar opinion, arguing that AV1 “needs to be tested more rigorously” and that these tests need to be “like-for-like”. Based on preliminary analysis, he says bit-rate efficiency seems to be “pretty similar” to HEVC, while AV1 requires considerably more processing power.

Jones is sceptical about AV1 winning widespread adoption. “AV1 is obviously being positioned as a rival to HEVC. If the market wanted to move to it, based on the fact that its performance was dramatically different – which is contrary to what we expect – we would go with that,” he says. However, based on findings so far, it seems that “AV1 has yet to be optimised and that it has a long way to go” before it can seriously rival HEVC.

This view is echoed by Peske. While he believes AVC will have many more years to run as the prime distribution technology, the next step is likely to be HEVC rather than AV1. “As we’re focused on live video, we are betting on HEVC,” he says. Peske says that HEVC is unlikely to deliver huge improvements on what it can already do in terms of bandwidth optimisation going forwards, but suggests that the more widespread use of adaptive bit-rate encoding will have an important role to play in how compression technology is deployed. He suggests there could be ‘mixed ladders’ of encoding in the future, with HEVC deployed for higher-resolution video and AVC used for lower-resolution versions of the same content.

While established technology players such as Ericsson and Harmonic remain convinced that MPEG standards such as HEVC will stay at the core of video delivery for traditional broadcast and pay TV delivery as well as much of online video, this doesn’t mean that AV1 will have a nugatory impact on the market. On the contrary, the availability of an alternative means that the HEVC community is under more pressure to get its act together – meaning that one of the biggest impacts of AV1 could be to boost HEVC.

“HEVC is great technology that has only been hampered by the licensing terms,” says Fautier. However, he adds, the presence of AV1 as a potential alternative has already concentrated minds, pointing to the decision of HEVC Advance, one of the technology’s three licensing bodies, to cancel the fee for streaming applications. This impact, combined with the fact that HEVC devices are already widely penetrated, means that HEVC has a strong future, he says.

“In terms of commercial deployments, for VOD OTT, all members of AOM have indicated that they will deploy the codec as soon as devices are available, which is not expected to occur until after 2020. By this time, the number of HEVC-only capable devices is expected to be more than 10 billion. In other words, the industry will still need HEVC for many more years,” he says.

Hardware availability

Christopher Mueller, Bitmovin

Not everyone agrees that HEVC will dominate video distribution in the future, however. Christopher Mueller (right), chief technology officer and co-founder at Bitmovin, a company that has been at the forefront of AV1 and a key player within the AOM, claims that the 50% improvement from AVC to AV1 and licensing uncertainty about HEVC will play in AV1’s favour. Regarding uncertainty about patent claims on AV1, he said that the “large number of companies” supporting the technology will protect those adopting it from the claims of smaller technology players.

Mueller says that AV1 will be available on hardware from 2020, which will kickstart wider adoption, and points out that the support for the codec of both Google and Apple should not be discounted. The technology is a significant improvement on its predecessor, the VP9 codec used by YouTube, adding DRM encryption for example. He says that the key industry factor that will drive adoption is the need to fit more video into the same amount of bandwidth. He dismisses the objection that AV1 is massively more computationally intensive than HEVC, arguing that for larger players in particular, the bandwidth savings more than offset the computational effort. Mueller also says that Bitmovin’s AV1 implementation reduces the processing power differential with HEVC to a factor of 10 rather than a factor of thousands.

For all these reasons, Mueller believes that AV1 will gain a significant market share. “In the broadcast space HEVC will still be the choice but in the OTT space it will be AV1,” he says, adding that, in his view, this gives a longer term advantage to the newer codec. “In the future linear TV will more or less go away – the younger generation is not watching linear TV anymore.”

The last point is moot. The view that broadcast is on the way out is certainly challenged by Ericsson’s Jones. “Anytime you switch to unicast there is a cost,” he says. “The idea that there is a wholesale drive towards internet-style delivery is naïve. There will be pressure to do things a bit more sensibly.”

For now, at any rate, broadcast remains a highly important part of video delivery – and this favours HEVC. “So far AV1 has only shown capability for OTT, not broadcast, while HEVC can address both,” he says, adding that HEVC is currently present in over two billion devices globally – an addressable market that is a powerful argument for the primacy of the technology.

Broadcast and managed delivery is one thing. Harmonic’s Fautier admits that AV1 has an advantage on the open web. “Interoperability with the internet ecosystem, including web browsers and Android/iOS stores for connected devices, is critical. HEVC only works natively with Internet Explorer and Safari while AV1 should be compatible with 100% of the web browsers, as all founders are AOM members,” he says. On the other hand, he adds, when it comes to mobile apps, HEVC is supported in both Android and iOS ecosystems.

Future technologies

HEVC and AV1 are not the only next-generation compression technologies available to video distribution companies. The rate of innovation in compression is, if anything, accelerating. The latest MPEG standard and potential HEVC replacement, JEM/H.266, is now on the agenda, the standardisation process having kicked off in October last year.

The latest MPEG meeting in San Diego in April was scheduled to discuss which tools to be included in the standard and, according to Harmonic’s Fautier, the industry is on track to achieve a 50% bit-rate improvement compared with HEVC by 2020, something he describes as “a real breakthrough in terms of compression efficiency, especially as it will not only improve on HEVC but also AV1”. The AOM meanwhile, is already working on a new version of its codec, AV2.

While H.266 promises great things, its development could be stymied by the same licensing disputes that have dogged HEVC and, before it, AVC. To avoid this, says Fautier, the video community is discussing the creation of a group outside of the key standards body, ISO/ITU-T, that will “facilitate the selection of technologies to be deployed commercially using H.266 technology”. This group, he says, should address licensing issues “at the front end of the standardisation rather than at the back end”.

Public broadcasters organisation the European Broadcasting Union (EBU) commissioned the University of Applied Sciences of Wiesbaden to carry out a subjective evaluation of the competing video compression technologies in January. The evaluation included a comparison of the speed of encoding and decoding and was designed help EBU members choose the best codec for their individual needs.

According to Hans Hoffmann, EBU technology and innovation, senior manager media fundamentals and production, the decision to assess rival codecs to HEVC came after it was clear that licensing issues were holding up adoption, even though HEVC is “a fantastic codec”.

“The appearance of AV1 is interesting since the players involved have proclaimed it to be ‘royalty-free’, and if this is true and it meets certain performance requirements it could be interesting,” says Hoffman. He adds that, at the very least, the possibility that broadcasters could use an alternative technology could “shake up the market”, the inference being that HEVC licensing bodies would be more likely to think again – something that appears to be happening. Hoffman said that initial tests last year were done at a time when the bitstream version of the codec was not yet frozen. “They are now a step further along and we take further steps ourselves. The interest is still there, but it is too early to say what the results will be,” he says.

The EBU is also focusing on whether AV1 can realistically be used for real-time encoding applications – something that is obviously crucial for the organisation’s broadcaster membership.

“We don’t know yet if it could be used for broadcast,” he says. While the codec is computationally more intensive that HEVC, this could change over a relatively short time as a result of further innovation.

Hoffman says that the EBU is looking at HEVC, AV1 and JEM/H.266 together to have “a state-of-the-art comparison” and to work out “if it is worth spending resources to study them further”. He is hopeful the organisation will be able to publish a paper and possibly conduct a demonstration at this year’s IBC.

Arguing about which next-generation codec to adopt is academic if there is no compelling commercial reason to choose any of them.

Arris’s ME-7000: the company believes that AVC will dominate encoding for some time

Key to any technological shift in video is a killer application that drives innovation. In the case of the existing, widely-used H.264/AVC technology, that application was HD video. The need to fit more bits into existing pipes, combined with the need to upgrade set-top boxes and other equipment to enable HD signals to be delivered, meant that the industry had every incentive to adopt the new technology to replace the less efficient MPEG-2 standard. For HEVC, the homologous application that could drive change is 4K UHD TV. However, demand – and supply – for UHD video has been relatively slow to take off so far.

All new video formats take time to gain the attention of consumers, of course. But in the case of UHD TV, there has been uncertainty about the advantage to consumers in terms of the viewing experience, and uncertainty about the commitment of broadcasters and pay TV providers that are already facing commercial pressure on a number of fronts.

There is now a widespread consensus that the killer ingredient for UHD is HDR rather than higher resolution on its own, but arguments about which HDR formats to support and scepticism about the need to deliver higher resolution video in the first place, given the average size of TVs and the viewing habits of consumers, have contributed to slower than expected deployment of services.

Nevertheless, says Ericsson’s Jones, there is now a consensus that “there will be no UHD services without HDR and no HDR services without UHD”. He points out that consumer adoption of UHD and HDR-capable TVs is rising and that this will have an impact on expectations about the viewing experience. Moreover, he says, broadcasters will be forced to offer UHD and HDR services because they see that their online competitors are already offering these features.

For Jones, UHD is the key application that can help drive adoption of HEVC. Further down the line, he says, other applications will drive upgrades to H.266, which he does not expect to be ready for mass-market adoption for another five to eight years. “UHD TV will be established by them and that opportunity will have gone, so you have to look at mobility. Everyone is talking about that all the time and at some point [mobile bandwidth] will become such a significant problem that something has to be done – H.266 could be a solution,” he says.

Beyond mobility, 8K and wider adoption of virtual reality could serve as drivers for adoption of H.266.

While UHD TV will help drive demand for new compression technologies that can help operators save bandwidth, the video distribution world is much more complex now than it was when it shifted from MPEG-2 to AVC, which may have an impact on the speed with which new technology will be adopted, and will certainly have an impact on the complexity of the compression landscape.

Video is now distributed and consumed in many more formats than ever was the case previously, with growth consumption on small mobile screens outpacing other forms of video at the same time as the industry experiments with ever higher resolutions aimed at super-large TV screens.

Add to that the switch from linear to non-linear viewing, driven by SVOD and time-shifted TV – and the accompanying shift to unicast distribution and adaptive bit-rate encoding – and the result is a recipe for lots of different enabling technologies to gain market share simultaneously.

“The emergence of AV1 makes a multi-codec world a reality. As soon as AV1 devices will be available, operators such as Netflix, YouTube and Amazon will have a choice about which codec to use. Netflix has been very vocal about the fact that AV1 would be its codec of choice.

This situation is good for the industry, as AV1 will offer an alternative to HEVC. On the economics side, operators still need to address the HEVC installed base, which will increase encoding and storage costs,” says Fautier. “It is fairly unlikely there will be a single winner, as the HEVC installed base will be huge in 2020 when AV1 will only start to kick in. AV1 is being used for unicast delivery, but it will not work for broadcast where only one stream can be sent. In the latter case, HEVC is the only option.”

In other words, there is probably a market for all compression technologies, even if some are more equal than others.

In Focus: Making the most of AVC

While a lot of attention has been focused on the emergence of multiple new compression formats and on the licensing battles surrounding HEVC, AVC/H.264 remains the most widely used standard technology for compression in advanced markets.

With this in mind, technology companies have looked at ways to further optimise AVC, which could obviate the need for an immediate upgrade to HEVC or AV1.

Guido Meardi, V-Nova

Video compression specialist V-Nova developed Perseus Plus, a version of its Perseus codec that CEO Guido Meardi (left) says can be deployed without the need for new hardware acceleration.

“One of our core assumptions is that with 10 billion video-capable devices in the world and turnaround cycles [slowing], it is going to be difficult to wait, and it is costly to serve different markets with different codecs,” he says.

While a Netflix could afford to deploy services using different codecs economically because of its huge scale, for small and medium-sized players this is all but impossible, says Meardi.

“We need a next-generation codec that can lift up the quality of video on every device and web browser with 100% coverage,” he says.

Hence the launch of Perseus Plus, which Meardi says distributors can combine with an existing baseline codec – in this case AVC/H.264 – and use existing hardware decoding blocks to decode Perseus Plus. “You can decode this in hardware and have the fallback that if the decoder doesn’t know Perseus Plus it can just ignore that element of the stream and still decode the baseline codec,” he says.
According to Meardi, Perseus Plus can deliver 1080p HD video at less than 1Mbps with less processing power than AVC.

“By enabling higher quality at lower bit-rates we expand the time of viewing and lower the cost of delivery in terms of marginal CDN cost per minute,” he says.

Meardi says that V-Nova has already deployed Perseus Plus with Sky in Italy on 12 year-old set-tops, as well as in a number of other deployments around the world. The company has now developed a second, updated version of the technology, which it debuted at the NAB Show.

Meardi says that Perseus Plus is one of a number of ways in which operators can enhance the capability of existing infrastructure, and claims that it provides “the least expensive bang for your buck” in terms of delivering bandwidth improvements on AVC at multiple bit-rates.
Other companies have also developed technologies that could broadly be perceived as ways of extending the life of AVC, enabling the delivery of more and higher-resolution video services to existing devices. Harmonic’s EyeQ ‘content-aware’ encoding format is also designed to work using a standard AVC codec and is marketed by the company as an optional enhancement for its software-based PURE encoder platform. The technology can be used to optimise variable bit-rate AVC. Harmonic claims that the technology can deliver bandwidth savings of up to 50% on standard AVC.

V-Nova’s Meardi adds that other technologies such as intelligent caching are also important in delivering bandwidth improvements. “Codecs are important but there are other elements that make a car work aside from the engine,” he says.

For Meardi, next-generation codecs are most likely to be deployed for a few specific use cases where the service provider has full control of the decoder, but for the majority of mass-market applications, next-generation technologies are unlikely to gain much traction for at least five years.

“Even though HEVC was standardised in 2012, as of today we still have loads of non-HEVC-enabled devices that are likely to stay in the ecosystem,” he says.

With economics driving a slowing down of the frequency with which people swap or upgrade devices, the trend towards a multi-codec world will be even more pronounced.

“The mobile [device] upgrade cycle is now becoming more like the upgrade cycle for TVs,” he says. “I don’t believe we will see AVC disappearing any time soon.”