Making AI and machine learning pay

The blending of video production and distribution techniques with artificial intelligence (AI) and machine learning (ML) is now at the stage when it needs to move from an expensive experiment to a profitable mass market success story. But, how feasible is this and over what time frame can it be achieved?

AI and ML have been trialled in various parts of the content chain over the last few years and some applications have even been quite widely adopted. The problem, argues Jacques-Edouard Guillemot, SVP Insight at TV technology outfit Nagra, is that the industry is yet to establish a roadmap for the application of AI and ML. He does however see a strong focus on developing such a roadmap and once this is in place the potential is huge.

“People are really thinking about what it will take to properly use AI or ML,” he says. “They first need to work out the basics and to do this the questions that need to be asked are: Where is the data and which business processes do you want to enhance with this technology?”

It’s important to keep coming back to the fact that AI and ML are fed by historical and real-time data. The extent to which they can enhance business processes and increase revenue is reliant on accurate data being captured, processed and stored. Once this data is collected and backed up, AI and ML can do so much more than a whole team of human data analysts and, discounting the initial investment, much more cost effectively.

“What is great about this technology is that it tends to put one brain behind each customer,” explains Guillemot.

“Before, you had one person who was an expert in redemption or marketing acquisition, for example, and this person was using his or her knowledge and experience to try and increase redemption or acquisition. But the segments used in traditional marketing are not very relevant now because they are not precise enough. AI allows us to model the thinking of an expert and also to use the data as a basis of reference and then to really take the best decision behind every single subscriber in terms of pricing, packaging and redemption.”

Traditional marketing siloes users into groups, such as mothers, single men, and pensioners. AI and ML disbands the groups and concentrates on individuals, which allows for much more targeted marketing.

Using AI to reduce churn

Reducing and minimising churn is a crucial part of any content network’s business model and, as already alluded to, AI and ML can play a big role in this. If you can employ AI and ML to better direct users to content they want to watch and subsequently enjoy watching, you will reduce churn by default, says Marcus Bergstrom, CEO of Vionlabs.

“There are a couple of key performance indicators that we have seen that lead to churn. The first is time to play. The time from when you enter a TV or OTT service to when you actually find something to consume, we are now seeing this to be between 30% to 40% of the total time we spend on a platform and that is wasted time and that leads to churn,” he says.

“The second factor leading to churn, is the quality of what you decide to consume. If it’s not a worthwhile use of your time, giving the wrong recommendation is equally as bad as not giving the consumer anything to watch at all.”

Churn is a KPI often included in financial results and so if AI and ML are proven to increase customer retention and growth, businesses are going to be keen to invest in this area.

Charles Dawes, senior director international marketing at TiVo, explains how his company is already using AI and ML to strengthen its customer base. “At TiVo, we realised that we could analyse a wide data set of how consumers were interacting with their digital television service to be able to identify behaviours that are indicators of churn,” he says.

“We are able to relay these results back to our customers who can then surface more individualised content to a user and use it to power marketing touch points – such as personalised emails. This is all powered by ML, as it allows us to process these individual actions at scale.”

Understanding user behaviour

Marketeers across all sectors have been trying to second-guess customer behaviour since the dawn of capitalism. AI and ML are the magic wands they have always dreamed of. It’s just a case of deciding which areas to invest the money in and wave them on.

“AI brings a hyper-scalability to understanding content,” explains Bergstrom at Vionlabs. “It allows us as technicians in the media world to see patterns that help us understand our viewers and our content in a way that wasn’t possible before, because the data was too much to dig into. And, that enables personalisation and discovery beyond what we see today or have seen in the past.”

Jacques-Edouard Guillemot

Embedding AI and ML into data processing requires a hefty investment, and consequently must guarantee an even heftier return. Guillemot at Nagra sees two “bottom-line generators” resulting from this AI and ML personalisation. “The first is around subscriber management, and what we see with working with our customers is that the impact here can be very big. And, the second is around content,” he says.

“Content costs a lot of money and to precisely know which type of content you want and which you don’t want and which you can sell to your audience and which is not very relevant is going to be very valuable.”

TiVo has already stepped up its use of AI and ML to further improve its content discovery and recommendation, says Dawes.

“As the industry moves to greater and greater levels of personalisation the methods and techniques applied have become both more nuanced and more complicated. Many of the initial forays into recommendation were quite simple cases of collaborative filtering – i.e. matching one viewer’s set of views or likes against another to come up with common items,” he says.

“At TiVo we’ve taken the next step by introducing predictions in addition to recommendations. Where recommendations can be used to expand someone’s content horizons and might be applicable for a Saturday night movie session, at other points we need to get them to the right piece of content for that moment. In addition, we’ve taken a ML-enabled approach to generating a meta data Knowledge Graph for the universe of video entertainment, which continuously generates insights into the relationships between different content. In testing with our consumers and business customers, we’ve seen increased viewership and reduction in churn as a result of deploying these new types of meta data to catalogues and increasing effective catalogue size.”

The backlash that has befallen smart-listening devices shows that there is a fine line between analysing and providing users with the content they want and raising the hackles of users, because they are spooked by how personalised the content they recieve is.

Vionlabs’ Bergstrom says the key here is only to give users what they ask for directly. “If you are in a video service and the service is making accurate recommendations allowing you to have a better experience, I think that that is something that we can accept,” he says.

“When you’re using a platform and all of a sudden the discussion point that you had with your friend on Saturday night is being promoted, that is intrusive.”

Monetising live feeds

Distributing live feeds is expensive and although live events may attract huge audiences, there is often little demand for the content after that first viewing. Dominik Michaelis, associate director at Boston Consulting Group (BCG), predicts that AI is going to be a key asset for better monetising live content, something that will welcomed by both the OTT and traditional broadcasting worlds.

“AI can play a big role when it comes to preparing and tailoring a VOD offering based on live feeds like sports events,” he says. “What we are seeing is broadcasters are automating the metadata tagging of live. They use AI to read the live feed and in real time produce the metadata that describes what is going on in that live feed. Based on this metadata, AI can help to identify specific content that can be automatically tailored, bundled and pushed to specific user groups or segments. That’s a very strong opportunity to really optimise live content monetisation and increase customer engagement.”

As well as looking at the overall picture, AI and ML will also increasingly be employed to target very specific content in both live and recorded feeds. Using AI and ML to mine content is a focus for video compression specialist Bitmovin’s, says Sean McCarthy, the company’s senior technical product marketing manager. “Our biggest challenge is syphoning through the hype, finding the real business applications and then only investing in those really enabling technologies,” he says.

“We are heavily invested in encoding in general and anything that we can do during the encoding process as a feature and right now that’s object detection. We’ve seen specific requests for object detection, so we released an object detection feature able to recognise certain objects within our encoders during the encoding process and you can use that image indexing or video indexing for a lot of different things.”

Whether it’s picking out car number plates that require blurring for a news programme or tagging a particular brand of trainers for advertising in a movie, object detection has huge potential for both content and advertising.

Hyper-targeted advertising

Object detection is not the only way the advertising industry is looking at using AI and ML. One of the biggest opportunities for broadcasters to apply AI is with cross-inventory campaign optimisation in a total video approach, says Michaelis at BCG.

“This will mean that advertisers can communicate their campaign specifics, such as target group, time horizon and budget to a broadcaster, and the broadcaster can use AI to best allocate the budget across the various outlets – linear inventory, addressable TV inventory, and in-stream inventory. With the ability to act on net reach across these inventory classes, broadcasters will be able to optimise campaign ROI using AI across the inventory classes in real-time,” says Michaelis.

“Some trials of this are happening now, mostly in continental Europe, but I think the proper go-to-market for this will be in around 18 to 24 months.”

Charles Dawes

By ingesting multiple data points in real-time, AI and ML can deliver ads that speak directly to the user and TiVo is already implementing this. “TiVo has brought to market a unique ad unit within its personalised discovery experience that allows content creators to advertise their content to an already engaged audience, as part of a recommendation carousel,” says Dawes. “This product called ‘Sponsored Discovery’ has seen average uplifts greater than 150% for the content that is featured in this unit.”

Employing AI and ML is expected to be particularly effective for the growing ad-based video on demand (AVOD) OTT industry too. “A personalised linear channel exposing hyper-personalised attributes to the programmatic ad market would be extremely powerful,” says Bergstrom at Vionlabs.

Using AI for QoS

AI and ML technologies are being used to improve the QoS of content too. “Content-aware encoding is gaining significant attention, as it can differentiate between different kinds of broadcast content, either static ‘talking-head’ style interviews or more dynamic sports images, to ensure that the quality, latency, and bitrates of each piece of content is properly optimised,” saysMiguel Serrano, vice president of cloud at Haivision.

Bitmovin’s McCarthy adds: “Using AI or ML in context-aware encoding you can use your clients’ analytics to understand either QoS streaming conditions, so what your rebuffed rates are for a given bitrate of a title, or to understand what your codex support looks like across devices and across clients and then inform your encoder in real time during a live stream to either spin up and encode a new codec, or to change the bitrate ladder to optimise the QoS performance. I think that this is probably the opportunity with the most immediate application.”

McCarthy believes that as devices and screen sizes get bigger there will be more demand for upsampling or user resolution “and that’s another interesting space that we think is going to see more AI and ML applications. Here you upsample video and there are two ways of doing this: temporal and spacial upsampling”, he says.

“Spatial upsampling essentially increases the resolution of a video, so having it go from SD to HD or upsampling an old film to 4K or being able to zoom in on an image while maintaining quality. There was a traditional way of doing that which relied upon bicubic mathematics, so it’s basically one mathematical formula that applies to all video, but with ML you can have this neural network that you feed a lot of data into and it constantly trains a model. Over time it will learn how to upsample in a different way to bicubic, because it will begin to understand the content of the video, very similar to a per title algorithm, that’s something that we’re actively researching.”

Bandwidth boosting

By automating smart real-time decisions using AI, it is also possible to optimise bitrates to free-up valuable bandwidth without sacrificing video quality. “This is exciting for a number of reasons.Chief among these reasons is the ability to maintain quality while reducing bandwidth use, and, therefore, cost – an important tool as the demand for content, especially high-quality content like 4K, continues to grow,” explains Serrano at Haivision.

“This is also exciting as unlike many other technological innovations, AI is not replacing anything, but simply adding something. This is an optimisation that won’t take away any human intervention, but rather create new opportunities for broadcasters and publishers looking to optimise their content.Another way in which AI will have an impact is in routing. As more broadcasters begin their shift to the cloud, routing streams through the cloud becomes a more important challenge. With AI, we can optimise not only the bitrate of a video stream, but also the route it takes from camera to production to the viewers, minimising latency and increasing predictability along the way.”

Used in conjunction with accurate data, AI and ML will dramatically improve workflows, enhance content and advertising streams. In a crowded market, customers will be looking for personalised content that they find quickly and enjoy at the best price. If the ads aren’t too intrusive they will likely accept these too.

Now it’s a case of working out the balance between how much must be invested in this new smart tech and where to focus it to maximise the financial returns. This formula is still being worked on. Those that perfect it first are set to strike gold and if you’re looking for a time frame, it will likely mirror the speed of mass adoption of electric cars.