The evolution of the second screen app space

Zeebox app on an Android tablet.

The ‘second screen’ app space has grown up in recent years, but how are these services complementing the TV experience and what comes next? Andy McDonald reports.

The second screen has come of age in the last few years – not only as a way to consume content but increasingly as a means of discovering, engaging and interacting with it. A revolution in the OTT space with the emergence of players including Netflix, Hulu, and Amazon’s Lovefilm has changed how people consume content – and on what devices. Meanwhile, initiatives including TV Everywhere in the US and similar operator app deployments around the world, along with a wave of broadcaster catch-up services – for example the BBC’s much-praised iPlayer– have firmly established in the mind of the public that ‘TV’ is no longer a viewing experience that is confined to the living room.

Yet when it comes to social, interactive and other value-added services – be it a programme app for a specific TV show or a broader TV check-in service – the second screen is still in its infancy. While a first wave of US social TV check-in services, including GetGlue and Miso, helped to establish this space a couple of years ago, broadcasters, programme makers, platform operators, social networks and tech firms are all now jostling to take a piece of this market and to move it in new directions.

TV app specialist Accedo makes Xbox, smart TV and TV everywhere apps for clients that include Australian pay TV operator Foxtel and Viasat-owned OTT service Viaplay. CEO Michael Lantz sees a clear division line in the market for multiscreen services. “At Accedo, we separate the definitions of TV Everywhere apps and second screen apps. TV Everywhere apps are stand-alone apps that will provide access to premium TV content via a connected device, whereas second screen apps are applications that enhance the first screen TV experience by providing additional information, content and user experiences. Of course a TV Everywhere app may contain second screen functionality and vice versa, but we think the use cases are normally separate,” he says.

Albert Lai, chief technology officer for media and broadcast solutions at Brightcove agrees. The online video technology specialist shifted focus earlier this year towards deeper and broader native frameworks and developer tools for premium video apps – away from its now- discontinued App Cloud solution.

“We, as a company, have seen second screen take on many different definitions within the marketplace,” says Lai, claiming he would divide this into “three different buckets of use-cases.” The first, he says, relates to ‘social TV’ or social viewing, “meaning that a device, typically a handheld device, is used in conjunction with a lean-back viewing experience”. This could be either a social network or a dedicated check-in service, but one where there’s “not necessarily a connection between what’s being viewed on the television and a mobile device, beyond the fact that the viewer’s watching a piece of content”.

The second example, Lai says, is “using the mobile device to augment the viewing experience with directly correlated or directly related content. That could be either advertising, or secondary content .” The difference here is this is typically delivered using audio content recognition technology. The third, he says, are dual screen experiences that are “redefining what second screen means” – where “instead of thinking about the TV first, you think about the mobile device or the mobile application as your primary device.”

The Twitter factor 

One area that Lai says has seen a lot of partner interest is ‘Twitter cards’. This Twitter initiative enables “media experiences” to be attached to tweets that link to content – for example a video media player card. This is just one example of how the micro-blogging network is getting savvy to the opportunities presented by the broadcast world. As well as a platform for disseminating media – be that video, audio, or pictures – Twitter is  increasingly positioning itself as a second screen TV app in its own right.

Twitter UK’s head of broadcast partnerships Dan Biddle has previously referred to Twitter as the “world’s biggest sofa” and talked up the power of the microblogging network as a means for people to connect and discuss live TV in real-time. Speaking at TV trade fair MIP in April he revealed that in the UK alone 40% of Tweets made during peak TV hours are about TV. It made sense, therefore, in December when the firm partnered with media measurement company Nielsen in the US to create a ratings system based on the  Twitter conversations around TV shows.

According to research released earlier this year by Nielsen and SocialGuide – the social TV measurement and analytics firm that Nielsen acquired in November – the amount of buzz a TV show gets on Twitter directly correlates to audience figures, with increases in Twitter volume related to increased ratings, particularly among those aged 18-34. In this age group, an 8.5% increase in Twitter volume corresponded with a 1% ratings jump for premiere episodes, the research claimed.

This was demonstrated at this year’s UK Wimbledon tennis tournament. The climax of the men’s final sparked a Twitter frenzy, with more than 120,000 messages per minute sent on the micro-blogging network – a bigger spike than was recorded during last summer’s Olympic Games. The match also happened to draw the highest UK TV ratings of the year, with BBC One hitting a peak audience of 17.3 million viewers, a 79.6% share.

While Twitter was rumoured to have been holding talks with TV networks earlier this year in a bid to bring more TV content and clips to the site, the fruits of these efforts are already being seen. ESPN reportedly agreed a deal with the site in May to show in-tweet video highlights of American football, soccer and the X Games. BBC America meanwhile launched its first ever in-tweet branded videos for an entertainment series this week, with the new season premier of car show Top Gear.

In the UK, SecondSync is a company that analyses social media conversations around TV shows on Twitter to provide insights for clients in broadcast and advertising. Company co-founder and managing director Andy Littledale explains the firm has a three-step process for mapping tweets. This includes extracting and tracking key terms from TV listings, as well as using in-house software to search for ‘microtrends’ in conversations generated by the audience and for hashtags that are pushed out via on-air prompts.

Littledale says: “I think more and more broadcasters are seeing social media as a kind of litmus test for how well their programmes are received. It’s not just a case of integrating social media into their actual TV shows, although that is definitely something they’re doing more and more.”

While SecondSync primarily offers data gleaned from Twitter, the firm has also partnered with programme makers to offer a live version of its dashboard. For instance, FremantleMedia used this in the last season of The X Factor UK so that on-air presenter Dermot O’Leary could feed back to the audience real-time stats about audience response to acts during the live broadcast.

The power of this kind of offering comes as more and more people take to Twitter while watching TV. “At the very top-end we see 25% of the audience sending a tweet around a TV show, and that’s for things like Made in Chelsea on E4,” says Littledale, noting that “structured reality genre generates a lot of engagement” thanks to the style of these shows and their core youth demographic. At the other end of the spectrum, children’s programmes on channels like Cbeebies generate no Twitter traffic, with a “myriad of different types of show” falling in between these two extremes. “In the UK there are 10 million active Twitter users. We’ve mapped seven million unique Twitter accounts in the UK to TV shows in the last 12 months, which is an awful lot of people,” says Littledale, adding that “the conversation on social media is far larger than the number of people using second-screen companion apps.”

Tvinci’s OTT 2.0 platform offers social content discovery and interaction.

This assertion is backed by an NPD Group study from April. The research firm said that though 87% US entertainment consumers use at least one second-screen device while watching television, viewers are less willing to use these devices to interact directly with applications designed specifically for the TV programs they are watching. Indeed, among those who use second-screen devices, only 47% participated in second-screen activities, according to the study. Those that did interacted most by visiting IMDb, Wikipedia, and social networks.

Zeebox and Shazam

Social networks may have an advantage over third-party entrants in the second screen space in that they have an engaged, pre-existing audience. However, newer TV-focused services can offer viewers much more in terms of functionality, according to Anthony Rose, co-founder and chief technology officer of much-hyped second screen service Zeebox.

“Twitter is obviously a fantastic social medium, but the problem is it’s hard to have a conversation on Twitter. The messages are short and you can’t readily reply and have a threaded conversation. You can do an @reply, but you can’t follow a threaded discussion on a given topic – it all disappears into a stream of thousands of tweets,” says Rose.

In an effort to solve this problem, Zeebox has recently added new features to its service. TV Rooms is one of these and lets TV fans communicate to each other in virtual chatrooms based around specific shows, characters and themes. This is designed to improve the way that fans can interact with TV content, but also allows brands and advertisers to engage with fans in a new way.

“You can have multiple topics, you can have threaded conversations, and importantly you can link it to a TV schedule – so you could create a car-lovers room and link it to Top Gear, Fifth Gear, or a range of shows. You can make them public or private, there can be official rooms that the broadcasters own, or they can be your own room. We’ve recently added user-generated content uploads so you can upload images and videos and so on and we’ve added moderation tools for broadcasters,” says Rose, claiming TV Rooms create a “completely new use-case in Zeebox”.

Former iPlayer boss Rose co-founded Zeebox in 2011 with former EMI president Ernesto Schmitt. Initially launching in the UK, it has since gone live in the US and Australia, aided by partnerships with the likes of Comcast, NBCUniversal and Australia’s Network 10. The service is designed to turn users’ smartphones, tablets or laptops into an interactive companion allowing them to use it as a remote control for the TV, buy what they see on screen and get information about whatever programme they’re watching – including access to exclusive interactive content.

By partnering with Zeebox, the firm says that broadcasters and content owners can augment their TV shows, engage viewers and monetise this medium in sync with live TV. However, it is now looking to innovate further and to make the service more suitable for viewers watching time-shifted content, moving away from the first iteration of Zeebox, which Rose admits was “all about live television”.

“I think when you take a step back, you can probably imagine three concentric circles of user behaviour. Number one, the inner circle, is you’re in front of your TV and the TV show is on live right now. The circle around it is you’re in front of your TV and the show you’re watching, you’ve PVRed or recorded, and you’re watching it catch-up, essentially. The outer circle is you’re not in front of your TV at all, but actually you’re still a fan of Game of Thrones and you’re going to be watching it at some point, but right now it may not be on.”

Though Rose says “the first version of Zeebox was all about the inner circle,” the reason for this, he claims, was that “there are only so many things you can do in a day, so you start with a focused proposition.” Two years in, he says “we’ve now been able to move from that inner circle to be able to address that next circle, and so a lot of the features you see in Zeebox are to bring the same synchronised experience with both live and recorded shows.”

Part of this can be seen with the recent addition of automatic content recognition to the service, following Zeebox’s partnership earlier this year with Gracenote, which identifies TV shows using audio fingerprinting technology. The reasoning behind the deal is clear. “Our US-partners tell us that for things like comedy and drama, more than 50% of viewing for significant amounts of that content genre is now watched not live and is PVRed. Obviously some programming like The Voice and big social events are still very much a live experience, together with news and sports, but drama and other genres are increasingly [being watched] on-demand,” says Rose.

The shifting focus from live to time-shifted TV is putting Zeebox in closer contention with services like Shazam – a music recognition service that got wise to the potential of tapping into the TV market. As executive chairman Andrew Fisher explains: “Since Shazam was originally launched in 2002 – long before the smartphone – people were already using the service to find out the name of the music featured in their favourite shows and ads, so we knew that the behaviour pattern was there. And, TV advertising offers an amazing opportunity for innovation as it is a US$300 billion business. So, we have remained true to our core capabilities, using our audio recognition technology to enable people to quickly and easily access the information that interests them in seconds, with the press of a single button.”

Shazam officially launched its ‘Shazam for TV’ service in 2011. Users of the app can find out additional information about the show they’re watching, including what music is used in the programme, entertainment news related to the cast or guests and trivia. It also includes Twitter integration and, crucially for revenues, offers second screen advertising opportunities for brands and marketers. A recently launched Shazam Engagement Rate tool makes it possible for the latter to measure the impact of their ad campaigns, offering data on when and where ads are resonating with their target audience.

Fisher says that since Shazam for TV went live it has seen tremendous growth, claiming that “millions of people each month” use the service in the US alone. “At the beginning, we were working with only a handful of pre-recorded TV shows and ads in the United States. Since then, we have expanded our Shazam for TV advertising service to 28 countries in North America, Western Europe and  Asia Pacific, making more than 250 TV campaigns interactive for over 150 of the world’s top brands. Additionally, we have innovated for Shazam for TV programming, so that all shows – live or pre-recorded – on 160 channels in the US can be Shazamed.”

Audio recognition 

Red Bee Media used audio watermarking in its Walking Dead companion app.

Audio content recognition technology has made it possible for independent second screen applications to layer services on to linear TV without requiring the permission, or co-operation of broadcasters. Typically, these independent services use audio fingerprinting technology in order to do this. Audio watermarking is a second powerful solution for triggering second screen responses, though this requires signals to be embedded into a show, making this solution more attractive to the broadcasters themselves.

“If you’ve got access to the content, then watermarking tends to have the lowest synchronisation latency, because you’re basically putting a data signal all the way through the programme broadcast. No matter what time you switch on the programme, the application is listening for that unique, non-audible-to-a-human signal, so it’s very quick to sync. Fingerprinting is slightly different. You don’t need to actually change the media, so if you’re not the broadcaster then fingerprinting’s great, because you don’t need access to the media to be able to synchronise. But it has some challenges. For example, most programmes have a beginning opening sequence that’s exactly the same, so it can take a little bit of time to figure out what episode you’re watching and if the content is specific to an episode, the user is sitting there waiting for something to happen,” says Steve Plunkett, chief techonology officer at Red Bee Media.

The TV technology and media services company used audio watermarking for a second screen app it built for UK channel FX around the hit US series The Walking Dead. Launched last year to coincide with the zombie drama’s second season run in the UK, the companion app added a gaming element to the show on the second screen, allowing viewers to predict things like the number of zombies that would be killed per episode, which they could track in real-time on their iPhone or iPad.

Plunkett claims that the success of the app – which notched up 62,000 downloads and 300,000 gameplays in the first 60 days generating what Red Bee says was a 37.8% viewer uplift – has lead to “many conversations with clients keen to do something similar”, particularly “in relation to drama and other formats that don’t automatically lend themselves to that sort of [play along] experience”.

However, progress has not been as quick as might be expected, due, in part, to the different interests of a typical broadcast client, with marketing, content and audience research teams often looking to use the second screen in different ways, according to Plunkett. “What I’ve observed is that those different constituents within a broadcaster have slowed down the deployment in some cases, because you then get into a much bigger question about what sort of common central platform you might want to use to accommodate these things?”

“I think in the early days, each of those different groups would have tried to go alone and just create a small app. Now I think people are trying to say ‘how do we represent the channel brand more consistently and how far should we try to engage the audience across the programming schedule?” says Plunkett.

Part of the answer could be the rise of a more general-purpose broadcaster apps, for example Channel 4 in the UK’s soon-to-launch 4Now service. Announced last month, Channel 4 claims it is the first UK broadcaster to develop a dedicated, centralised app to accompany its channels and programming, offering additional content for programnmes across the Channel 4 portfolio.

Technology company Civolution works with app developers, content owners and broadcasters to provide synchronisation tools – and has both audio watermarking and fingerprinting technologies in-house. While it provides the audio fingerprinting technology for US-based social TV app ConnecTV, CEO Alex Terpstra believes that “in Europe in particular, most broadcasters now indeed prefer watermarking over fingerprinting, because of its robustness and because they have more control.”

“A trend that we have clearly seen over the past let’s say nine to 12 months is that more and more broadcasters already have or are now in the process of installing watermarking as a permanent feature in their playout facility,” says Terpstra, claiming that this means they can “provide some sort of interactive experience around any programme, rather than just specific formats”. However, he believes that there is definite scope for independent third party players and broadcasters to step up their cooperation. “The hype factor for the second screen I think is already over,” says Terpstra. He claims that with many second screen businesses burning through their venture capital funding, they are now “pressed to start becoming a profitable business – or at least to show a very strong growth – and that’s not always that easy. It’s not always that obvious, because you need an audience in the first place.”

“At the end of the day, the question is whether the platform is viable enough to run a healthy business, or does it make more sense to  co-operate with broadcasters and have more rich content in your application as well? Time will tell, but I would think that a level of co-operation between broadcasters and these app platforms will make the business a lot healthier and will make the user-proposition, at the end of the day, a lot more interesting,” he says.

Ido Wiesenberg, co-founder and vice president of business development at Tvinci, believes that there is still room for operator and independent second screen apps to co-exist. Tvinci is a pay OTT TV platform aimed at companies interested in offering linear channels, catch-up TV and VOD across different connected devices. The firm’s current focus is on its OTT 2.0 platform, which is designed to bridge the world of OTT and second screen apps – offering personalisation, social content discovery and interaction, as well as control of the TV via the tablet or smartphone.

While its intention is to offer all this fully integrated into operators’ core offering, not in the form of add-on apps, there is still a place for independent players, says Wiesenberg. “I believe that end-users would like to get more from their operator applications – so the ability to connect between the tablet and the TV and to discover content. I still believe there is room for companies like Zeebox. People would like also to have something which is natural, which is not part of their pay TV subscription. I believe there is a place for those two.”

However, with the general TV-watching public yet to widely adopt dedicated second-screen services, there is still substantial room for the market to grow. Red Bee’s Plunkett says: “As a technologist, I tend to get excited by many of those things and would be an early adopter, but if I look at a broader group of people, even amongst family friends and so on, I think a lot of people have dipped into those applications and found an interesting novelty and then haven’t really stuck with them. So I do think as an industry we still are really only scratching the surface on the second screen.”

Civolution’s Alex Terpstra agrees: “I think it’s very exciting – there’s a lot of things happening in this part of the industry right now and I think we will see a lot more coming. But I also think that the time to play and experiment is a bit behind us now. There’s going to be more pressure on actually making money out of these platforms, so that will also probably cause a little bit of a shakeup and the ones that have most value to offer will survive, and that’s going to be a very interesting period.”