Jonathan Tweed

abc-ip: the final wrap-up

And we’re done!

Yes, it’s hard to believe but ABC-IP, our project with BBC R&D to automate the linking of broadcast content and archives, is a wrap. We’ve been working on it over the last two years, indeed I’ve been working on it since I started at MetaBroadcast, so there’s plenty to look back on.

the final quarter

The final quarter was about tidying things up, stabilising components and generally preparing the best of what we’ve built for productisation and commercialisation.

This meant we did two things:

  • Built a new Tellytopic homepage, with a greater focus on topicality
  • Moved everything to Atlas 4.0

new tellytopic homepage

We rebuilt the Tellytopic homepage to ensure that the content displayed is always the most high value content for the current schedule period. This replaces the previous homepage, which focused on content broadcast the previous evening.

The new design is centred around three things:

  • Top content for the people most popular now
  • Top content for the places most popular now
  • A list of the topics currently most popular on Twitter that provide further onward journeys through topic pages

Additionally, there is a menu to switch between content sources. This defaults to all sources, but can also be used to restrict the content displayed to that from a single provider.

This new homepage provides a useful tool to discover the archive content that is currently most relevant, and hence most valuable, to a large cross section of the audience.

tellytopic

moving everything to atlas 4.0

We also worked on finalising the enhanced support for topics in Atlas 4.0. This has been achieved by migrating both People Match and Tellytopic to the new API, in order to prove the new functionality works as expected.

During the course of this many issues with the new API have been fixed, and many small features that make the kind of requests needed easier to make have been added. This ‘dogfooding’ has been invaluable in shaking out the new functionality and topics in Atlas 4.0 are now complete.

As a further benefit, both People Match and Tellytopic are running against the latest version of the Atlas API and are able to take advantage of the improvements that this brings. These include better results, by combining and sorting data server ­side, as well as significantly improved performance.

a look back at what we achieved

But that was just the last three months. Over the course of the last two years, we’ve built an amazing toolset for automated linking. Here are the highlights.

Atlas data support Extended Atlas to support the ingest of many new data sources, including the BBC World Service Archive. Also added new HTTP POST functionality that makes it significantly easier for future content providers to integrate with Atlas.
Atlas topic support The Atlas data model was extended to include topics from multiple sources that have varying relationships to content. Topics describe what programmes are about, or what subjects audiences ascribe to the programmes they watch.
Magpie Magpie is a service that provides both topic extraction and Twitter hashtag generation for programmes in Atlas. It reads programmes from Atlas and writes back equivalent items with automatically generated topics and hashtags. This means that programmes that do not have topics in the source data can be included in topic­-based discovery mechanisms and tweets about those programmes found on Twitter.
Twitter topics This component performs realtime topic extraction from tweets about programmes on Twitter. This enables a realtime link to be made as the programme is being broadcast between the content of a programme and what audiences are saying about that programme.
Topic recommendations Used to generate the topics related to a topic, to display as similar or related topics. Works by finding the topics that co­occur on programmes most often with other topics, taking into account both topic weightings and topic frequency.
Programme recommendations Used to generate the programmes related to a programme. Works by finding the programmes that share the most co­occurring topics, taking into account both topic weightings and topic frequency.
Zeitgeist Zeitgeist is a service that analyses programme topics in Atlas and provides popular topics and programme recommendations based on current topics. It determines what is currently topical by looking at popular programme topics and popular Twitter topics in the current schedule period.
ALSO ALSO (Audience Loves Something Other) is a powerful visualisation of audiences from the conversations they have on Twitter. For each tracked programme, it shows two things: what the audience talks about during the programme and what the audience for that programme talks about at other times.
People Match People Match is a browser extension that augments any existing website with links to related content in any archive that has been indexed in Atlas. It works by extracting topics from the web page currently being visited by the user and then queries Atlas for matching content and displays it to the user.
Tellytopic Tellytopic is an end­-user prototype that enables browsing programme archives by topic. It includes programmes from the BBC, World Service Archive, Channel 4, ITV and Channel 5 and provides various methods for navigating between them using topics from each of the sources developed during the project. Methods for programme discovery include a Zeitgeist powered homepage that shows content for the topics that are currently popular, as well as topic pages, similar topics and programme recommendations.

Of course, ABC-IP was an R&D project, so we still need to do work on each of these components to get them to a production-ready standard. This work is underway now and we expect the first users to go live over the next quarter.

final thoughts

So, what was our big learning from ABC-IP? Well, quite simply: topics. Structured data for what a programme is about.

Topics are becoming increasingly important for automated linking and more granular recommendation. These are areas we’re already exploring further in other projects, so expect to hear us continue to talk about them over the next few months and beyond.

We’ve certainly built a lot of things over the course of this project, but the work we’ve done puts us in a fantastic place to now start working with commercial partners on automated content linking. Do get in touch if you’d like to see more and discuss further.

blog comments powered by Disqus