How we successfully developed a personalised news application.
We were lucky enough to have been among the participants of the Digital News Initiative. After 12 months of work, our project officially ended this July and now we believe it is time to share our insights so that everyone can grow and learn with us. In this report, we provide information about what exactly we did in the project and, more importantly, what we learnt about personalisation and how we will we proceed with the service.
NZZ Companion
In a world where anonymous ‘one-size-fits-all’ news experiences are all around, we intended to create something different, something that would be as individual as our users. A product that would be able to take the reading habits and reading preferences of our users into account while ensuring that nothing important would be missed. With that in mind, we set off on a quest to develop a personalised news experience that helps users to find news relevant to them with less effort. The outcome was the NZZ Companion Mobile Application.
The NZZ Companion Mobile Application for iOS and Android consisted of two main visual parts: the curated NZZ.ch-Stream on one side and a user’s personalised stream called Companion on the other side. Additionally, a navigation element named ‘Bibliothek’ was introduced where more user-centred features such as personalised reading lists and bookmarks were found.
The content in the curated NZZ.ch-Stream was identical to our existing NZZ-News Mobile Application or mobile site (that is, it was similar to many news apps created by our competitors). Articles provided in this section were selected by our editors and the same content was shown to all users. The NZZ Companion-Stream, on the other hand, combined editorial curation with algorithmic technologies for a personalised news experience. This separation was chosen deliberately so users in the beta phase could better experience the difference between the two.
During the project, the personalised NZZ Companion-Stream differed heavily from the curated one due to the following features:
-
It showed older articles that were published up to 48 hours earlier
-
It used a variable mix between ‘editorial relevance’ and ‘personal interest’
-
It displayed super-relevant news to all users, regardless of a user’s indicated preferences or interests (‘filter bubble’)
-
The article’s visual interface was lighter and its user features more obvious
What we learnt about personalisation
It was our goal to develop a personalised mobile application that would help users to find relevant news with less effort. This is in contrast to other personalisation efforts taken by other services, which are more likely to show more of the same content to a user to increase the Click-Through-Rate.
To be able to develop a valuable service for our readers we decided to roll out a beta mobile application early in the project. Doing so allowed us not only to co-create the data products and the application in accordance with the needs of our users but also to gather answers to the following questions, which would, in turn, provide valuable and statistically proven feedback to our initial hypotheses (listed below):
-
Do beta testers value personalisation at all and does this perception change over time?
-
Do beta testers actually use a personalised service and are the articles recommended relevant to them?
-
What benefits do beta testers see in the personalised NZZ Companion-Stream?
In the following, we would like to assess the answers to these questions and apply them to our three hypotheses.
Personalisation creates added value
The first hypothesis laid out the ground for any future actions in personalisation because it was all about the value creation of such a service. Our data shows that around 84% of beta testers believe that the personalised NZZ Companion-Stream created added value to them. More importantly, this perception changed and improved by 20 percentage points compared to the beginning of the beta test. We believe that this increase is due to improvements in the algorithms and the addition of new user-centred data products (e.g. personalised weekend reading list) and features (e.g. article rating). As advocates of user-centred design, we also believe that the more users are engaged in a product development process, the more useful they attach to the product, which leads to an increased perceived value.
People need time to get used to personalised content
The usage of the personalised NZZ Companion-Stream increased by 16 percentage points within the beta testing period. While at the beginning of the beta test around 45% of users read articles in the Companion-Stream, this number rose to 61% in the final month.
It is difficult to attribute one single factor to this increase in usage. We suspect the following factors played a major role:
-
People are creatures of habit: Users needed time to get accustomed to the companion stream and then develop a positive attitude and trust towards the algorithm over time
-
Improvements in the algorithms: The balance between editorially and personally relevant articles got better over time — we believe we managed to significantly develop the algorithm over the months of operation
-
Release of new features and new data products: Providing users with an easy-to-use bookmarking feature or the introduction of desired data products such as a personalised weekend reading list could have led to more usage
-
Increased stability: Fewer crashes and bugs
However, what we know for sure is that the perceived relevance of articles in the personalised NZZ Companion-Stream increased over time. In the surveys conducted, we found that beta testers found the personalised articles to be significantly more relevant in the final month as compared to the launch of the product in March.
The value proposition of NZZ Companion is to find relevant news with less effort. We are happy to confirm that we could increase the perceived relevance of personalised articles over time.
There is no single reason why beta testers use the personalised stream
One of the top benefits of the app, as perceived by our beta testers, is the way it provides quick access to interesting content since the companion stream decreases search times. Further, the beta testers appreciated having access to older content that they may have missed in the curated stream. Lastly, beta testers turned to the personalised stream if they wanted to discover articles that they would not have read otherwise.
In line with our goal, we were able to successfully offer quick access to interesting content as this is among the top benefits of using a personalised stream. Additionally, we believe that we succeeded at building personalisation that still allows its users to discover new and surprising content so they do not get trapped in a filter bubble.
Facts and Figures:
Five people worked part-time on NZZ Companion for over one year
A fully data-driven and user-centric approach
Iterative development cycles and lean team management
Five months of beta testing
More than 400 beta testers were used for both iOS and Android
Algorithmic releases roughly every three days
Features: read/unread status of articles, article rating, bookmarks, sharing function
Data products: personalised reading list, weekend and evening reading list