‍

“As soon as we started using Utopia AI, I was curious to know how this will impact the daily work of our human moderators. So one morning I asked one of our moderators if they could quantify the changes in their workload now that we have this new solution in place. At first, he wasn't sure how to compare, but just a few hours later, he came back to me and said that the queue of comments to moderate was almost empty and that he had to moderate way less comments than before, which allowed him to make great progress on other tasks and that the number of published comments can be increased accordingly. I realised then that this is a game-changer for us.

We've been using Utopia AI to moderate our German news comments ever since and it's of great help to us. Due to the impressive results, we are expanding the usage of Utopia to moderate the comments section in French as well”

Manuel Sutter
Product Owner at 20 Minuten

‍

1. Challenges

As one of Switzerland's top news portals, 20min.ch receives nearly half a million comments per month on its articles in German as well as French. The ability for readers to comment on articles play an important role in driving reader engagement, which is crucial to their business. The readers expect their comments to be published right away, and of course, no one wants to be harassed or abused with inappropriate comments. This is no easy task for the community team at 20 Minuten and 20 minutes. For example, the sheer volume of comments causes significant publishing delay, and quite often the decision whether to publish or to reject a comment depends heavily on the context. This can easily result in mistaken rejections or publications, which is not good for the user experience, nor the brand.

2. Earlier approaches

Over the years, in an effort to maintain the quality of their comment section and speed up publishing of comments, 20 Minuten’s community team has been looking for a solution that could help them improve the effectiveness and efficiency of their comment moderation. These solutions have included various rule-based filters that rely on words, dictionaries and ontologies. Unfortunately, none of the tools they had tried earlier had met their needs and expectations on quality. They still needed a sizable team of 15-20 dedicated human moderators to shift through a large number of comments day in, day out. This was not only tiring routine work, but it also meant high costs, and due to the manual nature of the work and the volume of comments, the comments in many cases were only published after a considerable delay.

3. Solution

When 20 Minuten’s team heard about the great results Utopia had achieved with other companies in the German speaking part of Europe, for example with Kronen Zeitung and Kleine Zeitung, they were intrigued. However, having tried several solutions in the past and being well aware of the challenges involved, their CTO & Product Officer was understandably very sceptical at first. They requested a meeting with one of Utopia's current news media clients and based on the positive feedback they received, 20 Minuten ultimately decided to give Utopia a chance to demonstrate the capabilities of their patented moderation technology through a pilot.

20 Minuten’s team was positively surprised to learn that the pilot was a very straightforward process, and needed little resources from them. 20 Minuten’s tech team only needed to provide Utopia with earlier human moderated news comments as training data, and integrate from their news commenting system to Utopia API, which took only a couple of days. Next, over a two week period, Utopia trained a tailor-made AI moderation model for 20 Minuten for the German language. The AI moderation was made available in the Utopia API so that 20 Minuten could send every new comment for automatic moderation, and receive a publish or reject decision for each comment in real time.

Before the actual 1 month pilot period started, Utopia provided 20 Minuten’s community team a detailed report on how their current human moderation looked like. Utopia explained that in all cases where there’s a team of people handling moderation, there is always variation between people and thus inconsistent quality. 20 Minuten was aware that having a team of 15-20 content moderators comes with a certain amount of inconsistency in moderation decisions that their manual moderation process needed some additional attention. The details provided by Utopia proved to be eye opening as it provided never before seen insights into their human moderation process, and crucially concrete suggestions on how to improve it.

4. Results

During the trial period, 20 Minuten was impressed with how well Utopia AI was able to catch the inappropriate comments. Now, all the clearly acceptable comments are published in real time, and all the clearly unacceptable comments are automatically rejected, so it’s only the borderline cases that need a human moderator to review them.

Based on the results, it was an easy decision for 20 Minuten to take Utopia AI Moderator service into daily use. As the integration was already done during the pilot, it was only a commercial decision to continue using the Utopia service. Additionally, they used the pilot report's findings to identify areas where manual moderation needed improvement and requested Utopia to keep on providing such reports on a bi-weekly basis to allow them to provide their team with the necessary training to improve their performance.

Today, 20 Minuten uses Utopia AI to pre-moderate 100% of their German comments in real time around the clock, which has helped unify moderation decisions. As a result, the news outlet’s speed of moderation, quality of comments, the user experience, the reader engagement, and moderator wellbeing have all notably improved. They also experienced an increase in the number of comments since implementing Utopia AI, and the moderation team is no longer under the same time pressure to keep the comments section safe, enjoyable and engaging. Based on the great results, 20 Minuten have also decided to expand the Utopia AI moderation to the French comments as well as images on their website.

With the majority of moderation tasks being handled by Utopia AI, 20 Minuten has been able to significantly reduce their reliance on manual moderation. The reduced workload has allowed their team to redirect their efforts towards other areas of their moderation process. As a result, the team has been able to perform their job more effectively, reducing decision fatigue and increasing job satisfaction. These benefits have helped to create a more streamlined and efficient moderation process, allowing 20 Minuten to focus on providing their customers with the best possible experience.

‍

About 20 Minuten

Launched in 1999, 20 Minuten has quickly become one of Switzerland's leading daily commuter newspapers, providing readers with the latest news and insights on politics, business, sports, entertainment, and more.

Since 2005, 20 Minuten has been published by the TX Group, one of Switzerland's leading media organisations. In 2006, 20 Minuten also launched a French version called 20 minutes, further increasing its impact and providing a platform for French-speaking readers to engage with its content.

Since 2011 20 Minuten has also been present in the Italian-speaking part of Switzerland. This expansion reflects the paper's commitment to providing high-quality news and information to as many people as possible, regardless of language or location. With its tabloid format and lively language, 20 Minuten is known for its brief reports and plenty of photos, making it an engaging and accessible read for people on-the-go.
‍