[Editors note: This article was originally published on Entrepreneurial Journalism, created by James Breiner. You can sign up for his newsletter here.]

Big databases help news organizations predict who might subscribe or donate

If it seems like this blog is turning into a book review section. It could be because classes have ended here in Pamplona and I am reading more to “sharpen the saw”. (Can you guess which book that quote is from? Answer at the end of this post.)

A book on artificial intelligence caught my attention recently because its findings have so much relevance for news media and journalists. The book makes robots seem more like a friend and less of a threat to producers and writers. I’m talking about Prediction Machines: The Simple Economics of Artificial Intelligence by three economists, Ajay Agrawal, Joshua Gans, and Avi Goldfarb.

The relevance for news media is that prediction machines (algorithms) developed by Google and Facebook ravaged the business model of news organizations, but could also rescue them. The tech platforms’ strategic advantage is that they know more about publishers’ users–their preferences, online behavior, personal and professional relationships–than the publishers themselves, which often collected little more than a subscriber’s name, address, and telephone number.

The tech platforms used machine learning to refine their algorithms and target ads relevant to the interests and habits of consumers, thus generating more sales for advertisers at a lower cost than the publishers. Of course advertisers fled to the platforms.

For publishers to compete, they have to find ways to acquire enough data at an affordable cost to develop their own algorithms (prediction machines), and not just for advertising. As they transition to user-generated revenue, they need to be able to predict which online users are most likely to subscribe and target them with appealing offers.

Not enough business intelligence

A new study of 130 artificial intelligence projects by news organizations found that nearly half of them were focused on “augmenting reporting capacity” and about a fourth on reducing costs. Only 12% focused on optimizing revenue.

Among other potential applications of AI, the report, funded by the Knight Foundation, suggested using shared data to develop dynamic paywalls and subscriber prediction algorithms.

In machine learning, the algorithm keeps refining its predictions the more data it collects. Most publications don’t have these kinds of richly detailed databases nor the technology staff to develop sophisticated algorithms. But Google does, and has been using what they have learned to coach publishers. (More on their work with news media here: These metrics predict who will pay for news.)

Let Sophi decide

Canada’s Globe & Mail offers an excellent example of how this can work if a news organization has enough vision and financial resources to take advantage of AI. In an interview with the Press-Gazette, Publisher and CEO Phillip Crawley described how the Globe & Mail accelerated the growth of digital subscribers to 170,00 through a software program they developed. It’s called Sophi.

A bit of background. In 2012 the Globe & Mail established a metered paywall, which allowed users to view a certain number of articles a month before requiring them to pay for a subscription. But the executives weren’t entirely satisfied, so they brought in some tech experts to develop a program to drive up digital subscriptions.

The result was Sophi, a computer program which decides whether articles should be free to access or put behind the paywall based on article content and reader information.

Crawley said Sophi is largely responsible for the rapid subscription growth. The program learns from user behavior how to decide which articles to place in which positions on which pages in order to maximize engagement, loyalty, and propensity to subscribe. Each day, Sophi adds more data on user behavior to refine its predictions–machine learning.

The program reviews all content every 10 minutes to determine which articles merit greater promotion on the website or on social media, Crawley told the Press-Gazette. Now the G&M is selling the software to other media.

The villains become saviors

It is somewhat paradoxical that Google, a destroyer of publishers’ business model, has been helping them in a significant way. It offers a low-cost service for small publishers called Newspack, which harnesses machine learning to, in its own words, combine “the best of what the industry has learned about publishing and revenue generation”–machine learning, in other words.

Probably no one knows more than Google about internet users’ tastes and buying habits. The Institute for Nonprofit News, working with the Google News Initiative, has developed tools for raising money from events and sponsorships. Another Google News Initiative partner is Lion Publishers, an association of 275 local news entrepreneurs in the U.S and Canada. The platform sponsored an eight-week bootcamp for 24 digital news startups.

More from The Fix: Building a modern and sustainable news outlet is easier than ever

Twitter gets into subscriptions

Twitter has recognized the value of this type of data for subscriptions. It just bought the Scroll subscription network of more than 300 news media sites, which has aimed to serve consumers and publishers by offering an ad-free reading experience.

Twitter evidently sees an opportunity in combining its enormous database with Scroll’s network to generate subscriptions on its own platform. Scroll aggregates data on consumer engagement in all its member sites and distributes revenue proportionately. Consumers pay $5 a month, and each publication gets a share based on users’ engaged use of their content.

Twitter, like Google, has a database on news consumers’ content preferences, behaviors, and relationships that news publishers can’t afford plus the programming team that can identify potential revenue opportunities and create attractive offers.

More from The Fix: Twitter is betting on a “cleaner” Internet

Increasing probability

Like some doomsayers, journalists typically see artificial intelligence as destroying jobs in the profession by having machines produce routine stories in finance, sports, weather, and other fields that have large structured databases. But actually, AI might be helping to save quality journalism.

In economic terms, the news has little economic value. Within moments, everyone in the world knows what President Biden or the Federal Reserve did, and why would anyone pay to get access to that?

As the authors of Prediction Machines demonstrate, algorithms can use data to make educated guesses using millions of data points on thousands of variables far faster and better than any human marketing director ever could. And they can improve the probability of accuracy the more data they have.

And as for that little quiz I gave you at the beginning of this post? “Sharpen the saw” is the seventh habit of The Seven Habits of Highly Effective People, by Stephen R. Covey. The phrase is a metaphor for continuous learning and personal development. It’s what we humans have to do if we want to keep beating the machines.

Subscribe to The Fix Telegram channel: https://t.me/thefixmedia

Photo by Markus Spiske on Unsplash

sustainable

Latest in sustainable

For-profit publishers race to collaborate

How AI and data boost journalism sustainability

Building a modern and sustainable news outlet is easier than ever