Dmitry Shishkin on using audience data to inform editorial decisions

Editor’s note: this interview is part of The Fix’s email course on audience building and engagement for editorial leaders. You can subscribe for free to access the whole course.

“But what does all of this MEAN?!?!?!?!?!”

That’s a question I’ve asked myself more than once when poring over the data that show me how many people read my last article, how many converted, how long they stayed on the article, how much they engaged with it, were most of the reads from casual or loyal readers and so on.

It’s possible to get so much more data on our audiences now than ever before, but amid all the data, statistics, figures and facts, I sometimes wonder: do we really know them any better?

I’m sure many of you taking this course have also been frustrated at times when your audience tells you that they want you to cover a certain topic more – but then none of them convert on those stories! And the stories they tell you they hate the most are the ones that get clicks!

As many things, it’s partly an uncertainty we have to live with. As useful as it is, audience data will only ever give us an approximation of what our audience wants. Our audience isn’t a homogeneous mass, it’s made up of lots of people, as self-contradictory as people are.

But shifting my focus from “audience wants” to “audience needs” has helped me. 

The former tends to focus on our most-read topics, but leaves us vulnerable to becoming too reactive. If a story about buying a car hits your KPI sweet spots, should you really write more about cars, or was it instead successful because you helped your audience with a problem?

I find the concept of audience needs, on the other hand, more easily lends itself to drilling down into what people need to navigate the world around them, to understand it, connect with it, make the most of it, improve it – some of the fundamental reasons why they consume news.

Developed by Dmitry Shishkin while working for the BBC World Service, the User Needs Model is now used by many newsrooms around the world. Subscribe to the course to read more about the User Needs Model and how to put it into practice in your own newsroom.

I sat down with Shishkin, who is now the CEO of Ringier Media International, at the International Journalism Festival in Perugia to chat about how audience data can inform editorial decisions.

You’ve got a new job now, do you ever get tired of being called “the user needs guy”?

I’m actually very proud of it. It’s incredible to see that something we started in 2015 is suddenly everywhere almost ten years later, and on so many different levels, niches, international, national, regional, local, any publication – we have so many examples of good things coming out of it.

I feel like a few years ago, or at least ten years ago, there was an attitude that we, the journalists, know best what stories to cover and how to cover them, but today most newsrooms would describe themselves as user-centric. Do you agree there’s been a shift, or is it just something they’ve understood that they should say they are?

I think it’s both things. I think they understood it, they changed, and they became audience-centric at the same time. 

The beauty of the user needs model, generally, is that you are not teaching anything new to people. People instinctively understand it because it’s nothing but a collection of angles to a story, which we have been doing forever and ever and ever. Once you start measuring the audience data according to that variable, then really interesting things start to happen.

I really like that it doesn’t tell you what topics to cover, but how to cover them.

Precisely. Cover whatever you want. Even recently, we had a User Needs Lab with a company called Smartocto, which is an editorial analytics tool that takes the user needs model seriously. 

We had ten participants with nothing in common apart from a passion for using it and one of them was Chemistry World, a very niche UK publication about chemistry. They understood the model and started applying it for their publication and you immediately see results.

If you work in a big newsroom or one that invests in audience data, you might have someone whose role it is to analyse that, but if you don’t have that, it’s often left to editors and journalists who might not be experts If you forgive a stupid question, can you talk me through in a simple way how you analyse audience data?

It’s completely not a stupid question. First, I don’t recommend having too many journalists and not a single audience data person. I always say that if you have ten people and one of them resigns, hire an audience person. 

That audience person will make the nine other people much more competitive and productive.

There are three really important things that I consider crucial for any audience analysis. Every CMS needs to have three drop-down menus that you manually, or maybe with AI, add to each story. 

One of them is the topic, which you always do because you place the story in a particular part of the website. If you have a secondary tagging, it’s even better. For example, you can say economy is the primary topic, personal finance is the secondary. 

Then, I always recommend agreeing on about ten formats that your newsroom uses, and you select what kind of format you’re using for a certain article.

And the third one is you always assign one user need to a story. This way, when you start extracting the data, every story and every URL will have a lot of data points.

My recommendation is to always start small with one section rather than with everything when you analyse the data. Don’t take the output of the whole of your website for a week, but take one section and three months. Then you will be able to start analysing things on a section by section basis. 

You need to be topic specific. You can’t really do that analysis as a whole because it doesn’t really tell you anything. My recommendation is to do audience analysis based on the topics.

The person you hire, should they be a data person or more an audience relations person?

No no, it should be an editorial person who loves audience data. Somebody who loves and is respected by the newsroom. 

It’s really important to make sure that this is not seen just as a number crunching person. That’s why I always say never hire audience managers but always hire audience editors. People in the newsroom respect people who have job titles similar to theirs.

What about metrics? If you only have the resources to focus on so many things at the same time, what piece of audience data is the most important? Do you look at clicks, conversions, read depth?

Ideally you capture everything. But you don’t want to do any change management with too many messages and too many things that people need to hear about, so select one. 

If you start on this journey, you’re not likely to be able to agree on your North Star immediately because that will take some time, so you inevitably probably will need to start with reach. 

Once you are confident with your reach metrics, then you need to start looking at engagement metrics. So anything about pages per session, what is the user behaviour that consumed that story, loyalty frequency etc. Start with the most important basic numbers and then go deeper into that.

For example, if you’re a display ad enabled business and your business model is all about selling ads – that’s a reach game. If you are a subscription model, then it’s likely more about engagement.

In a lot of newsrooms when things go badly, we tend to revert back to reach…

It’s inevitable. We need to normalise that behaviour because I think it’s just a human reaction to trying to fix things. 

But at the same time, if you are going through those types of challenges, you need to have somebody who is very direct and precise about what exactly you want to do – the editor-in-chief or deputy editor-in-chief or audience development editor, or maybe it needs to start with the CEO, frankly, because they need to say that no matter what, we need to pay attention to this. 

Generally it’s about being clear about what “good” looks like. If you bombard your newsroom and your editors with a number of priorities, you’re not likely to achieve anything, because then everything becomes important – and if everything is important, nothing is important. So stick to your guns and have that vision.

Do you have any tips about specific tools to use?

Well, as I said the three pillars of analysis need to be put in place.

To give you an example: Be specific about what type of content you are creating. Have your website optimised so that you’re not producing the wrong type of content. I know websites which for some bizarre reason have a section which will produce five articles a year or something like that. Why do you have that section in the first place? Don’t have it. That’s point number one.

Point number two is the formats capturing – that is assigning formats to articles. So you will say we have features with first-person interviews, we have news write-ups, Q&As etc.

And then you also agree on the user needs. Those three things are really important.

Then be precise and organised about how you go about introducing it. Don’t do everything at once, go section by section, and you can probably turn the website around in maybe six months.

Once you do that, work with editorial tools, whether they are distribution tools or editorial analytics tools that can reflect all the information that is important to you. 

Ultimately, in my dream scenario, every editor will have two dashboards in front of them: a dashboard of the current situation or yesterday, and the dashboard of what happened to your section editorially over the last three months or so. That’s nice because the first one allows you to react immediately – maybe, you know, “commission that content” or “do better follow-ups”. But the second one actually allows you to start changing your output.

How does AI play into all of this?

I’m super excited about AI in this particular respect because it can be a kind of sounding board which you can disagree or agree with. 

Imagine you come to work and in the morning meeting, instead of just reading the planning meeting notes, you can rely on things that AI has created for you as options. 

So an AI will, say, send you a notification in your Slack or email saying “this story did really well yesterday – you clearly will do a follow-up today and based on what we know about that story, the format you chose and user need that you used, the best combination of those three things will be X”. The AI can offer you maybe five or six options of those and it will say “if you are thinking of leading your newsletter with this particular topic, this is the best user need”. 

I’m super excited about AI being a copilot for those types of editorial experiments.

Source of the cover photo: Dmitry Shishkin’s photo, courtesy of Ringier 

---