Staring at the number of viewers as it rises and falls on a website’s analytics chart can be mesmerising. In real time you see who entered an article, who decided to stick around for more, and who bounced on to another website. Such quantitative audience metrics are widely prevalent in news organisations around the world – often displayed on a big screen in the newsroom. They’re easily accessible, addictive to watch, and for many outlets they’re something to show to the advertisers that help to keep the lights on.

I’m no stranger to the rush of excitement when a story I share on social media gets traction. But I was convinced there was a better way to show journalism’s contribution to society. After some digging, it turned out it’s a question more people in the media were starting to think about. But no one had yet unlocked the whole puzzle.

Journalism’s measure of success

How does one measure success? If you’re a business owner, you’ll try to sell more of your product in February than you did in January. If you’re driving a race car, your aim is to cross the finish line faster than anyone else on the track. But what if you’re a journalist? If a thousand people read your investigation, does that mean you’re successful? Or did Facebook’s algorithm show your story to more people than usual for a reason that you’ll never be able to replicate (or even understand)? Out of those thousand views, do you know if most of the people actually agree with your conclusions?

While there’s a myriad of ways to interpret quantitative audience metrics, I think it’s fair to agree that if your investigative piece attracts less eyeballs than a 15-second viral TikTok dance, it doesn’t make your news work that went into it any less important.

Website clicks come with a certain lack of meaning along with the risk of tempting newsrooms to chase photos of kittens instead of doing valuable investigative work.

What if we were to consider changing journalism’s definition of success? What if we were to start caring more about the impact we create in our communities and use that as the measure with which we evaluate our work? To find out whether this was even possible, I spoke with 15 different news organisations from around the world for my master’s degree in journalism. I wanted to find out what they consider to be journalistic impact and how they measure it.

Defining impact

Impact can mean different things to different people and different newsrooms. A newsroom that aims to educate its audience might be more interested in measuring if it changed people’s opinions about a controversial topic. Investigative newsrooms may focus on entirely different kinds of impact.

Over in the UK, Impact Producer Grace Murray who works for The Bureau of Investigative Journalism (TBIJ) said they define impact quite broadly. For TBIJ impact can be when they spark conversations on a national or international level, when they change individual perceptions, and if they empower people through their journalism. TBIJ measures from the micro; one man having a new kitchen built as a result of its investigation into home adaptations, to the macro; a French police investigation into email hacking around the Qatar World Cup that TBIJ had uncovered.

“For me personally, when I came into the environment team, I wanted to prompt political change in terms of legislation and to bring about business change. So our food and environment work focuses very much on the supermarkets and supply chains, and how UK supply chains are linked to deforestation of the Amazon rainforest,” Murray said.

Former public relations and marketing manager for Czech outlet Investigace.cz Nikola Klepáčková said they classify impact in three different ways – how many people a story reaches, the discussions and mentions a story triggers, and what tangible changes a story leads to.

Tracking impact

Sometimes measuring the defined impact is straightforward. Measuring changes such as resignations, new laws, and arrests of criminals can however be particularly tricky.

“Change comes a long time after we publish a story. said Klepáčková.”

So I think it’s really difficult to measure change, because it could even be that something happens 10 years after we publish a story
marketing manager for Czech outlet Investigace.cz Nikola Klepáčková

To complicate matters, if for instance a government implements a change that was triggered by a journalistic investigation, that government rarely gives public credit to that news organisation. Unless you’re consistently looking in the right place at the right time, some impact arising from your journalism is likely to fall through the cracks.

Klepáčková said the type of impact the measure determines the method they use to track it. So to track reach they use tools like Google Analytics, and Facebook’s Business Manager. When it comes to identifying media mentions they use a tool called Monitora that covers different platforms and even printed newspapers – the team determines what keywords they want to be looking out for and Monitora does the rest.

From her end, Murray said they originally used an impact tracker developed by the Center for Investigative Reporting (CIR) that was then being redesigned by a private firm called Impact Architects.

Another tool TBIJ uses to track how and where their stories are picked up is Meltwater.

“Meltwater tracks where the stories are picked up across the world. We had an investigation about oxygen supplies in Mexico and it was useful to see on Meltwater that other Latin American press picked it up,” Murray explained.

Getting started

Measuring clicks and shares is a very sophisticated business with a number of (relatively pricey) tools available to analyse all kinds of numbers. When it comes to measuring journalistic impact though, the field is more rudimentary. While practically every organisation I spoke with was able to track it in some way or another, for the smaller ones it sometimes became a chore that was completed whenever their team found some spare time – a luxury many journalists are unfamiliar with.

If you want to start measuring your investigative newsroom’s success through your impact, you can start small. Gather your team and have a conversation about what you want to achieve through your work. Create a working definition of what impact means to you and improve it as you go along.

Then start tracking it. If you’re interested in tracking which other outlets are citing your latest investigation but can’t afford tools like Meltwater, consider creating your own alternative. Turn on some relevant Google Alerts and start recording your findings in a spreadsheet. Aiming to track everything is probably unrealistic and some pieces of evidence of your impact are likely to fall through the cracks. Don’t let imperfection dishearten you.

Resources at small investigative newsrooms are usually already stretched. Adding impact tracking to the list of tasks that need to be done can seem like an unnecessary burden. However, it can come with potential benefits.

Communicating your impact with your audience can be used to attract donations to fund future investigations. New sources might reach out to you with a new story if they see that your work is creating change. Having a positive impact on your community can also serve as a useful recruitment tool to hire the best talent. Overall, achieving impact and demonstrating it can be a way for your news organisation to justify its existence and maintain legitimacy.

Source of the cover photo: https://unsplash.com

Performance

Latest in Performance

How to track the impact of your investigative journalism