#jcarn: Measuring journalism

Note: This post is another installment for the Carnival of Journalism project, where people passionate about journalism are sharing ideas in the blogosphere about ways to preserve and improve the craft.

This month’s prompt:

What’s the best way — or ways — to measure journalism and how?

The root of journalism is truth, and the time-tested method that journalists have to uncover that truth is verification. If we want to measure journalism, it must begin here.

Respected journalists Bill Kovach and Tom Rosenstiel explored this idea more than a decade ago as part of their work with the Committee of Concerned Journalists, which they codified in their book The Elements of Journalism. As they noted, what sets journalists apart from every other type of content creator — entertainer, technologist, pundit — is verification.

Simply put, verification is checking out everything with multiple sources. I think of the adage I learned early in my journalism career:

If your mother says she loves you, check it out.

It’s only semi-facetious. It doesn’t mean your mother is a liar. It just means she may not be quite accurate.

Maybe she wasn’t talking to you.

Maybe her definition of “love” doesn’t jibe with conventional wisdom.

Maybe she’s not really your mother.

In today’s environment of the immediate and the ephemeral, it’s ever more important to verify the drive-by comments, the frenetic machine gun of news. It’s too easy to retweet, reblog, or republish what’s happening with the claim that the rush of information requires us to be first rather than accurate.

What does the audience want?

What’s clouding journalism today is the need for audience. In such a fragmented environment, how do we generate enough interest — in terms of page views, time spent on site, or other measures of engagement — to secure advertising to pay the bills?

We tap into Google Trends and try to SEO our headlines and content to lure people to our website. We crank up the level of content with frequent updating — sometimes including links to hot content that we haven’t independently verified. The goal is noble: Get them to our site so they can see our real, verified journalism.

But some pockets of experimentation are turning that idea on its head. Gawker has begun finding that original content may be generating more meaningful connections than its SEO junk. Salon recently noted its recent upsurge in traffic came from abandoning the aggregation gold rush. And ProPublica has figured out how to win Pulitzer Prizes and remain financially viable (PDF) without buying into the hype.

Maintaining credibility

As Philip Napoli notes in his book Audience Evolution, a large audience may not be the most engaged. In the age of social media, a small but motivated group can make a big noise.

But the information has to be accurate. Witness the backlash regarding Invisible Children and the #stopkony campaign. After the initial rush of eye-candy addicts, more thoughtful writers began questioning the group’s methods, spending, and accuracy.

Even credible, well-read organizations such as Mashable fall into this trap. The all-things-social-media site recently posted an item with the misleading title “Top 10 Social Media-Savvy Universities [STUDY].” As you dig deeper into the study, you find that the list is actually drawn from already culled list of 25 UK universities and 25 U.S. universities.

Even the study’s author notes this clarification in the comments of the story:

Journalism’s value/impact can come from reaffirming its role as the verifier without agenda. Places such as FactCheck.org and PolitiFact serve as role models for our aspirations of verification.

Measuring verification

Measurement assumes quantification, and some ideas — such as verification — are better evaluated qualitatively. Creating a measure requires including some attributes and excluding others; inevitably, such measures are always imperfect approximations, especially when it comes to complex concepts.

But we can establish a few expectations, with a little help from Kovach and Rosenstiel:

• Transparency: Where did the information come from? Are you transparent about your sources? Post your spreadsheets through Google Docs and your documents on DocumentCloud. Embrace an open-source ethic. Disclose what you were unable to confirm.
• Consistency: Test information from every source, even your mother. Don’t rely exclusively on secondary sources; check the primary information yourself to ensure it is presented accurately with context.
• Context: Link to sources you’ve used to verify your content, and provide access to full interviews to bolster confidence that you’re not cherry-picking information to support an agenda. Make sure you’re representing all nuances and resisting the urge to simplify everything to two sides.

Italian schoolteacher Tommaso De Benedetti turned humilating news organizations into an art form with his mock Twitter accounts of Harmid Karzai, Bashar al-Assad, and other world leaders. He found that some news organizations quickly retweeted information without verification — including news that Fidel Castro had died.

In his interview with The Guardian, he may have created the journalistic warning for the Internet age: “Twitter works well for deaths.”

Or maybe we should craft a new adage:

If your mother tweets her death, check it out.