#jcarn: What is the best way to measure meaningful content?

David Cohn, our Carnival of Journalism ringmaster, has graciously allowed me to serve as host for this month’s #jcarn blogfest.

For this month’s prompt, I offer two related questions:

  • How do you define meaningful content that has long-lasting value?
  • What is the best way to evaluate content that fosters deep engagement with the audience?

Take the prompts in whatever direction you wish. I’ll start gathering the results the week of June 25 and post a compilation post the first week of July.

I’ve jotted down a few thoughts below.

Today’s media environment is a flood of immediacy.

With the focus on virality, content creators frantically update to ride the waves of Google, Bing, and Yahoo. They hope their content will light up the social-media landscape on Twitter, Facebook, and Google+. They crave the buzz that may last an hour or two or perhaps even a day, if they’re lucky.

To that end, they often gauge success in terms of page views or unique visitors, largely because today’s advertising metrics are still focused on exposure. It concentrates on the actions of the audience, instead of the inherent quality of the content itself.

We default to these measurements because they make sense. They are something that we can move. And they are something we can tangibly connect to revenue.

News Corp.’s Raju Narisetti makes a persuasive case for the pragmatic reality of metrics that — as he puts it — “drive conversions”:

 …“guests” (your one-and-done type visitors) to “readers” (often registered but not paying) to “subscribers” (paying readers) to “members” (those who avail themselves of other non-content led benefits of subscribing.)

These metrics remain a work-in-progress in various News Corp newsrooms globally but, eventually, will be the primary focus of how we need to manage our newsrooms and news publishing companies when it comes to audience data.

But exposure is limited in what it tells us. Most engagement models (such as the definitive one from Philip Napoli’s book Audience Evolution that I’ve included below) place the concept at the beginning stages of audience connection and participation, as measurements such as page views let us know only whether visitors saw the content. They don’t reveal deeper reactions.

Screen Shot 2014-05-28 at 8.40.49 AM

What is engagement?

Screen Shot 2014-05-30 at 6.55.41 AMMy research partner Dr. Carrie Brown-Smith and I have spent years working with traditional newsrooms moving to the digital realm, and we’ve chronicled a clash of ideals related to the energy spent on generating large numbers of page views.

Since we first connected with the Christian Science Monitor in 2009, we have heard reporters and editors struggle with exactly how to define engagement. Is it loyalty in the form of return visits? Is it regular visitors to the home page? Is it someone who shares content elsewhere as a word-of-mouth advocate?

As one editor told us during one visit:

What’s interesting is that the SEO side is a lever you can pull. You know how to work it, or we’ve figured out how to work it. We know if we do X, then we’re going to get Y, just like a machine, spit it out. We have no idea how to do the other one, so we’re just blind and fumbling around in the dark. And I think sometimes there’s a frustration that comes with that, so when in doubt, go back to what you know. But, yeah, if we could pull that lever, that would be great. I’m not sure we know how to do it.

We are not the only industry watchers who have sensed a critical point in the conversation on metrics. Poynter’s Rick Edmonds noted in March that several corners — including the metrics firm Chartbeat — have begun exploring concepts that better capture what it means to have an engaged user.

Nicholas White, editor-in-chief of the Daily Dot, told an audience at this year’s International Symposium on Online Journalism that the site doesn’t look at traditional metrics such as pages per visitor anymore. “We’ve gotten past (that),” he said.

Authors Henry Jenkins, Sam Ford, and Joshua Green dug into this idea with their notion of “spreadable media,” from their book of the same name. They took issue with the label “viral,” which removes the human component from the action of sharing.

Their definition:

“Spreadability” refers to the technical resources that make it easier to circulate some kinds of content than others, the economic structures that support or restrict circulation, the attributes of a media text that might appeal to a community’s motivation for sharing material, and the social networks that link people through the exchange of meaningful bytes.

Our use of “spreadability” is perhaps most effective as a corrective to the ways in which the concept of “stickiness” has developed over time to measure success in online commerce.

Another important distinction the authors made: Sharing doesn’t necessarily mean engaging. Perhaps users shared content to reveal something of themselves. Maybe it’s the headline that led them to post the article on their Facebook pages or in their Twitter feeds, and they did not truly engage with the content.

Thinking about longevity

For a while now, I’ve been mulling these related ideas and thinking about a concept that I recently started calling “longevity” — a piece of content that people return to repeatedly, whether for reference, connection, or enjoyment. It consistently draws users over weeks, months, or even years. It touches them on all levels: emotional, cognitive, and perhaps even spiritual.

I first began thinking about this notion watching viral tweets, trying to determine what made certain tweets survive for more than a couple of hours through retweets and sharing.

I also noticed it with certain useful blog posts I had written over the years. A post I wrote explaining the Twitter abbreviation “MT” in 2011 continues to drive hundreds of page views to Drury’s Social Media Certificate site every month.  I believe part of the reason for its success is it explores the culture of Twitter, beyond the practical explanation of “MT.”

The label longevity hit me earlier this year, as I was looking for a simple primer on project management for my students. My searches took me to the Microsoft Developer Network, where I found a superb treatise from Scott Berkun — written in 2005. Even nine years later, Berkun’s advice remains relevant and meaningful in today’s age. And it remains on the first page of search-engine results in response to the query “how to manage a project.”

I began to wonder: Is it possible to create such long-lasting, meaningful content in a news context? Does everything have to be about immediacy? 

How do you think about meaningful content? Does it mesh with these definitions? And how might we define and measure it?

The phases of engagement and the best answer ever

Since 2009, my research partner and I have been studying the changes at the Christian Science Monitor in its efforts to morph into a Web-first newsroom.

Its success in increasing page views has been well documented. Using search-engine optimization and frequent news updates, its site regularly tops 30 million page views a month, and the organization has earned a spot among the 250 most visited U.S. sites on the Web.

But during our research visits, we’ve often heard reporters and editors struggle with how to transform the drive-by SEO audience to one that is more engaged, more invested in the Monitor itself.

Many other news organizations, marketers, and Web producers have become obsessed with this idea as well. As someone working with the next generation of content creators, I have dedicated much time developing my own model to understand how engagement happens.

A simple model

For years, I’ve been attracted to the uses-and-gratifications model of media behavior, which simply states that people use media to satisfy certain communication needs. But how does this process evolve? And how do we build regular usage and make our sites a media habit?

Integrating several threads of innovation and media research, I came up with four primary phases:

Engagement

  1. ATTENTION: It begins with an appreciation for SEO and messaging. Do we have something that will grab people’s attention from the mass of information already available to them?
  2. USAGE: Ease of use is vital for luring new users. Usability experts like Jakob Nielsen regularly remind us of the importance of clean, responsive design, and if a site/app/program is too difficult to use, users often will abandon it, especially if they are not emotionally attached or invested in the content.
  3. HABIT: Once users have overcome the trepidation of trying something new, we need to use consistent content to keep people coming back. And that consistency should come in quantity (frequency of updating) and quality (meaningful, well-produced content). It is finding the right balance between quantity and quality that so many organizations struggle with. This part of the process is more transactional and informational: Are we providing the type of content users need to satisfy what they’re looking for?
  4. COMMUNITY: After securing the intellectual commitment, we must establish that emotional investment in our site by building community. It is this phase where so many news organizations and businesses fail, and the social media succeed so dramatically. To develop this human connection, we must be willing to share our site with the audience. Allow them to contribute photos, comments, and other user-generated content. Make them feel a part of the site.

I find this simplified model — which incorporates ideas from diffusion of innovations, the theory of media attendance, the Media Choice Model, and engagement research — resonates more effectively with my students than more complex formulations (such as Philip Napoli’s definitive  conception of engagement in his book Audience Evolution: New Technologies and the Transformation of Media Audiences).

The best answer ever

I teach this model to my students in my Web Communication class, a junior/senior level class in our Multimedia Production and Journalism program, and a question about the idea inevitably appears on my exams. As I tell my students, I am most concerned with concepts rather than terminology; I want them to articulate clearly what’s happening at each stage, rather than just regurgitating my lecture blather.

And so, this past semester, one student took me at my word.

I share this answer with her permission. (The spelling-error comment refers to an error on her own site that we discovered during her final presentation.) Enjoy.

  • First: Discovery of a site. “Well, hello there,” its viewer purrs. “What have we here? Is this worth my time? Is it going to play games with me and break my heart with a lack of updates, shoddy information, or spelling errors (that may very well be tragic mistakes and not actually errors because son is actually a word, just not the right word)? Hmm. Let’s take a look…”
  • Second: Getting to know a site. “That’s a snazzy nav bar,” its viewer giggles. “And I can’t get over how nicely this content is organized! Holy cow, those tags group things well! It seems like I can find exactly what I’m looking for before I even start looking for it.”
  • Third: Regular use of a site. “We’re in it for the long haul,” its viewer promises. “You and I are an item now. You’ve proven your worth, and I want to commit my valuable Web time to you.”
  • Fourth: Interacting with a site. “I’ve gotten up enough courage to make my presence known to you,” its viewer proclaims. “I am going to post in your comment section and like you on Facebook and re-tweet your tweets to show the world how much I’m devoted to you. I’ll shout your existence to the world.”

#jcarn: Measuring journalism

Note: This post is another installment for the Carnival of Journalism project, where people passionate about journalism are sharing ideas in the blogosphere about ways to preserve and improve the craft.

This month’s prompt:

What’s the best way — or ways — to measure journalism and how?

The root of journalism is truth, and the time-tested method that journalists have to uncover that truth is verification. If we want to measure journalism, it must begin here.

Respected journalists Bill Kovach and Tom Rosenstiel explored this idea more than a decade ago as part of their work with the Committee of Concerned Journalists, which they codified in their book The Elements of Journalism. As they noted, what sets journalists apart from every other type of content creator — entertainer, technologist, pundit — is verification.

Simply put, verification is checking out everything with multiple sources. I think of the adage I learned early in my journalism career:

If your mother says she loves you, check it out.

It’s only semi-facetious. It doesn’t mean your mother is a liar. It just means she may not be quite accurate.

Maybe she wasn’t talking to you.

Maybe her definition of “love” doesn’t jibe with conventional wisdom.

Maybe she’s not really your mother.

In today’s environment of the immediate and the ephemeral, it’s ever more important to verify the drive-by comments, the frenetic machine gun of news. It’s too easy to retweet, reblog, or republish what’s happening with the claim that the rush of information requires us to be first rather than accurate.

What does the audience want?

What’s clouding journalism today is the need for audience. In such a fragmented environment, how do we generate enough interest — in terms of page views, time spent on site, or other measures of engagement — to secure advertising to pay the bills?

We tap into Google Trends and try to SEO our headlines and content to lure people to our website. We crank up the level of content with frequent updating — sometimes including links to hot content that we haven’t independently verified. The goal is noble: Get them to our site so they can see our real, verified journalism.

But some pockets of experimentation are turning that idea on its head. Gawker has begun finding that original content may be generating more meaningful connections than its SEO junk. Salon recently noted its recent upsurge in traffic came from abandoning the aggregation gold rush. And ProPublica has figured out how to win Pulitzer Prizes and remain financially viable (PDF) without buying into the hype.

Maintaining credibility

As Philip Napoli notes in his book Audience Evolution, a large audience may not be the most engaged. In the age of social media, a small but motivated group can make a big noise.

But the information has to be accurate. Witness the backlash regarding Invisible Children and the #stopkony campaign. After the initial rush of eye-candy addicts, more thoughtful writers began questioning the group’s methods, spending, and accuracy.

Even credible, well-read organizations such as Mashable fall into this trap. The all-things-social-media site recently posted an item with the misleading title “Top 10 Social Media-Savvy Universities [STUDY].” As you dig deeper into the study, you find that the list is actually drawn from already culled list of 25 UK universities and 25 U.S. universities.

Even the study’s author notes this clarification in the comments of the story:

Journalism’s value/impact can come from reaffirming its role as the verifier without agenda. Places such as FactCheck.org and PolitiFact serve as role models for our aspirations of verification.

Measuring verification

Measurement assumes quantification, and some ideas — such as verification — are better evaluated qualitatively. Creating a measure requires including some attributes and excluding others; inevitably, such measures are always imperfect approximations, especially when it comes to complex concepts.

But we can establish a few expectations, with a little help from Kovach and Rosenstiel:

  • Transparency: Where did the information come from? Are you transparent about your sources? Post your spreadsheets through Google Docs and your documents on DocumentCloud. Embrace an open-source ethic. Disclose what you were unable to confirm.
  • Consistency: Test information from every source, even your mother. Don’t rely exclusively on secondary sources; check the primary information yourself to ensure it is presented accurately with context.
  • Context: Link to sources you’ve used to verify your content, and provide access to full interviews to bolster confidence that you’re not cherry-picking information to support an agenda. Make sure you’re representing all nuances and resisting the urge to simplify everything to two sides.

Italian schoolteacher Tommaso De Benedetti turned humilating news organizations into an art form with his mock Twitter accounts of Harmid Karzai, Bashar al-Assad, and other world leaders. He found that some news organizations quickly retweeted information without verification — including news that Fidel Castro had died.

In his interview with The Guardian, he may have created the journalistic warning for the Internet age: “Twitter works well for deaths.”

Or maybe we should craft a new adage:

If your mother tweets her death, check it out.