A framework for measuring social media

Do you think it's tough to measure social media?  It might be simpler than you think.

I've blogged before about how game mechanics describe social media participation. Games have scores.  Scores track progress, which results in success or failure.

Although social media channels seem to be mostly qualitative in nature, user activities can be easily quantified.  Although users interact with channels in different ways, four common factors quantify social media success:  Attention, Participation, Authority, and Influence.

Here's a framework for measuring social media:

1. Attention.  The amount of traffic to your content for a given period of time.  Similar to the standard web metrics of site visits and page/video views.
2. Participation.  The extent to which users engage with your content in a channel.  Think blog comments, Facebook wall posts, YouTube ratings, or widget interactions.
3. Authority.  Ala Technorati, the inbound links to your content - like trackbacks and inbound links to a blog post or sites linking to a YouTube video.
4. Influence.  The size of the user base subscribed to your content.  For blogs, feed or email subscribers; followers on Twitter or Friendfeed; or fans of your Facebook page.

There's an "x-factor" that comes into play well:  sentiment.  The spirit driving user participation matters.  The net result of these adds up to a score for social media engagement.

So what's the monetary value of a visit, comment, link, or friend?  Well, the only honest answer is "it depends."  Only you know how much these interactions matter to your brand, regardless of industry, channel, or competitive results.

Now...was that so tough?

Comments

[Dave Mattia]

"So what's the monetary value of a visit, comment, link, or friend? Well, the only honest answer is "it depends." Only you know how much these interactions matter to your brand, regardless of industry, channel, or competitive results."

This has been old media's position for decades (minus the nuances of interaction). In a recessionary environment ROI demands are increasingly defined. So while my position is that cooperation and trust are what makes social media so powerful, how do you ultimately quantify that? Higher quality metrics seems to be the answer but at a time when consensus is so hard to come by in any vertical, how do you get them? Cooperation and trust is the answer on a few levels for measuring effectiveness. The cost of media 2.0 is sharper definition by consumers and producers of content. How do you create enough trust and cooperation to get there? I do not think there is any doubt that social media will show us.

[jon b]

While I agree with your framework as a whole, I would caution against a one-size-fits-all metric. These factors can be manipulated or combined to generate unique metrics that demonstrate various aspects of your brand or campaign.

For example the sales figure in finance can be arrived at using many accounting techniques. However, that figure it itself is often further utilized in secondary equations to deliver insightful and insight generating performance scores and indexes.

This is the cause of much of the confusion around quantifying social "media" via a singular metric. We need a set of metrics that include all of the above mentioned criteria, and possible a few others including expediency/virality/timing, relative spillover or unrelated conversation, penetration of brand presence against total relevant audience, timing of this penetration, search trending (as search is still a gateway to much social content), and social as part of a broader communications plan.

Social is one piece of a pie, one that is often bolstered by external communications.

Performance can be boiled down to a dumb metric, but insight will only come from a series of metrics that reflect and support the social ecosystem, the brand plans and goals, and the market as a whole. It's going to be fun, but in the meantime, I would say that the metrics you've proposed set a great foundation on which we can build moving forward.

[Jackie Huba]

Peter,
I like your framework. But I do think that the 4 elements are weighted differently for different people/companies.

I've spoken to a number of people who don't care about "attention" aka page views, etc. They are more focused on subscribers, building a larger network. They don't write Digg-bait blog posts or create "viral" videos.

Personally I think "attention" is overrated. If you have to focus on achieving one metric above all the others, I would opt for building up your base of people who want to hear from you on a regular basis. This the holy grail of loyalty.

Then perhaps start working on the participation goal.

[Jay Deragon]

Interestingly enough I just posted the three "A" factors to successful use of social media. Close to what you've written but a different slant. see http://www.relationship-economy.com/?p=1952

Attention
Attraction
Affinity

[Chel]

Great framework for companies to judge the effects and the worthiness of social media for their company.

From my POV because the value is restricted to each company's determination of what's valuable to them, there's not really a way to created standardized metrics for social media in a way that will work for every single company.

The framework however, will answer some questions if a companies should even consider getting involved in social media. Their goals should determine their direction, instead of speeding down the road because it's what the cool kids are doing.

[Ryan Moede]

Peter - interesting post. While I think digging deeper into each factor will be important, I like the idea of beginning with a framework that can be realigned and focused for varying companies and their needs.

[Taylor Davidson]

Great start, but I think the difficult part is not in measuring social media, but in measuring the impact and linkage back to financial / operational results (i.e. ROI). It may be hard, but it's where the opportunity exists.

Fact is, most of the traditional ways we try to measure and link marketing efforts back to financial results simply don't fit in this new environment. The metrics we see easily in social media are valuable, but they simply don't capture what's really happening.

[Peter Kim]

Hey - I wish I could've jumped in sooner, first time I've been online all day.

Marketing measurement is tough - we all know that. On the complex end, we have correlative measurement models, which are sometimes more art than science. On the overly simplistic end, we have basic action/response models, which don't capture the detail that Jon mentions.

What I've attempted to outline here is a basic framework, based on game mechanics, to help people understand how to create a baseline for activity and build from there...would love to see someone take this the next step further.

[Tom O'Brien]

Hi Peter:

I like your list, but it is missing my favorite metric. Brand Advocacy. Are people recommending your brand to others or not?

If they are, it will show up in your sales or market share - and that means real ROI.

TO'B

[C.H. Low]

Love the framework. Thanks. I'd also like to add Knowledge as another "x-factor" type element.

The social interation adds knowledge that can help to make good decisions. And knowledge is not necessarily quantifiable. It could be a new fact. It could also be improved confidence on business assumptions you are working with.

After following so many conversations on this topic, I am beginning to develop a feeling that Measurement of Social Media may remain largely (not just partly) an art rather than science that includes all of the elements of the framework discussed here. Some parts can be measured but I think most will be hard to quantify.

And the many suggestions that the measurement is different for everybody also leads me to my feeling that there may never be a complete science for Social media ROI. My opinion is based on the premise that social media is about relationships. Relationships is a state of mind not a metric.

I saw a presentation on Quantifying ROI a few weeks ago and the presenter suggests that even though it is hard, you have to have metrics and assign values even if it is ARTIFICIAL so that you have a forcing function for decisions on allocation of scarce resource and even if you cannot prove the causality or value assigned. To me, working with bad numbers is not a solution. Art and instincts may be the core foundation to decision making for social projects. Yes, this makes it hard to justify projects in most traditional decision making process in companies. Perhaps, those decision processes need to change to the new environment rather than the overly focus on effort to develop the nebulous metrics to fit into the archaic decision processes.

Have you ever thought about what is the ROI of your relationship with your church, charity, even your colleagues? Kind of hard to quantify. But you'd know if it was worth your time to be engaged with them if you FEEL in the end there was good outcome.

So my current feeling is that in the end, ART (especially when involving the X-factors) may always be the larger part of the Social media ROI measurement equation.

Maybe we need to work on defining a new decision making process not a new metric. Just thinking out loud!

[Christine Morrison]

Interesting post, great comments. I generally agree with Jackie Huba about networks being the holy grail of loyalty and advocacy. For me the holy grail has gone one step further: I want to measure latent conversion among those who are recommended *to* by our promoters via social networks/online conversations, and not just in the "canned" environement of a specific campaign. So far I can't find any tools to let me do that, but I have high hopes for the future!

[Gianluigi Cuccureddu]

Great four points as a kind of 'Return on Engagement'.
I do miss Word of Mouth, or like Tom o'Brien said, Brand Advocacy.

[owen mack]

great outline peter-- i love reading this kinda stuff, 'cuz i'm trying to figure out how to charge for it everyday. as a professional social media producer/consultant, what i see is that right now brands just say "i have X dollars, see what you can do" and they're not paying for specific interactions. the interactions are the "viral" component that everyone wants for free. measurement of that stuff lets the numbers guys feel good about money spent, and there are various games being played to get the number up there...
In the end I agree with C.H. above-- it's more of an art than a science, because the companies that are doing this stuff well devote ALOT of man-hours to one-on-one relationships. That's why so much credence is given to "authenticity": it's something we feel emotionally. Science is not emotional, it's cold. Not art.

[Bjoern Negelmann]

Hi Peter - I thought about your framework and discussed some additions on our Social Web WORLD blog - I wrote it in English and would love to have your thoughts on this. Regards, Bjoern.

[Augie Ray]

This is a fine framework, but it doesn't exactly break new ground. These are the same metrics used by all sites--traffic, interactions, inbound links, and subscriber database.

My concern is that while this gives you some basis for measuring success over time, it still doesn't come close to providing any sort of Return on Investment (ROI.) Marketers are coming under increased pressure to prove the value of what they do, and I don't believe telling the CMO that you spent $250k to increase traffic from 5k/day to 7k/day and add 1,000 inbound links won't really cut it.

You're "it depends" is where the real magic must be discovered in order to validate investments online and in social media.

Augie Ray
http://ExperienceTheBlog.com

[Peter Kim]

Augie, Taylor - that's exactly the point! The metrics are all able to be used today. I think you miss the point about the "it depends." The value of an inbound link may be $10 to one company and $1 to another. Building an ROI model is easy (IMO). Figuring out how to bridge from a "friend" to a dollar can appear to be difficult, but that's why I wrote this post. Then again, I'm comfortable with discounted cash flow models and monte carlo simulations, so building ROI models seem pretty simple to me. You?

C.H. - yes, there's the known, the knowable, and the unknown. The unknown would be the quality of relationships with no tangible support - thus waiting for something trackable to materialize (e.g. a post, a tweet, a fan), moving into the knowable and then known.

[Michael Cayley]

I like thinking about measuring social media in the same way that you keep score in a game.

I think the important shift in measurement that is taking place is that instead of measuring media, we are beginning to understand media as artifacts of actions. What is important is that we measure the resources or results that the social connections represented by the media deliver.

I think we are pointing in similar directions. You are suggesting a new scoreboard. In Social Capital Value Add, I am suggesting that we use an established scoreboard that already has meaning in the boardroom, i.e., a new school of value based management that is designed for the networked age.

Your approach is rooted in game mechanics and I have build SCVA upon the established approaches of social capital and social network analysis academics.

I hope that you will check it out and help develop the ideas.

[Jacquelyn]

Peter, I'm working on a paper and wondering - do you have any examples of particularly memorable academic research that points to these factors being preferred? Any information would be great!

PS: Michael, I'll likely reference yours too...

[Peter Kim]

@Jacquelyn - I'd direct you to my colleague Kate Niederhoffer, who could probably give you some suggestions: http://socialabacus.blogspot.com/

[Jacquelyn]

Oh great, thank you so much!

[Katie Delahaye Paine]

Interesting string of comments and a great discussion. First of all, most of the outcome measures are tied to either regular marketing analytics or web analytics. I'm stunned at how many people aren't talking to their web guys about this data. You can use unique URLs or basic multivariate testing to get evidence of what consumers are doing with the site. However, the problem with your suggested model is that most of the conversation isn't happening on your blog or your facebook page, it's happening somewhere else and you need to decide what kind of conversation you want to be a part of. There are basically 27 different kinds of conversations that take place - whether in your living room or on Facebook -- and you need to understand how people are talking about you. In some cases, organizations don't want to be mentioned at all -- making your "it depends" even ore important. We advocate for a "optimal content score" where an organization decides for itself what an "optimum" conversation looks like and a "worst case scenario" looks like and then you rank each posting and each comment along that continuum. Just a thought.