Predicting Super Tuesday winners with brand monitoring

I've seen a lot of brand monitoring solutions that claim to do a lot of things.  Many vendors say they can predict future events based on chatter levels.  Most often, the backup happens in hindsight, which you and I both know is 20/20.

But this brand monitoring vendor Collective Intellect is doing something different - they are publicly releasing projections of US Presidential primary outcomes, based on analysis using their technology.  Why does this matter?  Because the outcome is publicly available so you'll know whether they were right or wrong.

So how did they do on Super Tuesday?

Democrats:

• California.  Prediction:  Obama.  Outcome:  Clinton.  Result:  Incorrect.
• Colorado.  Clinton (close).  Obama.  Incorrect.
• Georgia.  Obama.  Obama.  Correct.
• Massachusetts.  Obama (close).  Clinton.  Incorrect.  [Oddly enough, I haven't seen much Clinton support around Wellesley, the town that shares a name with Hillary's alma mater.]
• Missouri. Obama (close). Obama. Correct.  Very close race.
  

Republicans:

• California.  McCain (landslide).  McCain. Correct.
• Colorado.  Romney (landslide).  Romney. Correct.
• Georgia.  McCain (landslide).  Huckabee. Incorrect.
• Massachusetts.  McCain (landslide).  Romney.  Incorrect.  I guess we still like Mitt here.
• Missouri.  McCain.  McCain.  Correct.

Overall, the predictions went 5/10 (results based on CNN projections).  Could have been better, but marketer decisions typically aren't as black & white as this and more directional.  Credit for putting your neck out there, Collective Intellect.

Contrast this with brand monitoring predictions around the Super Bowl.  All hindsight.  Will anyone else prove their system publicly?  Next American Idol winner?  New hit TV shows?  The next President?  Or does this show that these systems aren't ready for prime time?

Comments

[Jim Nail]

Agreed -- kudos to Collective Intellect for sticking their neck out. But it seems 50/50 results show there is a lot more work to do on figuring how to use brand monitoring predictively.

Before we get to that stage,I think we still have to show that social media derived data isn't hopelessly skewed and unrepresentative. To that end, TNS did a traditional ad effectiveness study of the Super Bowl ads which is up on our site http://cymfony.blogs.com/superbowl/2008/02/tns-reports-on.html

In comparing the TNS effectiveness list with Cymfony's list of most-talked-about brands, is interesting. And one of the findings seems to be that the pre-game PR program is a key variable in predicting whether the ad will get talked about. It also shows that an effective ad doesn't necessarily generate WOM.

Read more here: http://cymfony.blogs.com/superbowl/2008/02/how-does-social.html

[robin seidner]

Hi Peter,

Thanks for publishing our results. As we mentioned in the report, this was a methodology that we tested for single state races, which we were correct on for all but 1. The 20-state match up was a different kind of thing altogether, so we were curious how it would work in that case. Not too bad. To me, its most interesting that we were more right on the republicans, as there are less right-leaning than left-leaning blogs.

Certainly, this isn't the typical kind of prediction we do for our customers, but was an interesting experiment . The polling firms ought to conclude that they could be helped by more data from different kinds of sources.