David Armano of Critical Mass, author of an excellent blog on social media - what he calls micro interactions - gives us his perspective in a podcast interview (10 mins) on ways to measure in social media, and how to judge ROI.

(P.S. Because I couldn’t figure out how to work the CallBurner software the conversation ends with a namecheck for French philosopher Jean-Francois Lyotard - coiner of the term “grand narratives” - whose idea of “petits récits” has some resonances with David’s micro interaction idea)

To get a better understanding of what constitutes best practice in social media measurement and evaluation, we thought the simplest thing to do was to “eat our own dog food” and use our own social media measurement tools to determine who had influence on the topic of “social media measurement” and then look in detail at what they had to say. We wanted to know who owned the consensus around the topic.

First we used Market Sentinel’s proprietary web crawler to undertake a MarketInfluence study. In this case, a “stakeholder” is any individual or organisation who is mentioned in the context of “social media measurement”. Our web crawler finds these stakeholders by following a three step process:

1) Search the web and identify documents (web pages, Word, pdf or PowerPoint documents) that reference “social media measurement”

2) Search those documents for references to other documents

3) Visit the referenced document and determine whom they reference

The analysis is focused on identifying who references whom. The process continues until the context is lost. When all possible citations have been found we perform a “citation analysis” to determine who in the conversation is

a) most popular (i.e. most linked in the context) b) most influential (i.e. most linked by those who themselves are most linked) c) more central to the network

Citaton analysis has been used since the sixties to calculate the authority of academic journals. Here, we use it to analyse web documents for references. For example, a document by Seth Godin refers to a document by BBC News in the context of “social media measurement”, then we take it that Seth Godin deems BBC News to be relevant to the issue. It also means that, on average, BBC News to some extent influences Seth Godin. References like these are turned into systems of mathematical equations which we can solve to determine “popularity”, “influence” and centrality.

Our influence metric works much like that of academic journals, where academic authors cite the works of other academics. They mainly do this because they believe that the articles they cite are relevant to the context. In doing so, they point to other publications that are relevant to their topic thereby revealing which publications have influenced them. We use a similar approach to measure influence in social media measurement.

At the heart of this type of influence measurement is a simple idea: Person X has influence on Person Y regarding a particular issue if Person Y is dependent on Person X for information or to support point of view about the issue. For example, BBC News influences Seth Godin regarding “social media measurement” if Seth Godin depends on BBC News for information. Seth Godin may demonstrate this dependency by linking to a BBC News article in a blog post. Our influence measurement works by searching relevant documents for such links and transforming these relationships into a system of equations which we then solve as a relative measure of influence.

What emerges from this process is a league table of influencers, which might be compared to Google’s pagerank, except that it is topic-specific, a network map showing how these speakers or entities are linked, and various mathematical measures showing how their status in the network can be determined.

The value of this procedure, which Market Sentinel has used many times for its large brand customers, is to achieve a God-like understanding of any given conversation, who is involved in it and how it is articulated, and to understand where the major themes in that conversation spring from and can therefore be influenced.

Here is the leage table of influencers, showing who is most authoritative on the topic of social media measurement/evaluation. [Update in response to Katie Paine’s comment] The analysis was completed during August 2008 and updated earlier in October.

[Click for full table]

In calculating influence, we obtain a “virtual social network” which shows how the top influencers influence eachother. The map helps us determine if any stakeholder groups cluster together in interesting ways.

(Click for full-size network. Please note that the figure does not show all the stakeholders, but just a very detailed close-up of the centre of the influence network. If all stakeholders were included it would be virtually impossible to identify individual stakeholders.)

The size of each stakeholder’s marker indicates their relative influence.

The network maps are constructed using an approach called “minimum spring force layout”. Imagine the relations between the nodes are springs and that the stiffness of each spring is proportional to the strength of the relation. The nodes are then arranged so the total force used to “stretch” all springs is at a minimum.

The main advantage of the network map is that it clusters stakeholders based on the strength of their influence relations. The closer two stakeholders appear on this map the more they influence each other, even if that influence is indirect via one or more other stakeholders. This gives an idea of how closely related individual stakeholders are.

So what’s the story here? Google is here because of its omnipresence in matters internet-related. It has also recently applied for a patent for applying a version of pagerank within social networks. YouTube, Facebook, Technorati and Yahoo! are prominent for similar reasons: a combination of mindshare in the context (particularly true of Facebook, where the lack of metrics is a major issue) and of relevant content.

The story thereafter is of a conversation dominated by Forrester, whose Charlene Li (she left in 2008, but has been consolidated in this data) has been extremely important in atomising the debate around the development of social media and whose “Groundswell” encapsulates many of the great cases studies of the last 3-4 years. Her co-author on the book Josh Bernoff maintains the Groundswell blog. Next is Jeremiah Owyang. He has not been consolidated with Forrester because he was part of this conversation in his own right before he joined them. Jen McClure’s Society for New Communications Research has - equally - published a number of great resources on the topic. Katie Paine is a guru of PR measurement in all its flavours and so finding her here is no surprise.

Jeff Jarvis of BuzzMachine and Robert Scoble are here in part because of their relevance in two of the stories that defined the era which witnessed the arrival of social media: Dell Hell (the story of Dell’s failure to comprehend social media and then their enthusiastic conversion to it) and in Scoble’s case the opening up of Microsoft to the outside world (symbolised by Scoble’s work) and Scoble’s own subsequent career as a “star” of social media in his own right. Scoble’s co-author Shel Israel’s Global Neighbourhoods follows hard on Scoble’s heels.

The presence of Seth Godin (marketing guru) and of Steve Rubel (PR guru) demonstrates the twin poles around which the area of social media evaluation revolves. Marketing folk need to understand how people have responded to their campaigns, public relations folk want to understand key conversations and influencers with whom they can connect.

Actually, they have interests that are increasingly identical and interchangeable, but marketing and PR stubbornly refuse to converge, partly because they employ such different methods. Marketing people create assets (campaigns, websites, virals, creative) and PR people create conversations, person to person engagement. The pressure of social media’s rise is forcing the two disciplines to borrow increasingly from one another’s toolkit, but they are not yet in the same world.

As we examine the measurement methodologies each apply it will be interesting to establish how much they are beginning to ask the same questions of the data. How many responses received? How many conversations started? What is my awareness level? How many products have I sold?

An interesting sidelight on this story is provided by a glance at the list of the top ten connectors. This group is defined by their characteristic of disproportionately citation of the most authoritative sources.

1. Jeremiah Owyang
   
2. Constantin Basturea
   
3. Josh Hallett’s Blog
   
4. Livingston Communications
   
5. Forrester
   
6. Peter Kim
   
7. Kami Watson Huyse’s Blog
   
8. Shel Holtz
   
9. Market Sentinel
   
10. Nathan Gilliatt
    

These folk offer a kind of directory of quality sources in this context and I suppose this blog post continues Market Sentinel’s work in this area.

In our next posts, we will look in detail at the ideas of individual stakeholders about how social media should be measured and see if we can define a growing consensus.

Well, here is a concrete example of the return on investment from monitoring and responding to social media. Last year our client Cadbury relaunched the Wispa bar in response to a campaign for its reintroduction, advised by us. Their UK sales are up 11% over the quarter, global sales up 6%.

How do you build brands post-Google?

As the recession begins to bite, brands are finding that getting through to customers is tougher than ever.

Offline advertising is showing diminishing returns. McKinsey predicts that by 2010, traditional television advertising will be one-third as effective as it was in 1990.

This is partly because online media is growing at the expense of offline. A UK survey by Media Week showed that time spent for both live/realtime TV and teletext tv decreased by 1% and 2% respectively between 2006 and 2007, while internet usage by 50%; an IDC study of U.S. consumer online behavior found that the Internet is the medium on which online users spend the most time (32.7 hours/week). This is equivalent to almost half of the total time spent each week using all media (70.6 hours), almost twice as much time as spent watching television (16.4 hours), and more than eight times as much time as spent reading newspapers and magazines (3.9 hours).

It is partly because of the rise in ad avoidance strategies. DVRs owners (according to an IBM survey) watch at least 50% of television programming on replay, thus avoiding television advertising. It is partly because of a major decline in public trust in brands. People trust people more than they trust the media. In a 2006 survey of U.S. consumers, Forrester found that 83% of respondents trusted friends’ opinion, but only 75% trust product reviews in a newspaper, magazine or TV. (Groundswell, Charline Li and Josh Bernhoff).

The answer would seem to be to move the business online. More than half of the world’s Internet users have made at least one purchase online in the past month, according to Nielsen. The web also seems to offer promising growth for advertisers: Nielsen estimates that spending on online advertising will escalate at 19.2 per cent annually till 2012 and will surpass the TV advertising budget in the US in the next decade.

But advertising on the web poses challenges. Online banner click-throughs on Yahoo!, Microsoft and AOL have declined from 0.75% to 0.27% according to ad monitoring firm Eyeblaster.

Paid search ads now represent the lion’s share of online ad spending. Contextual search ads are great for selling specific factual propositions (flights to Malaga, hotels in Brussels) but they are less effective at communicating emotion. In a recent report from The Wharton School, marketing professor Patti Williams observes that it’s unclear how a company like Crest can leverage search advertising: “How many people are going online to search for toothpaste? It’s not [obvious that] a little ad on the screen is going to attract them. For the biggest bulk of media spending, online is just hard to figure out. The Internet is not that good at big brand-building objectives, so there are a lot of companies struggling with a way to take advantage of the tremendous opportunity Google and other searches offer.”

A 2007 global Nielsen survey found that consumer recommendations are the most credible form of advertising among 78 percent of the study’s respondents. And there are perhaps clues for advertisers in the shift of online consumers to social networking sites. In the UK, social networks overtook webmail by percentage of visits in 2007, with social networks accounting for 5.17% of all Internet visits compared to 4.98% for email services. Advertisers want to follow consumers but that’s difficult. When you are chatting to a friend the last thing you want is to be interrupted with a clumsy brand message. Privacy settings in most networks preclude direct marketing. Facebook recently announced that it was opening up key pages to allow for contextual advertising.

So how do Brands engage with the consumer in a way that provokes conversation and endorsement? The most successful strategies for engagement with social media is for a brand do something which allows people to pass on a key message about your brand.

People can talk about you for three reasons:

You have given them useful information.

H&R Block set out to build awareness of their online tax return offering by creating content customised for channels. The budget was 5% of their annual digital spend only 0.5% of their total ad spend. They grew awareness 52%, and saw an 11% growth in tax services business, feeding net income which rose to $544m from an $86m loss the previous year.

Giant Food Stores increased monthly consumer website visits by 400% after lauching a “Super Shopping List”, which lets customers easily browse recipes, view weekly specials, and create a personal shopping list.
Brand discussion goes beyond the product itself. The entire process and value system around which a product created is also a source of conversation.

You have entertained them.

The hugely popular Cadbury Dairy Milk campaign which featured a gorilla playing the drum solo of Phil Collins’ “In the Air Tonight” received 7m views online, more than 6,000 comments and boosted Cadbury revenues by 5% for that year. (Gorilla ad works its magic on sales of Cadbury bars ).

This jokey video from Philips Norelco Bodygroom raised the issue of persuading men to shave “below the neck” in the summer 2006. The video (cross-posted at youtube.com and at heavy.com) has been viewed 1.8m times, it boosted unaided awareness 8% and contributed to year-on-year growth of 17% for the DAP division (of which shavers represent 45%) to Q1 2007.

There is something in your product that they respond to.

Jeep’s “Have fun out there” website aggregates communities from where they already exist, such as Facebook and Flickr, to create its own uber-community where members drive the content.

The t-shirt company Threadless has used community as a mode to build its business by allowing members of the website to submit and vote on t-shirt designs. The top designs are selected for printing and sold through an online store with winning designers getting a cash prize and store merchandise. What started as a hobby in 2005 by founder Jake Nickell has been growing quickly with annual sales on track to hit $5 million in 2008.

The last is the best because it means that consumers have engaged with your brand and are doing your marketing for you. With the additional benefit that they are marketing to people who are inclined to believe their testimonials. It is also the hardest to achieve. New online measurement techniques (such as those used by Market Sentinel) offer the opportunity to chart how effective brand building in online by directly measuring response to creative campaigns, by gauging consumer engagement and by changing the creative to take account of live consumer responses. But how do you measure such responses. A consensus about this is only now beginning to emerge and we will deal with this in our next post.

H&R Block used social media marketing to boost their profile and raise awareness of their digital accounting product, reports Ad Age. The lady responsible was Amy Worley (Photo: Jonathan Fickies). They used YouTube, Facebook, MySpace, Twitter and Second life. As AdAge comments, this kind of marketing in social media is “about stacking up many small ideas to create a big total impact”.

Key stats:

H&R Block boosted overall brand awareness by 52%.

They spent 0.5% of their ad budget in doing so.
…

Yes, 0.5%`

[UPDATE] More details on the H & R Block campaign including an interview with Paula Drum, VP of Marketing on Podtech’s Marketing Voices talking about how the campaign played on YouTube and SecondLife. Here is their social media site Digits.

A lot of brands are dipping their toes into social media marketing. They naturally want to know what ROI they can expect. For that reason we are delighted that our client Avis UK has gone public with the results of their 2 year experiment in monitoring and responding to online conversations. They are experiencing double digit growth in a sector (car rental) that is growing at 1% per annum. They attribute much of this to their online actitivity, which shows that social media marketing can have a direct impact on ROI (details here). Kudos to Xavier Vallée, Rob White and their colleagues and thanks to our partners at Web Liquid - Avis’s digital agency.