Can you measure the ROI of customer case studies?

Mixing metaphors, my view is that having REAL metrics on reference material usage, influence and therefore ROI is pretty much a holy grail but also an area in which there are some folks selling snake oil :o). 

Yes we can look at web-stats, make folks register to view assets (poor practice) or view asset usage statistics linked to associated deals via reference fulfilment processes or systems and this does give us numbers but it tells us nothing really about the business impact of case studies. 

We operate global reference helpdesk for a few organisations, managing thousands of request per year and yes we can say how often specific case studies were supplied and (in some circumstances) even if they were used and  can correlate these against specific deal values. It’s way better than nothing but does not measure ROI. 

Here’s the thing: 
Imagine you are trying to close a ten-year, billion dollar outsourcing deal and you are part of the pursuit/bid team. You might invest half a million dollars in the bid itself and one of the items you will use at some point will be a written or video case study, assets that might exist anyway that cost $2,000 to $20,000. 

Even if the case study was used right at the beginning of the process in order for you to get through to the last round of the pitch, and it gets used just once no-one is going to say that it was not worth creating a $2,000 case study for just one $1b use. Nor can anyone say that the case study had an ROI of 5,000,000%. 

It’s the same in the volume space. I’ve seen all sorts of claims about video, and white papers etc. yet we all know that direct sales teams sell most of their stuff using PowerPoint slides, most of which would be unrecognisable to the teams that lovingly created them and also many of which will be for customers that are not approved references! 

A correlation between two sets of numbers (we changed this, the numbers did that) is not the same as cause and effect and I’m not always sure the due diligence has been done to independently verify results. 

What we DO know is that every piece of research out there and also research we have done for clients shows pretty much the same thing; irrespective of format, customer-approved content in the form of a case study is the most-requested deliverable, used right throughout the purchase cycle (with an emphasis on the early stages) and that measured use is only a small indication of real use because folks keep their reference libraries on their laptops and like to share. 

Short of interviewing the entire customer team that was involved with the purchase, the influencers, the decision makers as well as the sales team themselves and asking them which materials were effective and by what percent (and even then you will get subjective answers), measuring usage via request tracking and downloads is pretty much it. 

My advice to anyone in this space? CRPs have much better measurement than other influencer marketing activities and we can measure much more interesting and business-impacting stuff with ROI numbers attached than the number of times case studies are used

It's not the number that counts; it's action

Recently I've seen a couple of organisations embracing the measurement of customer/client satisfaction.

This is good, and there's even been 'Net Promoter-esque' measurement methodologies employed based on The Ultimate Question.

Now they have their numbers both these organisations have 'wimped-out' when it comes to the important bit - taking action.

Receiving of the feedback is just the beginning of the real interaction. Sure, you have your number but where's the value for your customers/clients?

Don't ask if you don't intend to act. When you ask for feedback there is an unwritten deal made; that you will act on it, or say why it does not make sense for you to do so.

Surely asking for feedback then ignoring the opportunity for two-way dialogue is worse than not asking at all.

Next-Generation case studies. Where are we going?

10 Customer Reference Jobs

The jobs market for experienced customer reference professionals has never been healthier. Currently we know of over ten open positions:

1) UK - Honeywell - home counties
Customer Reference Manager

2) UK - Unnamed Technology Company - London
Customer Reference Communications Manager

3) UK/EMEA - Alcatel-Lucent
Customer Programs Marketing Contractor

4) US - VMware - Palo Alto - CA
Customer Reference Manager

5) Canada - RIM - Waterloo - Ontario
Blackberry Customer Success Programme (contract) job number 1102148

6) US - DELL - Round Rock - Texas
Customer Reference Manager (link to the CRF jobs page)

7) US - Netsuite - San Mateo - CA
Customer Reference Program Manager (link to the CRF jobs page)

8) US - inEvidence - Silicon Valley - CA
Account Director/West Coast Lead

9) US - inEvidence - Silicon Valley - CA
Account Manager

10) US - inEvidence - Silicon Valley - CA
Account Executive

We also know of another couple of contractor positions - contact the customer reference geeks to find out more

Make your case studies FABulous

Things go in cycles, the cuts in marketing budgets caused by the recession have had a positive impact in as much as large organisations are thinking about value as opposed to volume (believe me, as a creative and a businessman there is nothing more depressing than being measured by the number if items you deliver for a client, it's like creating case studies by the gallon/square mile/cubic metre and not a smart business strategy long-term).

So what IS value when it comes to case studies?I don't mean cost. I reckon value = great stories that meet a genuine need from prospective clients, making our client's solutions real in a way no amount of marketing bumph ever could (fifteen genuine words from a happy customer must be worth 100 words of messaging, feeds and speeds).

This also means maybe NOT doing so many, or doing them later (when there are benefits to talk about).

Quantifiable benefits...hmmm

The fact is that anyone creating case studies for global organisations will be aware of the drive for metrics with everything (I like fries with everything but that's another story).

Do you know how to be FAB?

• Feature
• Advantage
• Benefit

I never ceased to be amazed how many good marketers get confused about this; we all learnt it, maybe we forgot so here's recap:

Features provide Advantages, Advantages provide Benefits.

• Feature - The Reference Geeks blog is online
• Advantage - This allows you to access it from any Internet-enabled device, anywhere
• Benefit - Which means that no matter where you are, if you read it you can stun your boss with your customer reference knowledge and gain the promotion you have always dreamed of (OK I'm dreaming here)

The benefit statement is the WIIFY (what's in it for me?) and once you start to think this way you will be amazed how rarely you see marketing that really nails this, people seem to get stuck at the Advantage stage (describing the features).

Of course you know this stuff, my next blog post is about sucking eggs...

Just for fun though, go back and read the last case study summary you wrote and see if it was as FAB as it could have been.

Webinars for beginners

We hosted our first customer reference pros webinar today. A really interesting experience as it's the first time we've set one up from scratch (we normally do these for clients using their infrastructure and support). The content and presenting was good, as was the turnout (39 folks from all over the globe) which was great and it was a learning experience for everyone, not least of all for me.

I learned:

1. Even though it's a pain to have to register for an event, avoiding this step is not the right strategy long term. Better to make it as easy as possible and do all you can to ensure folks realise you won't spam them.
2. Test the technology. We had a line drop a few mins into the event however we had tested earlier and so could carry on with just a moment of hesitation. Having 39 of your peers on the line certainly focuses the mind!
3. Ensure people that ask questions state their names first (oops).Maybe use a 'hands-up' process or questions via email
4. No matter how many ways and times you state the dial-in and code there will always be a few people that cannot see it. Have email support running during the event, on another laptop and run by another team member. 

I'm looking forward to gettting the replay online tomorrow and to doing the next one!

Out and About

It's been a really interesting couple of weeks; I've been lucky enough to work with some international marketing students, had dinner with another 17 Customer Reference Pros and also met a few other new people, including a specialist AR consulting company, an English freelancer in France and a very smart Spanish reference pro in England.

What have I learnt?

• B2B marketing is not really covered by Degree courses, probably as its not as sexy as consumer
• Think not just about what your customers may say to analysts, but how responsive they are (does it take ten phone calls to get hold of them? the analysts will give up before this point)
• Marketing students can get enthused about what we do and see the value of advocacy very quickly
• Before taking a film crew up a mountain check the customer actually uses your product
• Automated Voice Recognition is being deployed to get reference content from salespeople
• Online communities need a critical mass and nurturing. Just build it and they probably won't come

I've also been organising a series of six customer reference pro webinars, a first for me so fingers crossed the technology works!