Planning with JICMAIL Series: Issue 2

Marketing is blighted by the Cobra Effect - and the way we measure campaign effectiveness creates the risk of unintended consequences. However, new functionality offered by JICMAIL is one of the ways we can start to ward against that danger, writes Richard Shotton

One of Tony Blair’s signature policies was introducing evidence-based government. As an aim it’s hard to argue with – after all, who doesn’t want evidence?

But the negative repercussions of this strategy should interest all of us toying with evidence-based marketing.

The government policy imposed an array of quantitative targets on departments. No department was more affected than the NHS. One of the targets set for cities was to respond to urgent calls to the ambulance service within 8 minutes.

At first, the policy appeared successful. In London, for example, the proportion of emergencies that the ambulance service reached in that time-frame rose from 40% to 75%.

However, when researchers Gwyn Bevan and Richard Hamblin investigated, they revealed that the situation was far murkier. In their 2009 paper for the Journal of the Royal Statistical Society, they observed anomalies in the call data. As you can see from the graph below, which plots response times for one health service, volumes spike just before the target time and then collapse immediately afterwards – an unlikely occurrence that suggests inaccurate reporting.

Beyond this doctoring of the data, health services were also shaping their behaviour to hit the letter of the target, rather than the underlying intention.

As Tim Harford comments in his book, Messy, the target perverted behaviour. For example, health services prioritised investing in motorbikes rather than ambulances. The logic was impeccable if just the target was considered: a bike needed one staff member, whereas an ambulance required two, meaning more calls could be responded to with the same number of staff if bikes were used.

However, the overall value of the new transportation was less clear. Bikes couldn’t take the neediest patients straight to hospital – an unfortunate consequence, and not one captured by the statistics.

Targets were hit – but at what cost?

Harford argues that the target also created an unfortunate pressure on ambulance drivers to disregard call-outs that had taken longer than 8 minutes and instead prioritise new calls. After all, the target had been missed whether they responded in 9 minutes or 19.

Of course, drivers could resist this pressure and rely on their judgement. But what merit does a target have when ignoring it helps you achieve the overall aim – in this case, saving lives?

The unintended effect of targets not matching aims

The problem with the 8-minute target was that it did not fully reflect the desired outcome. The government wanted better survival rates for patients – which speedy treatment helps. But faster response time is just one of the many factors involved in successfully saving lives.

This sliver of difference between metric and result set in train unintended consequences.

This debacle is not a one-off. It is so common that it has a name – “The Cobra Effect”, given by the German economist, Horst Siebert. The name is inspired by a story from colonial India, where a target to cull venomous snakes backfired after a bounty on their skins resulted cobras being bred, not butchered.

The Cobra effect has troubled a remarkable range of areas – from universities to the army – and, most importantly for you, it is blighting marketing.

The Cobra Effect in Marketing

Too often, what we measure is not what we want to influence., Businesses want long-term, sustainable profit growth driven by brand loyalty. But this is tricky and expensive to measure.

That practical difficulty creates an incentive to measure simpler proxies. These are often short-term in nature: cost per clicks, immediate sales and the like.

But since these metrics are just parts that contribute to the underlying aim, they generate unintended consequences.

Marketers, like doctors, are a competitive bunch and once the metrics have been established the team’s collective intelligence is directed towards improving them. Within limits this works well, but eventually we end up optimising plans to the proxy metric, even if it’s to the detriment of the underlying aim.

The result in this case, just as Siebert would have predicted, is that we end up spending more on short-term elements than is desirable.

We stray too far from the balanced brand and activation split that Binet and Field’s analysis of the IPA database shows is ideal.

The solution

We need to complement what is easy to measure, with measurement of what matters. If important factors are left invisible, they are given too little attention.

There are many ways to resolve this – from long-term econometric analysis to a greater investment in more regular brand equity tracking.

However, if you work in mail media or across multi media planning, there is a new opportunity. JICMAIL, has been created to provide better mail media metrics. At the heart of the offering is a panel of 1,000 households who record their interactions with their Direct Mail and Door Drops. Analysis of the data allows for a holistic overview of the recipient’s reaction to advertising.

There are two broad opportunities for JICMAIL to counter the Cobra Effect.

First, it helps paint a rounded picture of immediate sales generated by Direct Mail or Door Drops.

Currently, the easiest way to track sales is by including a discount code and then recording how often these unique codes are redeemed. However, that means precious margin is given away and it only captures the proportion of people who remember to use the code – it’s an incomplete metric.

With JICMAIL’s database you can look at some of the households who received your mailings and then compare the volume who used the discount codes with the total number claiming to have bought something based on the mailing. It helps reveal the currently hidden interactions, beyond code redemption. The measurement begins to more closely resemble what you actually want.

Second, you can analyse the non-sales impact the mailings have had. For example, you can see whether the mailing prompted a conversation among household members. For door drops, recipients are three times as likely to discuss the content with someone else as use the discount code. By making longer-term reactions more visible it becomes easier to give them a weight and not just focus on those in market.

Marketers need to be wary of the Cobra Effect. Unless you introduce tactics to remedy it, such as those offered by JICMAIL, you could be in for a nasty surprise.

This is the second of “Planning with JICMAIL” – a series of articles and guides by industry experts and practitioners that seek to unlock the value of JICMAIL data for advertisers throughout all stages of the planning cycle.

Advertisers can access JICMAIL data for a free subscription period. If you’d like to see the value of JICMAIL data please contact tara@jicmail.org.uk or go to www.jicmail.org.uk.