Searching for Better Planning Assumptions? Start with the Unknowns

 

 

It’s that time of year again. The week on Nantucket is a fading memory, the kids are back in school, and a CMO’s attention turns to ... budget season. That glorious time when predictions are prevalent, assumptions run wild, and commitments are due. What better time to do a reality check on what you actually know about the performance of your past, present, and future marketing investments — and, just as importantly, what you don’t know?

A marketing team’s ability to plan effectively is a function of the knowns and the unknowns of the expected impact of each element of the marketing mix. Too often, unfortunately, the unknowns outweigh the hard facts. Codified knowledge is frequently limited to how much money lies in the budget and how marketing has allocated those dollars in the past. Far less is known (or shared) about the return received for every dollar invested. As a result, marketers are left to fill the gaps with a mix of assumptions, conventional wisdom, and the occasional wild guess — not exactly a combination that fills a CMO with confidence when asked to recommend and defend next year’s proposed budget to the executive team.

The tenuous link between, for example, investing in brand development and the resulting economic value created is itself a high-risk activity for marketing teams. A CMO may be keenly aware of how brand advertising translates into brand preference, but what does he know about how brand equity influences purchase behavior? Is preference a cause or a corollary of a first purchase, a repeat purchase, or increased loyalty? And what is the strength of that preference — in other words, how much will the customer be willing to search for the preferred brand?

Rare is the CMO who can answer those questions, connecting marketing investments to incremental revenue and profit growth. Yet this time of year, every CMO is asked to take a blood oath that his budget and plan will achieve the requisite results.

See a problem here?

To build a more effective framework for resource planning and allocation, CMOs need to first get their arms around what they know, what they think they know, and what they need to know about their marketing investments. A three-step program, which consists of conducting a marketing knowledge audit, prioritizing the knowledge gaps, and getting creative about the methods used to capture that missing information, will provide a stronger foundation from which to make more disciplined (and defensible) budgeting and resource allocation decisions for this year’s cycle and beyond.

Step 1: Audit Your Knowledge
The starting point for a budget plan comes in the form of a question: What do we need to know? Some answers are required immediately to determine this year’s proposal; others are further out on the horizon (but must be asked now to lay the groundwork for the next year’s budget). From this meta-question springs a host of other queries:

Creating a comprehensive list of questions is critical for addressing as many scenarios as possible. Equally important is a candid assessment of the answers you already have to those questions. Is the information fact-based or presumptive? Being brutally honest is the only way to get an accurate picture of the scope and quality of the information at hand.

A useful exercise is to categorize the knowledge into several buckets: facts (e.g., hard data such as product sales or current and historical marcom spend); evidence-based opinions (young consumers spend more time online than watching TV); assumptions (our customer base will continue to grow at 2% annually); wild guesses (our new corporate blog will increase brand awareness); and unknowns (how many new competitors will enter our market next year).

A clear separation of knowledge into these categories provides the underpinnings of sound decision making. This exercise will allow you to weight different data sets based on their source, providing a better picture of the breadth of information being collected while shining a light on weak areas that require additional input to be redible.

Let’s be clear: Business does not run on facts alone. Managers are paid to use instinct and intuition in their decision making. Assumptions and guesswork are a necessary (and unavoidable) part of the planning process. But problems arise when marketers confuse assumptions with facts. They may assume, for example, that what worked in the past will work in the future; this approach often leads to good money chasing bad, year after year. Another misguided conclusion: If competitors are investing more in a new channel, then they must know something that we don’t — so we need to invest there as well. You get the idea.

A thorough assessment of the marketing team’s knowledge also can guard against planners who, in their eagerness to secure funding for a new program, may be tempted to ignore or dismiss critical information. “If I see a current trend altered significantly in a budget plan, I get suspicious,” says Michael Linton, senior vice president of marketplace adjacencies at eBay, a role in which he oversees the online auction company’s Stores and Half.com businesses, as well as its Canadian operations. “Customers and trends don’t change just because you are entering a new fiscal year.”

The key is to identify the knowledge gaps that, once filled, can lessen the uncertainty around the unknown elements, which will give you more confidence to make game-changing decisions. At this point, you can begin to prioritize the things you need to know.

Step 2: Prioritize the Gaps
For each gap or unanswered question, it’s time to pose another question: So what? In other words, how would a particular piece of information change the decision process? A certain piece of information might cause you to completely rethink the scope of a new program, which could have a material impact on marketing performance. The exercise is akin to handicapping a horse race: You are placing a value on the answers you need to gather. The data can take many forms: market research, analytics, call-center data, sales figures, and the like.

Next, try assessing the process and resources required to acquire the missing information — not just the hard costs, but also the degree of difficulty and the human resources that will be put to the task. Some of the information may exist in departmental databases or in the minds of various experts inside and outside the company. In some cases, finding what you need can be as simple as mobilizing the team to go out and interview a range of external stakeholders. “What you know is dependent on where you are. If you’re in your office, that’s all you know,” says Laurence Prusak, a researcher and consultant and co-director of the Working Knowledge Research Center at Babson Executive Education. “The most valuable knowledge is local knowledge — from salespeople, from vendors, and from customers. Go out and talk with them.”

It’s important to consider any source that could influence the knowledge you need to acquire. For example, tracking the number of calls received from the call center will provide some insights on customer satisfaction, but what additional insights could be gained from reviewing the actual transcripts of those calls? Similarly, tracking competitor ad spend is useful, but determining the return those competitors are receiving on that investment is far more valuable.

Those additional (and often subtle) inputs often may provide the most telling clues about evolving patterns with customers or the marketplace that could disrupt the marketing mix. “It’s very important to recognize weak signals,” says Prusak. “Don’t just listen to the major ones.”

A word of warning, however, as you cast a wide net in search of answers: The natural instinct is to gather every piece of readily available information. But what’s the point of allocating any human or capital resources to gathering data that has no economic value? A far better route is to focus on answering a few of the really tough questions. For example, what drives customer experience? Is it the product, the purchase, post-sale support, the billing process, or something else? Which of those specific components are the true predictors of customer profitability? It’s highly likely that one or two of those elements are the real drivers, but without knowing for sure, it's easy to fall into the trap of spreading resources across all the elements — thus reducing the impact of those investments.

 

To determine the relative importance of the knowledge you need vs. the cost or difficulty of obtaining it, consider creating a simple 2-by-2 matrix (above). With the quadrants filled, the next step is to delegate the high-priority items to members of the team who have a clear stake in gathering the required information. Get their proposals on the right methodologies to deliver the desired insights and find ways to include the costs in your plan so you don’t face the same knowledge gaps next year at planning time.

Step 3: Get Creative with Your Testing Methods
Marketers have many methods at their disposal for filling the gaps; some are commonly used, others are underutilized. The key, as noted above, is determining the most cost-effective methods to gather the most relevant information.

For example, secondary research — analyst reports, panel studies, and the like — can provide a fairly inexpensive snapshot of relevant topics regarding customers or the marketplace. This type of information, however, is unlikely to provide answers that are specific to the context for your business. Such a gap limits the effectiveness of secondary research in answering some of the tougher questions the marketing team is asking.

Primary research, on the other hand, can be completely relevant to your business. If properly conceived, designed, and executed, it can answer any question you want it to answer — but it can also be wildly expensive. It’s wise to focus this approach only in areas where the expected value of the information is high — the upper right quadrant of the matrix.

Even primary research, however, won’t always answer all of the really tough questions. In these cases, the only way to determine which activities will deliver the highest return is to experiment, running dozens (or even hundreds) of small tests. Experimental design is an underutilized option for many marketing organizations, but it’s often the best way to develop evidence-based opinions that add credibility to budget and allocation requests.

“If you say you tested a new ad campaign last year and it increased new customers by 5%, that’s fine,” says Linton, a 20-year marketing veteran who joined eBay last year after a four-year stint as Best Buy’s CMO. “But if you say you’re going to turn on this new ad campaign and consumers are going to run to the stores, it’s not enough. I need more evidence or proof.”

Linton is a big proponent of a test-and-learn approach, in which a steady flow of new ideas — anything from a loyalty program, to a new direct marketing initiative, to a set of sponsorship runs, or a new media platform — is tested to determine the ideas that warrant further investment. “Anything that has the potential to actually move the business is worth testing,” he says.

Numerous — and proven — experimental design techniques are available. Split-run or A/B tests are sufficient for testing one or two variables in a single market. But these test-vs.-control techniques can sometimes create hard feelings with salespeople in the control group who bristle at not receiving the same level of marketing support that the test group is getting. A/B tests also are ineffective across multiple segments or markets, or when multiple elements of the mix are involved.

This is where multivariate techniques come in. There are several types, including full or fractional factorial designs and Plackett-Burman methods (see Primer, right). These techniques incorporate many variables — five elements of the marketing mix, for example, across 20 geographic cells — and allow analysts to vary the mix of activities and spending in each cell to measure the impact not only of individual elements, but various combinations as well.

For example, one region (or cell) will run a program that includes a heavy dose of print and online investment, but no spot TV. A second region will emphasize TV and direct marketing but eliminate radio from the mix. In a third, investments are spread equally across all media. By examining enough variation, statistical patterns will eventually emerge that provide a more informed view of the mix.

If you’re inclined towards analytical modeling systems to add some discipline to resource allocation decisions, that’s fine too. Just be sure to interpret the results in the proper context: As a driver of incremental business growth. A packaged goods company may use sophisticated modeling techniques to determine, for example, that 18% of last quarter’s sales came from promotion, 10% was attributable to discounts, and 4% came from advertising. The remaining 68%? It’s usually attributed to the “baseline” — the portion of the business assumed to be driven by brand and customer equity. But here’s a hint — you haven’t really solved the knowledge gap until you can explain what causes that “baseline.”

Short of those elaborate analytical models, an often overlooked but very effective technique for filling knowledge gaps is decision calculus. Similar to Delphi techniques, decision calculus involves modeling a natural decision process — e.g., should we increase sales headcount or spend more on marketing support? — and breaking it down into smaller areas of uncertainty. Each uncertainty is discussed in a group by people with varying experiential frameworks (sales managers, marketing directors, finance personnel). Perm-utations of alternatives are explored until the group reaches consensus (or at least a narrower range of outcomes) on the best decision.

Take the sales-vs.-marketing example. The head of sales argues for more headcount and sees no correlation between marketing programs and increased sales. The CMO believes the existing sales staff needs more marketing air cover to increase effectiveness and drive revenue. A decision calculus exercise would look at the impact of various combinations of increased spending — 10% on sales headcount, 10% on marketing programs only, 10% on each, rinse and repeat at various increments — on current forecasts. By exploring multiple variables, assumed correlations between marketing expenses and sales headcount will become more mathematically evident — and will make the assumptions more explicit, exposing them to more rigorous (and beneficial) challenges. These techniques are very useful for moving key individuals past opinion-driven perspectives and stimulating more productive dialogue, thus avoiding a potential stalemate between sales and marketing.

The additional insights gained by these methods will help eliminate confusion between the correlation of activities or events with their causality (see “The Relevance of Rigor,” MarketingNPV Journal, vol. 1, issue 6). You may, for instance, have data that shows that advertising drives sales — but you may have other data indicating that the same advertising has no impact on sales. Taking into account all variables — not just those that support the assumption — will provide a more accurate picture of causality.

None of these approaches will return perfect results; business just doesn’t work that way. The goal is to develop far more clarity on exactly what you’re trying to learn and the right mix of variables — both internally and externally — that may have an impact on the answers you’re seeking.

Conclusion
Looking ahead to key questions that will drive your marketing programs will ensure that you spend precious knowledge-gathering resources wisely. Prioritizing learning needs will help stakeholders both within and outside of the marketing group understand that you are aware of your knowledge limitations and are working methodically to eliminate the most important gaps. This will build credibility in your recommendations while minimizing the risk inherent in launching new programs or entering new markets.

Through a combination of analysis, research, experimental design, and decision calculus, CMOs can increase the probability of satisfying the highest-priority knowledge gaps. A better grasp of the knowledge base will lead to better insights into the activities that provide the most return — enabling you to reallocate funds more appropriately (and more confidently) away from underperforming programs.

A more disciplined view of knowledge within the marketing organization is also an important step in the move away from status-quo planning processes, in which this year’s marketing budget mimics last year’s, plus or minus a few percent. Such inertia is replaced with a richer, more collaborative environment in which resource allocation decisions are made under the disinfectant of sunlight and adjusted dynamically to ensure that all marketing activities are performing to their maximum potential — and helping to grow the business profitably.

Read sidebar A Tipping Point for New Media?

Read sidebar Experimental Design Primer.