Anatomy of a Dashboard Failure (and Pending Resurrection)

 

“Build a dashboard.”

The request, passed along from the head of marketing to the marketing operations manager (who we’ll call “Tammy” to protect her true identity), was simple. The subsequent implementation was anything but. The entire project, in fact, was imperiled from the start, destined to collapse under the weight of dirty data, a decentralized structure, and a marketing culture more accustomed to “gut” marketing than accountability.

This is the story of a real B2B company (whose products you likely use almost every day) and its attempts to define and implement a marketing dashboard under less-than-ideal circumstances. The dashboard’s jagged journey — from construction, to destruction, to its current (and ongoing) reconstruction — provides several valuable lessons for any marketing team considering a new or improved dashboard implementation. A word of caution before you proceed: You may see a reflection of yourself, or your company, in this tale. Don’t take it personally.

A Solution in Search of a Problem
“What do you want to see in the dashboard?” Tammy asked her CMO in May 2005 after receiving the mandate to build ... something. The CMO’s response: “I’m not sure, but whatever you do, I’m sure it will be great.” Tammy knew that the lack of executive air cover would present a huge barrier to the dashboard development. But she was hoping that a hamstrung effort was better than doing nothing at all, if it at least raised the consciousness of the executive team on the need for better measurement systems.

Eyes wide open, she held a series of meetings with members of the company’s Marketing Steering Committee, comprised of the top corporate and business unit marketing executives, to probe the metrics they were using to drive decisions, as well as the areas where they needed more transparency and insight. She learned quickly that there was plenty of data being gathered — star ratings on PR, Web site metrics, and the like — but little attention being paid to it among the marketing teams or senior management. KPIs and other metrics were featured in the marketing plans of each business unit, but those metrics (and the plans themselves) were largely ignored. Business leaders were focused instead on the operating plans and lagging indicator outcome metrics for their particular group: revenues, installed base, growth of license customers, unit growth, market share, and sell-through. In other words, they were focused more on revenue than on marketing’s impact on revenue.

The first iteration of the marketing dashboard, released by the marketing ops team in September 2005, was a reflection of this disjointed corporate mindset around measurement: It contained plenty of metrics (presented in a 150-page document), but no clear objectives or conclusions about how the information was driving business decisions. That first dashboard included brand data (from brand research), high-level PR metrics (such as overall favorability and volume of mentions), market share, and marketing spend data. This last category was particularly frustrating for the marketing ops team overseeing the dashboard. Because of the decentralized structure of the company, each business unit managed and reported its own marketing spend, using very different definitions. “There was very little visibility across businesses into what the total marketing budget was and how it was allocated,” Tammy says. “So it was impossible to have a conversation about leveraging or reallocating resources.”

Along with the full dashboard, Tammy and her team provided a summary one-page scorecard with traditional yellow/green/red indicators. However, without clearly defined objectives or targets from the business leaders, setting those thresholds was very subjective — and rarely resulted in a call to action.

The initial response to the quarterly dashboard from the Marketing Steering Committee was tepid: Great information, they said, but not useful for making allocation decisions or driving the business. “The data was nice for them to know,” Tammy says, “but they weren’t using any of it.”

A Change in Direction
The reception was disappointing, but the marketing ops team also saw an opportunity: leveraging the negative feedback to engage the marketing heads in more substantive discussions about their objectives and the metrics that would help them meet those goals. “We got very clear about what they needed to measure in order to make better decisions,” says Tammy. “We were guiding them to evolve the dashboard so that it actually reflected our market drivers — acquiring customers, improving partner relationships, and other things of similar value. We were starting to have a more sophisticated conversation and creating something that could be shared.”

Those conversations led the ops team to publish a revised dashboard in April 2006, seven months after the initial release. This time, the metrics were more closely aligned with the business objectives. They included customer loyalty, marketing universe, and revenue growth for the core business lines, leads, PR metrics, Web metrics, marketing spend, and various channel and partner data. The summary scorecard was replaced by a monthly “Dashboard Digest” that broke the dashboard into more easily digestible pieces, broken out by revenue, customer, partner, brand/reputation, and other metrics. The Digest included an assessment from the ops team and a recommended course of action for each area flagged as yellow or red.

Unfortunately, while the metrics in both the dashboard and the Digest were more aligned with marketing and business objectives, the quality of the underlying data remained poor. Marketing spend data from the business units did not reflect what was actually being executed in terms of programs and mix; inefficient systems hampered the ability to track leads through the sales pipeline; and there was no consistency to the channel and partner data being reported across the product groups.

The good news: Stakeholders were beginning to pay more attention to the potential value of measurement. The bad news: The operations team, not the Marketing Steering Committee, continued to try to drive that mandate from the middle up, struggling to seat responsibility for using the information at the higher levels. As a result, this new quarterly dashboard improved the level of measurement dialogue and awareness, but without shared ownership at the business and marketing leader level, it seemed clear to Tammy that it would never have any lasting influence on resource allocation decisions at a strategic or tactical level.

Faced with a heavy workload for her team and conflicting priorities, Tammy decided to just stop publishing the dashboard. And no one on the Steering Committee missed it.

Enter the Catalyst
Sometimes, new leadership is required to jump-start a sagging initiative. In this company’s case, a new CMO provided the catalyst for change. She quickly boiled the measurement discussion down to two simple questions: What are the decisions we’re trying to make? And what information do we need to help us to make those decisions?

The answer to the second question was anything but clear — mostly because the Marketing Steering Committee had never successfully addressed the first question. In January 2007, the new marketing chief charged the Steering Committee, not the ops team, with defining the decisions they needed to make on a regular basis, and identifying the information inputs they required to make them. The group would need to develop shared goals for prioritizing marketing allocations that would move the needle on the business as a whole, not just on their individual business unit. It was a significant shift in the way the Steering Committee was to approach marketing strategy, but it had to be done if there was to be any hope of improving effectiveness and efficiency overall.

Since the new CMO’s charter, the marketing operations team has been meeting with the Steering Committee to hammer out the best way to measure marketing’s impact on the business. “That’s how we frame it now,” says Tammy. “We don’t even talk about a dashboard or scorecard — we talk about how we can demonstrate marketing’s impact, what questions we are trying to answer, and what decisions we need to make.”

The next iteration of the dashboard will begin as more of a balanced scorecard, focused on five areas: business drivers (revenue, channel, leads); brand and reputation; customer value; marketing spend; and talent development. “In each of those categories, we’re trying to get to the top two or three metrics that will help us make the decisions that that Steering Committee needs to make,” says Tammy. The goal is to build consensus more deliberately and not jump so far down the spectrum of detailed metrics until they can lock down the basic pillars of measurement for the next annual planning cycle.

Lessons Learned
Looking back over the past year and a half, Tammy can point to a couple of things she and her team might have done differently. At the top of the list: “We should have been upfront and assertive in pushing for clarity about the specific and detailed role of marketing in helping the organization achieve its objectives. That would have forced the issue of alignment to the surface earlier, before we spent so much time and energy trying to build consensus bottom-up.”

Tammy believes her team also could have been more strident in calling attention to the poor quality of the data — and the causal factors behind it. There were two key learnings in this area. First,  the lack of alignment between business units seemed, on the surface, to be causing obstacles to sharing information on marketing spend, resource allocation decisions, etc. But in reality the misalignment was preventing pooling of resources to attack some of the key companywide process deficiencies in collecting and analyzing data, making each individual business unit uncomfortable in sharing what little, unreliable information they had. And second, having never anticipated so many data gaps, Tammy wasn’t prepared to begin training the key constituents on tools and techniques for more reliable, structured guessing to fill the gaps. Most of them were already making significant guesses in operating their businesses, but the methodologies varied widely and were creating deep gaps in world views amongst members of the Steering Committee based upon highly subjective criteria.

In the harsh light of hindsight, the project had three critical flaws:

Data problems can always be overcome somehow, if the key constituents of the dashboard can work together collaboratively toward a collective goal. But these three critical flaws will always translate into dashboard failure, along with very inefficient use of key people resources.

The Renaissance
Fortunately, Tammy doesn’t consider her work on those early dashboards to have been in vain. In some ways, she knew going into the project that the dashboard might well be destined to fail. But she believes the results have been worth the effort, because it laid the groundwork for the company to adopt a mindset far more attuned to measurement.

“Where there was apathy toward measurement in the past, we’re slowly beginning to build a culture of accountability,” Tammy says. “This group is now very passionate about measurement and about holding themselves — and their peers — accountable for building the processes and tools we need to make smarter decisions about allocating our resources. And our emerging measurement structure will have the benefit of all the learnings we’ve had over the false starts of the prior two years.”