Palo Alto Software Updates Marketing Savvy with Multivariable Testing

 

 

Palo Alto Software develops software and books that help with business and marketing planning. They sell their wares on the Web. In fact, they sell them much better on the Web today than they did a year ago, thanks to some enlightened testing to optimize response.

A year ago next month, Palo Alto employed Web site analytics followed by a good ol' sample user group to determine what content it should add to its feeder site Bplans.com to improve closed-sales rates and decrease visitor defection from both that site and the product sales site, PaloAltoSoftware.com.

Supplemented by data from past Palo Alto packaging-design studies, it found that business plan-specific articles on Bplans.com Īݠintended to offer research-type resources without a strong sales message — generated more click-throughs to Palo Alto's product and purchase-decision pages at PaloAltoSoftware.com.

Palo Alto dumped its editorial plan to develop more broad-based business articles and achieved a lift in sales conversions from 1.01% to 2.39%. The company delighted in its win but recognized that the results made sense intuitively and the approach to determining the appropriate site changes was pretty basic, even dated.

Palo Alto was not satisfied with this success for long; improving marketing performance should never end.

The company recognized that the easy, intuitive work had been done. Pat McCarthy, executive producer of the company's Web team, couldn't see any more obvious elements to test and tweak. His first inclination was to implement an A/B/C split test in which "A" and "C" page views mirrored one another, acting as controls. When "A" and "C" received the same results, "B" was assumed to have received sufficient visitors to determine if it was an improvement or not.

Unfortunately, McCarthy figured, it would take three to four weeks per attribute to test new site designs by this method, and looking for elusive elements to adapt could take years by project's end.

Instead, McCarthy used multivariable testing (see 3 Ways to Accelerate Your Learning Process and Using Multivariable Testing to Find Hidden "Solutions" for more on multivariable testing) to try several changes at once.

 

"We'd been doing slower forms of testing aspects on our Web sites using Web analytics for a few years," he says. "I was already familiar with the concept of A/B or split-run testing, so the multivariable approach looked very powerful and interesting."

For this go-around, McCarthy chose to focus on the decision page on which Palo Alto's two top products are offered — one at a $99.95 price point, the other for $199. Not only was this a critical page on which to secure sales, but its design was similar to other product pages on the site. McCarthy looked forward to applying the results from this multivariable testing project to those pages as well.

Eleven variables were chosen for testing, including the prominent product photographs that appeared at the top of the page. The Web team developed 58 variations for the 11 attributes toward the creation of more than 41 million unique page views.

Part of the beauty of multivariable testing is that through a limited series of test pages, all selected attributes and variations of them are trialed so as to provide succinct recommendations for site renewal.

In the end, chart-like graphics with bulleted features won out over paragraphs of product-descriptive copy, and a "buy" button was more attractive than a text link to buy. McCarthy even had to lose a photo of one of the available products on the decision page (we told you this wasn't intuitive stuff).

A testimonial from ZDNet sold heavier than a product image. Quotes from well-respected industry authorities now are replacing small screenshots elsewhere on the site.

McCarthy's new page led visitors to purchase at an increase of 41.3% over visitors to the old page. "We'll use multivariable testing again as soon as we have a very important brand new page or concept to test," he says. "We'd prefer to use multivariable testing due to the speed and amount of options you can test."

 

Multivariable testing has a number of influences, among them engineering standards developed by Dr. Genichi Taguchi. Click here for more information on his methodology and its application in U.S. industry.

For even more on multivariable testing, see Classic Marketer Seeking Stimuli That Prompt Profits, a summary of the October 2001 Harvard Business Review article "Boost Your Marketing ROI with Experimental Design" by Eric Almquist and Gordon Wyner of Mercer Management Consulting.