Action Standards and Benchmarks Improve Your Market Research ROI

by David Wilson, CEO, and Dino Fire, director of marketing science

As we discussed in part 1 in this blog series (Market Research ROI: 5 Questions to Make Your MR Pay Off), market research, like any investment in information, should demonstrate a clear return on investment (ROI).  

It should speak directly to a business need or opportunity that’s crucial to your organization, whether it’s for an immediate business solution or part of a long-term strategy.  And it must be trusted by the ultimate decision makers and actionable.  In other words, you have to be able to use the research to make decisions that improve your results, or you shouldn’t do it at all.

Of course, this begs the question of timing—how do I know when to confidently act on the results and recommendations from my market research?  The answer, in many cases, is simply this: use action standards and benchmarks.

What are action standards and benchmarks?

The term “action standards” is the market researcher’s way of saying “decision criteria.”  Simply put, action standards describe what decision will be made and which objective criteria will drive that decision.

In most cases, the action standard is a market research-based numerical score or outcome that triggers the acceptance of (or selection of) a proposed concept or approach.  If the research results do not meet or exceed the unambiguous action standards, the “go forward” decision will not be made. The next steps are usually to remain with the status quo or go back to the drawing board and retest a new concept.

Importantly, action standards are predetermined, like goalposts in American football. This way, the results from the market research can be objectively and dispassionately interpreted and acted upon (or not).  And, as with football, there’s no moving the goalposts.

Along with predetermined action standards, corporate or industry benchmarks are often paired with action standards to help make the post research decisions quickly and confidently. These benchmarks are usually compiled, tested, and proven over time (by companies or research suppliers) using very similar question types, scales, and sampling methods.

Action standard examples

Here are 2 examples of how a CPG company (or a private label for a retailer) uses action standards and benchmarks to improve their trial, revenue, and share results from a package redesign and new product name.

Package redesign:

• In the first example, we’ll assume that Java Joey Coffee is planning to redesign the packaging for their popular organic coffee line.  
• Java Joey wants to compare consumers’ reaction to the new design concepts against its current package.  In this example, the existing package design result—think of it exactly as a control group—serves as the benchmark.
• Java Joey’s prior research and in-market results over the last 5 years have proven that new concepts succeed if they are at least 20 percent better than the current concepts.  As a result, the action standard is a 20 percent improvement over the existing package.
• So, if consumers rate the new package design at levels that are at least 20 percent better than the existing design, the new package concept will be selected.

New product naming:

• In the second example, we’ll assume that Java Joey is planning to launch a new type of instant coffee that is intended to rival Starbuck’s “Via” brand.
• Using several online community (MROC) and online focus group (OLFG) sessions, Java Joey has narrowed the brand names down to 2: “Zip” and “Pep.”  It wants to select the best name between the two.  
• In this example, Java Joey’s prior naming research and in-market results provide the benchmark for success: a consumer preference index score of 8.0 for new product names. 
• Java Joey’s prior research and in-market results over the last five years have proven that new names succeed when they meet or beat the 8.0 threshold.  As a result, the action standard is the achievement of an 8.0 or better on one or both names.
• So, if Zip or Pep scores 8.0 or higher, that name will be selected. If they both score an 8.0 or higher, the higher of the two will likely be selected, although other factors and results may influence the final selection.  The “tiebreaker” rules are also set forth in the action standards for the project.

Using action standards: parity vs. superiority

How you set your action standards will depend on your business objectives.

• If your new concept or product must outperform an existing product or that of a competitor, then it must be superior across certain measures that are most important.
• If it must perform at least as well as your current product or your competitor’s product, then it must be at parity.  

In either case, performance measures (value, trust, preference, price, convenience, overall satisfaction, etc.) must be predefined by the company (client).

Parity vs. superiority examples

• Java Joey’s new organic packaging is redesigned with the hopes that it will take share from competitor’s coffee.  Result:  Java Joey is seeking parity with its own coffee and superiority vs. its competitor’s coffee.  Importantly, Java Joey’s realizes that to increase its market share, it will have to acquire new customers.  So the package redesign must reflect attributes intended to drive trial from new customers, not just increased repeat purchases.
• Java Joey’s organic coffee is considering a reformulation and resourcing of its organic coffee to lower shipping costs and stay within the industry standards for organic and free trade coffee.  The reformulated coffee simply needs to perform as well as its current offering so that revenue and share at least remain the same while margins increase. Result: the goal is parity.
• Finally, if Java Joey is re-launching a line of coffee with the promises that it is “bolder and smoother,” the new formulation and offering should actually perform better in the research. Result: the goal is superiority.

Statistical hypothesis testing is used to determine if a concept or product is inferior, superior, or at par (or parity) with an existing concept.

In general, benchmarks should not be set too high or too low. When benchmarks are too high, the new concept may be rejected, when in reality the in-market results may have performed very well.  On the other hand, if they are too low, the new concept may show up falsely as very strong in the research, just to perform poorly on the shelves.  

When to apply action standards and benchmarks

Action standards and benchmarks are vital when large or risky investments and strategic decisions must be made about new concepts, including: products, packages, names and ad campaigns.

They predetermine the criteria, which will clearly, unambiguously, and objectively determine which new concept, if any, is selected or rejected.  In other words, action standards and benchmarks use hard metrics to guide companies to make better decisions and get better in-market results. For CPG companies or private label retailers, they are very frequently used when products are reformulated or lines are extended.

New concepts and products can make or break a company’s long term financial performance, consumer loyalty and brand equity.  And these new concepts and products are driven by clear objectives to increase revenue, share, and margins.  

It is imperative that the company (client) and research supplier work closely to establish the action standards and benchmarks before the research begins.  They must fully support the strategic and financial objectives of the new concept or product.  Once set, put your best concepts into the market research crucible, and never move the goalposts.

Line Review Ammunition: Market Inventory and Customer Segmentation Studies

by Dino Fire, director of marketing science

Retailers’ line reviews evoke the specter of sleepless nights for manufacturers, tense negotiations, financial forecasts, and product test proofs—an exhaustive and sometimes fruitless struggle for prime shelf real estate.

Instead, what is needed a solution as unique as your product, one that addresses the specific concerns (beyond just the numbers) of the retailer with whom you are trying to grow a relationship.

Market inventory and customer segmentation benefits

What that solution looks like is dependent on the research you’ve conducted on your consumers. And a detailed customer segmentation study, which will provide you with the ages, genders, purchase drivers, preferences, and attitudes of each type of consumer that will buy your product, is the key.  

Coupled with a comprehensive awareness and usage methodology, which measures your brand strength as well as that of your competitors, the market inventory is a powerful, meaningful, and actionable tool to help build your retail case.

With both of these studies, you can then:

• Tailor and target your marketing and product development efforts to these personas
• Identify and act on specific purchase drivers from a supporting marketing program
• Differentiate your goods from competitors by showing (with data) how fast the product is likely to move within the specific customers segments
• Outline, by store, which products will likely perform best by segment that shop at that location
• Establish successful price points for your products
• Grow market share and expand your product line

It’s a whole new story, and the information that you bring to the line review table moves up from a knock on the retailer’s door to a battering ram. It simply isn’t a fair fight—your competitors’ efforts will pale in comparison to the fact-based strategy and rationale you bring to the table.

More about the market inventory

To build these powerful profiles of your end users (and enjoy the benefits at the line review), you need a foundational study upon which to build—the market inventory (MI).

A market inventory study is a specialized and thorough awareness and usage survey conducted among consumers in your potential market. The MI provides:

• An overview of whole category, including awareness, usage, preference, and key drivers
• In the appropriate product categories, real information about what professionals buy versus other consumers and why
• What your competitors sell and how they rank against you with consumers
• Your competitors’ weaknesses and how to exploit them

This information, viewed in context of consumer segmentation analysis, allows you to hone messages and products really that resonate with your largest groups of potential customers.

This powerful tool—the market inventory study and segmentation study—will give you rare peace of mind in line reviews, and show the retailer you commitment both to the success of your product and their store.

Tackling Mistake 5: Improperly Using Your Custom Online Panel

Are you using your online panel too much? Or are you ignoring your panelists completely? Find out how to use your custom panel correctly (and keep your customers happy) in this fifth video in our eReport series.

The Power of Customer Segmentation

by David Wilson, CEO

Plenty of people like hot tea. Many others love iced tea. Anyone for lukewarm tea? Didn’t think so.

Well, if you’re launching new products and expensive advertising campaigns without segmenting your customers, you might as well be serving lukewarm tea.

Too many companies develop products, services, features, and ad campaigns that attempt to satisfy the average customer or the average preferences of their customers. Here’s the catch: there isn’t an average customer.

At best, when companies try to serve the mean, they underperform. At worst, they fail miserably. These people will be much more likely to buy a product from the vendors who understand them, appreciate them, and market their products correctly to them. How do smart companies achieve this? Through customer segmentation research.

Generally speaking, customer segmentation can be defined as a market research methodology that divides a customer base into groups of individuals that are similar in specific ways relevant to marketing, such as product needs and uses, age, gender, interests, spending habits, etc.

Using segmentation allows companies to target similar customer groups effectively, and allocate marketing resources (product development, packaging, pricing, advertising, and so on) to maximize market share, profits, customer satisfaction, and loyalty.

Taking care of your most loyal segments

As an example, let’s look at how retailers can put customer segmentation to good use. Today’s retailers are focusing more and more on repeat customers who are loyal to their stores and their store brands.

Using loyalty card data, custom panels, and customer segmentation research, they are able to focus their product development and marketing/promotional dollars on the highest priority customers.

Here’s the advantage of this approach: by “segmenting out” the non-loyal, bargain-hunting shopper segments, retailers can improve the results and recommendations of their market research and in turn, their sales and profits every quarter. Why? Because the opinions and preferences of the bargain hunters are removed to allow for a more pure and perfect view of the most loyal customers.

Using the lens of customer segmentation, these retailers are able to create the products and promotions that thrill and delight their most loyal, heavy-spending customers. The best customers get exactly what they want and the retailer grows revenue and profits, a true win-win situation.

Customer segmentation and custom panels

Whenever possible, customer segmentation research should be executed in tandem with custom online panels.

For example, it is standard practice at FGI Research to build a custom panel for our clients and score it with specific agreements. Using the segmented custom panel, clients are able to continuously test new products, ad campaigns, features, prices, and other strategies and tactics within targeted customer segments.

This gets done very quickly, accurately and cost-effectively. Without exception, this approach yields products and marketing initiatives that perform at much higher levels. In many cases, this approach is a true “game changer” for these clients.

So if you haven’t made an investment in customer segmentation (and a custom panel), we recommend you give it strong consideration. Once implemented, you’ll watch your customer loyalty, revenue, and profit measures go up (and market research costs, missteps, and delays go down). Then, you can sit back and celebrate your success with a cup of hot tea, or an invigorating glass of iced tea.

Want to Gain Market Share? 3 Key Questions for CPG Companies

The days are getting longer, the sun is shining brighter and the grass is starting to turn green again.  It’s finally spring, which for many homeowners is a time to start do-it-yourself (DIY) projects in and around the house. 

Whether for financial reasons, for fun, or just because it is that time of year, many of today’s consumers are actively expanding their palette of DIY projects and this industry has become fast-paced and competitive. 

One consumer packaged goods company was looking to gain more market share but were unsure of how to extend their product lines. It quickly became apparent that the first step was to get answers to these 3 essential questions:

1. How aware of our brand are our consumers?
2. What are the deciding factors for choosing certain products?
3. What is their motivation for doing home improvement or other related projects?

Surprises in the data

Right away, a number of surprises came out of the initial brand awareness study. First, it was determined that the company had very high brand awareness with consumers; however brand perception was a different story. 

Factor analysis was then used to help separate consumer feedback and data focusing on awareness, perception, and usage of their products.  This further analysis and a product-usage test ultimately paved the way for the company to develop a highly aggressive and targeted marketing campaign tailored to driving new consumers towards their brand.

Find out more about the new customer segment that the CPG company found, the new purchase drivers, and get suggestions and tips on how to better shape your marketing message and the power of customer perception in the case study: Finding New Customers and Purchase Drivers.

Tackling Mistake 4: Online Panel Mismanagement and Poor Marketing

In this fourth video from our eReport series, David Wilson, CEO, and Kara Davis, project manager, examine best practices around custom online panel management and how to correctly market it to internal clients.

Want to learn more? Get the full eReport: The Top 10 Mistakes to Avoid When Building and Using a Custom Online Panel.

3 Tips to Build a Successful Online Custom Panel

Building an online panel with the right composition of panelists is a lot like cooking vegetable soup. Depending on your personal taste and preferences (as well as budget), you bring together a variety of ingredients and stir.

But what ingredients you add and in what proportions makes all the difference in the end result. So what’s the perfect recipe to yield the happy panelists and the data your business needs?

1. Define what you want from the panel

Assuming that you have already figured out your specific research needs (explored in our eReport on building and using a custom online panel), you will need to define some expectations for your online panel.

This really depends on what you are using the panel for. Consider:

• Is it to field monthly customer satisfaction studies?
• Do you need a community to conduct product concept testing on short notice?
• Is the overall goal to create a benchmark study that will be used as a reference for the next 5 years?

It might even be a combination of these options, and a panel can accommodate that. But you have to build those capabilities into the panel from the beginning in order to get the best results.

2. Be proactive and consider your quotas

In other words, measure your ingredients carefully. As mentioned in this video on mistake 3 from our eReport series, keeping your panel and your research results balanced should be a big priority. Different research projects will require varying levels of participation from your panelist, and making sure they have the relevant experience and interest in doing so is essential.

As well, particular demographics are known to respond at different rates, so to get the results you want you need to have the correct people in your panel. Here are some questions to ask:

• Which key demographics do I want to hear from?
• What kind of response rates do those segments have?
• What kind of surveys and online activities do these market segments enjoy?
• Does my panel have room to grow if I need bigger pools of respondents?

An online panel’s strong point is its productivity over the long-term. Keeping the right mix of people in the panel keeps your research fresh and relevant. If you don’t know the answers to these questions, it is a good idea to consult a sample specialist.

3. Have a contingency plan

Say you step away from the kitchen for a moment, and when you get back dinner is burnt. We’ve all been there, but it’s a good idea to have a contingency plan in place so you’re not left hungry.

An online panel is versatile and far-reaching, but even today some demographics don’t have a strong online presence. Using the optimum mixture of research modalities can make a big difference in terms of reaching these populations, and keep your quotas balanced. Adding a telephone survey or mailed study to your panel results can give you better value while keeping data quality at a premium.

If you’d like to get more tips on how to build, manage, and benefit from a custom panel, see our eReport: The Top 10 Mistakes Commonly Made When Building and Using a Custom Online Panel.

What It Means that Google Has Entered the Market Research Arena

by Heather Woodward, director of research operations

Google’s announcement and launch this week of a market research offering has had the market research and technology industries buzzing.

And understandably so. As this article from the GreenBook blog points out, the world’s biggest search engine company already has access to millions of consumer data points (thanks to Gmail and Android phones), and now they are going to ask those millions of users and their friends about their shopping, product, and marketing preferences one question at a time.

But before anyone in either industries panics, it is worth doing a little research on Google’s offerings to understand where and when their tool is the most helpful.

What it’s good for

One question, thousands of responses. Google’s tool allows for quick responses to urgent questions, and access to immediate results. A great fit for those urgent one-off questions that come up in a meeting at 5 pm and need to be answered by 9:30 am the following morning.

Your Adwords campaign. The platform runs on Google’s publisher network. So if you are going to advertise via Google and you want to do some simple market research on that population, it is a good option.

DIY research with the Internet population. Unlike many do-it-yourself tools, the only responsibility for the user here is to design the research questions. Google handles the sampling to deliver balanced results, including weighting where it is required. Real time reporting includes charts with descriptive text highlighting interesting differences found in the data.

Demographic information is provided. Google uses a combination of factors, including IP address and cookies to make assumptions about demographic data based on information about the respondent’s originating computer (IP address and sites visited), so demographic questions do not need to be asked unless the demographics need to be precise for the purposes of the research.

The current limiting factors

One question at a time. You can ask a yes/no screener questions to qualify someone to answer that 1 question, but building a longer survey with related questions would be difficult. Piping from one question into another based on response would not be an option, nor would dynamic creation of a question’s content based on previous responses or sample data. 

You can certainly ask more than 1 question as part of a research initiative, but not of the same people (each person gets fed just 1 question or 1 screener + 1 question). Each question would need to be able to stand alone (or with a single screener leading into it) and not rely on previous questions in the survey to make sense. This would make it difficult to conduct such exercises as Van Westendorp price modeling, choice based exercises, or even aided and unaided awareness.

Balanced to the US Internet population rather than the US general population. This is not a limitation if the internet population is the target audience for your product, but if you require a traditional census stratification scheme for your sample, you may need to supplement with other online or telephone sample. This is a limitation Google recognizes and references in the white paper listed below.

Sample frame is limited to Google’s publisher network. There may or may not be bias introduced by this limitation, but it bears observing as the product rolls out and we begin to see the way it performs. 

Question types are limited to multiple choice, image choice, and 5-point Likert scales. 

Not able to append data from other sample sources. With a custom panel, data can be appended from customer databases of transactional data or segmentation schemes. With a general consumer panel, data can be appended from large consumer databases. And with a completely anonymous sample source, the option to do this kind of appending effectively is eliminated.

Sensitive questions may be a problem. Another risk factor noted by Google is that the nature of the interaction with the respondent (intercepting them as they navigate the web) may make very personal questions difficult to get responses to. For consumer panels, these objections can be more easily overcome as there is an existing relationship with the panel provider and a level of trust has been established.

One could argue that an intercept survey is more analogous to a RDD-based phone survey, since the respondents are not necessarily enrolled in an online research panel and are, in effect, randomly selected. Results from a 2010 study by Hines, Douglas, and Mahmood shed some light on the phenomenon of differences in the rates self-reported mental health issues based on the data collection mode. Higher incidences of some mental health disorders were reported among online panel respondents, but the RDD telephone sample reported higher indicences of others.[1]

Non-response bias.  All surveys suffer from non-response bias to one degree or another, but response rates for unsolicited pop-up surveys are notoriously low. Fricker (2008) cites an early study conducted in the UK in 2000 that yielded response rates of 15-30%.[2] The issue is less with the response rate itself, but the lack of ability to know anything at all about the non-responders.

Want to know more?

• PC Magazine online does a quick overview of the product
• Check out the GreenBook blog post mentioned above
• Read Google’s white paper on Comparing Google Consumer Surveys to Existing Probability and Non-Probability Based Internet Surveys

 

[1] Denise A. Hines, Emily M. Douglas, Sehar Mahmood, “The effects of survey administration on disclosure rates to sensitive items among men: A comparison of an internet panel sample with a RDD telephone sample,” Computers in Human Behavior, Volume 26, Issue 6, November 2010, Pages 1327-1335, ISSN 0747-5632, 10.1016/j.chb.2010.04.006.

[2] Fricker, Ronald D. (2008) “Sampling Methods for Web and E-Mail Surveys,” from Fielding: Online Research Methods (Handbook) p. 195-217.

From FGI Labs: Survey Research on Your Mobile or Smartphone

by Rick Reed, project manager and mobile survey specialist

As part of our ongoing innovation efforts, we are constantly testing new and promising market research technologies, platforms, methods, and tools. We call this testing ground FGI Labs.

Our most recent lab topic is mobile market research.  In addition to rigorously testing the potential pitfalls of response bias from mobile respondents vs. personal computer respondents, we are testing consumer preferences for surveys on mobile and Smartphones.

This development is likely unsurprising to anyone following recent trends on the popularity of Smartphones. In fact, the number of US Smartphone subscribers surpassed the 100-million mark in January 2012, up 13 percent since October 2011 (according to comScore’s MobiLens Report: January 2012 U.S. Mobile Subscriber Market Share).

For our clients, this means that mobile surveys have serious potential, as that’s where their customers already are.  Mobile technology allows consumers to take surveys anytime and anywhere, even while buying your product or visiting your store (gaining location-based feedback and top-of-mind reactions).

Not sure if your customers are on board? Consider this: 43 percent of our SmartPanelists (representing consumers all across the country in all demographics) said they would be willing to download a mobile app that detects location based on their Smartphone’s GPS in order to respond to a survey invitation when visiting a certain retailer. This number will continue to grow as more mobile opportunities are offered.

Best practices

But not all types of surveys and situations are right for mobile. Through our testing and research, we’ve uncovered the following best practices and challenges in mobile research.

5 main challenges: getting the right sample segment, finding the appropriate mobile mode, determining the type of survey to conduct, and creating the best-fit questionnaire.

What’s working: short text based surveys (sensitivity to respondent tolerance), image/video uploads, simple surveys, and survey apps (beware of download tolerance).

What’s not working: long surveys (more than 20 questions), complex question types, failing to alert respondents to privacy/security issues, and insensitivity to personal nature of cell phones.

Where to start: determine what business opportunity a quick mobile survey might present (or question it solves) and poll your customers on their willingness to participate via Smartphone.

We are looking for a research partner to help us continue investigating mobile research. If your business has an opportunity that might be the right fit, schedule a call with my colleague Andy Smith, business development manager.

The Affordable Care Act: Your Brand in Vegas

by Philip Atkins, account director and health care practice leader

In 2014, the landscape of the health insurance exchanges will look a lot like Las Vegas. Each health insurance company will have its sales message and product offerings couched in flashing lights and bright costumes.

So how do you make your products and services stand out?

Your differentiator—like the pyramid that defines the Luxor or the light and fountain show at the Bellagio—will be in how you brand and advertise your offerings.

Below are the 4 key questions you need to ask before 2014 in order to keep customers in house and bring in others off the exchange.

1. How well will your brand image serve you?
2. What price premium does your brand image imply?
3. How do customers feel about you now? How much loyalty have you built up with them?
4. How do customers and other consumers compare you to your competition in key drivers such as physician network, cost, and customer service?

Where to start: the brand assessment study

To find the answers to the above questions is no little task, but reassuringly one of the best places to start, a brand assessment study, may give you the information you need.

A brand assessment study, which is essentially a survey of customers and consumers, will reveal the following:

• Strengths and weaknesses of your brand image
• How your brand compares to others
• How your company performs on product attributes
• What percent of your member base is predisposed to stick with you through 2014
• Why others will look elsewhere when offered the chance
• What might help you retain those customers if you made changes now
• The perception of the current cost for services, products, and programs

This knowledge can be a powerful motivator and change agent for your current marketing and brand plans. You can leverage it into your own Eiffel Tower like the Paris Hotel, so that in 2014 at the health insurance exchange your current customers will remain engaged with your offerings and new customers will want to learn more.