By: Madeline Varney
At this year’s 2013 TSE Services Member Satisfaction Summit we heard how electric utility cooperative members use customer satisfaction research to more fully engage with consumers and improve their products and services.
Member engagement, member satisfaction and best practices were trending topics for speakers at the summit as different cooperatives shared the challenges they face and how those issues are being resolved.
Here's what some companies are saying about customer satisfaction research:
-“It’s a reality check.”
-“I go straight to the verbatim responses to see what customers are saying.”
-“[We’re] empowering customers to make good decisions.”
For Owen Electric, part of creating a “culture of customer satisfaction” involved increasing the scope of their market research efforts by joining over fifty other electric utility cooperatives to more accurately track satisfaction scores produced by TSE Services and FGI Research. Overall satisfaction scores now keep them on point, but delving into the verbatim responses from cooperative members has transformed Owen Electric’s internal sense of accountability.
When every member interaction can influence the next satisfaction survey interview being conducted, employees realize how they can affect the bottom line and are eager to know the cooperative’s satisfaction scores. Linking those customer comments back to office managers, linemen, and those responsible for the topic in question propels problem resolution.
During the summit FGI’s John Blunk and Rob Killough gave the annual Member Satisfaction Survey Update, detailing updates to the survey instrument and outlining best practices for phone research concerning survey length and how to create unbiased survey questions among other topics.
Satisfaction may be a state of mind, but exploring the hard data provides tangible benefits.
By: Andy Smith, New Business Development Manager
After attending the 2013 EMACS and my 5th Chartwell conference, the trending strategies for utilities is becoming clear: Listen. Connect. Understand. Participants at the conference show us how they're doing this:
Listening leads to better business.
On day one Southern Co kicked off the conference with one of the key themes “know the customer” and “giving customers options has value”. She described how they use segmentation, target offerings and being more like a retailer as their goals.
Home Depot took a potentially damaging problem and turned it into a 3-year, company-wide initiative that lead to a BIG win – improving sustainability and wood purchase policy. They launched their ECO line of products; low VOC paints, low flow toilets, and CFL’s to improve sales and keep pace with customer needs. All this was initiated by listening, and a good example of the benefits received from understanding customer attitudes and behaviors.
Ameren is also taking the step to listen to their customers, and discussed their assembly of a cross-functional customer experience team – the first for the company with the mantra “put the customer first.” By doing this, they will measure everything as a best practice to show ROI.
AEP described the fear of new marketing retailers moving in to Ohio and Texas that will be able to offer several plan options to their customers. AEP is using incentives, experimental rate plans, and choice plans to stay competitive.
Connecting leads to better communication.
Avista is staying current by using social media to drive brand value – to educate, inform and engage. Their strategy starts with “influencers” like media, news, and sports outlets. Advocates and Ambassadors are now part of the plan and include sponsorships and non-profits to spread content via Twitter and Facebook.
Understanding pain points leads to increased customer satisfaction.
Duke Energy discussed using customer journey mapping in its efforts to understand their customers and uncover issues. Their goal was to “take an outside view in”. They used digital personas and included channels and research. Customers were participating and in the room, some were moderate and others were heavy digital users to provide immediate feedback. The nine-day session brought to light 44 digital journeys and 150 pain points.
We also heard from ONCOR on using focus groups to revamp mobile apps, Tucson Power on customer analytics and how they used and analyzed multiple sources of data to improve rate plans and collect more revenue, KCP&L on consistent messaging and PPL on how “Storm Sandy” changed everything about handling outages.
Chartwell ended the 3-day conference with the 10 catalysts for change based on their member survey. This entails how the digital user will be a huge driver of change, along with big data analytics.
How are you using these trends and staying competitive in 2014?
by by Dino Fire, Chief Science Officer
Goodness good•ness [goo d-nis] the state or quality of being good, excellence of quality. (dictionary.com).
A good predictive model is only as good as its "goodness." And, fortunately, there is a well-established process for measuring the goodness of a logistic model in a way that a non-statistician—read: the senior manager end-users of your model—can understand.
There is, of course, a precise statistical meaning behind words we propeller heads throw around such as "goodness of fit." In the case of logistic models, we are looking for a favorable log likelihood result relative to that of the null, or empty, model. This description isn’t likely to mean much to an end-user audience (it barely means anything to many propeller heads).
Saying that your model will predict the correct binary outcome of something 81% of the time, however, makes a lot more intuitive sense to everyone involved.
It starts with a standard hold-out sample process, where the model is trained and modified using a random part—say, half—of the available data (the learning set) until a satisfactory result is apparent. The model is then tested on the second half of the data (the testing set) to see how "predictive" it is. For a logistic model, a "confusion matrix" is a very clean way to see how well your model does.
Using existing historical data, say we’re trying to predict whether someone will renew their association membership when their current contract is up. We run the model on the testing set, using the parameters determined in the initial model-building step we did on the learning set.
logit.estimate <- predict.glm(fit, newdata = testing, type = 'response')
Let's set the playing field by determining what the existing "actual" proportions of the possible outcomes are in the testing data.
# Actual churn values from testing set
testprops <- table(testing$status) # create an object of drop/renew (actuals)
prop.table(testprops) # express drop/renew in proportional terms
So historically, we see that 59% of people don't renew when their membership period is up. Houston, we have a problem! Good thing this is a hypothetical example.
The elegance of logistic regression—like other modeling methods—is that it provides a neat little probability statistic for each person in the database. We can pick some arbitrary value for this predicted probability—say anything greater than 50% —to indicate that someone will renew their membership when the time comes.
testing$pred.val <- 0 # Initialize the variable<
testing$pred.val[logit.estimate > 0.5] <- 1 # Anyone with a pred. prob.> 50% will renew
With those results in hand, we need to know 2 things. First, how well does the model do in pure proportional terms? In other words, it is close to the same drop/renew proportions from the actual data? This is knowable from a simple table.
testpreds <- table(testing$pred.val) # create an object of drop/renew (predicted)
prop.table(testpreds) # express drop/renew predictions in proportional terms
Recall that our original proportions from the "actuals" were 59%/41%...so far so good.
Second, and most importantly, how well does the model predict the same people to drop among those who actually dropped, and how does it do predicting the same people to renew among those who actually renewed? That's where the confusion matrix comes in.
In a perfect (but suspicious) model, cells A and D would be 100%. In other words, everyone who dropped will have been predicted to drop, and everyone who renewed will have
been predicted to renew. In our example, the confusion matrix looks like this:
# Confusion matrix
confusion.matrix <- table(testing$q402.t2b, testing$pred.val) # create the confusion matrix
confusion.matrix # view it
Drop 310 55
Renew 62 189
Assign each of the four confusion matrix cells a letter indicator, and run the statistics to see how well the model predicts renewals and drops.
a <- confusion.matrix[2,2] # actual renew, predicted renew
b <- confusion.matrix[2,1] # actual renew, predicted drop
c <- confusion.matrix[1,2] # actual drop, predicted renew
d <- confusion.matrix[1,1] # actual drop, predicted drop
n = a + b + c + d # total sample size
CCC <- (a + d)/n # cases correctly classified
CMC <- (b + c)/n # cases misclassified
CCP <- a/(a + b) # correct classification of positives (actual à predicted renew0
CCN <- d/(c + d) # correct classification of negatives (actual à predicted drop)
OddsRatio <- (a * d) / (b * c) # the odds that the model will classify a case correctly
At 81%, our model does a pretty fair job of correctly determining the proportion of members who will drop and renew. It is capable of predicting the individuals who will renew their membership 75% of the time. More importantly, the model will predict who will not renew 85% of the time…presumably giving us time to entice these specific individuals with a special offer, or send them a communication designed to address the particular reasons that contribute to their likelihood to drop their membership (we learn this in the model itself). If we send this communication or special offer to everyone the model predicts will drop their membership, we will only have wasted (aka "spilled") this treatment on 15% of them.
Now that's putting big data to work in a powerful way. I predict that it's some really useful analytics for your marketing managers.
Editor's note: Our short Q&As introduce you to the hardworking individuals that crunch the numbers, make the connections, and change the industry. This month, meet Kerri Meyer Forguites, Project Manager.
Where is your hometown?
I grew up in Paul Smiths and Lake Placid, New York.
I feel so fortunate to have been raised in such a charming area, and I still consider the Adirondacks to be one of the most beautiful places in this country. I was given countless unique opportunities growing up in the North Country that I wouldn’t trade for anything.
What do you do at FGI?
I’m a Project Manager at FGI. I began my career here in 2007, fresh out of college, as a Project Coordinator. Assisting fellow project managers allowed me to easily transition into a project management role in 2011.
I currently work on projects in the retail, nonprofit, CPG, publishing, and energy industries, among others. Working with companies in such diverse industries keeps my job challenging and exciting and I always feel like I’m learning something new!
Why do you work at FGI?
Every project that I work on at FGI is different from the last and next. This makes my job interesting, challenging and rewarding.
I also work with a truly amazing team! FGI is full of creative, caring, entertaining, and helpful people.
One thing not many people know about me is…
I am obsessed with my dog, Mr. Fredrick! Mr. Fredrick, also affectionately known as Fred-Fred, Nugget, Bad-Boy, and the list goes on, is a rescued Olde English Bulldogge with a killer personality.
I work remotely from a home office in Saratoga Springs, New York, so I consider Fred to be my Research Assistant. During the work day, you can usually find him napping under my desk.
If I’m ever on the phone with you, and you listen closely, you might even hear him snoring away in the background.
If you could be a superhero what would your powers be?
If I could be a superhero, I would definitely want to be able to read minds. I would find it fascinating to know what other people were thinking, especially if they didn’t know I was a mind reader.
Editor’s note: this post is by guest blogger, Corey Dall, who served as marketing manager for FGI Research for 2 years before starting her tenure as the digital marketing manager for a local bank. This is her reflection on her time here and some of the lessons she learned from our industry thought leaders.
I’ve been lucky these past 2 years, not only because I worked for a company nominated one of the best in the Triangle and the team here is like family, but because I learned a lot about marketing (in my day-to-day work) and about market research.
In light of this, I wanted to share with you some of the tips and advice I’ll be taking with me in the next step of my professional journey.
1. Ask the right questions
It’s truly staggering to think of the ways good, reliable research can shape a company’s future. But if you don’t proactively seek it out, you could end up making apologies and potentially back tracking on decisions, costing your business millions of dollars (Gap logo redesign, anyone?).
So whether I’m handling concept tests in the future, or brainstorming ways to boost customer satisfaction and retention, I’ll begin first by asking: what research can we do to make sure our conclusions are the right ones for this business? What are we missing? What changes will make the biggest impact (and how can we prove it)?
The right questions at the start make all the difference.
2. Qualitative research is really neat
It’s no surprise to me that online qualitative research is the next big thing in this industry. For example, in an insight community (aka MROC) platform, which looks like the social sharing sites we’re all used to, you can gather verbatim phrases and keywords to use in campaigns and in future pieces of contents.
You can also collect pictures, videos, and reactions to images, making it easy to shop, eat, or see inside the homes of your consumers. For a marketer like me, this stuff is a goldmine, whether it’s for testing web usability, ad impact, or the viability of a product.
And if that wasn’t enough, good MROC platforms allow for a little bit of quantitative research as well, in the form of quick polls or open-ended responses to carefully crafted questions.
How cool is that?
3. Persona development is a must
In the market research world, segmentation analysis is how a company can craft its personas from quantitative data. Not only do you find out who is buying your product, but also why and what they’re most likely to purchase in the future.
You can craft names and stories for these personas, identify—beyond your product category—their likes and dislikes, family size, and propensity for social sharing (positive or negative). All of which is backed by hard data and analysis.
Imagine being a home goods or tool manufacturing company and knowing exactly, down to the color preference, what works (or won’t) with your biggest spenders and most loyal customers. Is there a hidden group of consumers you might not have considered before (as was the case in this study FGI did through a combination of social media mining and segmentation)?
In light of a new focus on journey mapping and customer experience across a number of industries, I think segmentation will be more crucial than ever.
4. Predictive analytics are part of the future of marketing
Some companies spend a lot of money to boost their JD Power score (or NPS) to see it either stay flat or nominally increase. Others throw time and effort into customer engagement ideas only to see them go down in a blaze of glory (Jello’s recent #fml Twitter campaign comes to mind).
In addition to asking the right questions at the start, it’d sure be nice to know the exact ROI for a particular time or money investment. You can do that with predictive analytics (marketers insert cheer here).
So every time your CEO asks why that vertical was promoted through PPC rather than a case study, or why a brainstorming session about better customer service is more essential than one on product packaging redesigns, you can use predictive analytics for definitive proof.
At each stage, you can compare your future scores against your current benchmark, see what shifts each step forward might make, and move swiftly and smartly through decision making. That’s why I think marketing and predictive analytics will have a nice future together, and I’m glad to get the heads up now.
5. Don’t forget about big data
FGI has covered big data a few times on this blog, from what it is and how to get started to its use with the aforementioned predictive analytics.
With each post, I got a better grasp of the potential that lies in data mining and appending, especially when transactional data meets behavioral. Undoubtedly, it’s a big mountain to climb for most companies, but I think the view from the top must be pretty darn nice (just ask Amazon).
I’ll likely keep an eye on this space as the FGI big data experts are just getting warmed up on the topic.
We’ve already blogged about the challenges life insurance marketers will face this year, and how these insurers can effectively reach untouched consumer segments (e.g. low to middle markets) and adapt to changing consumer bases.
And in April 2013, FGI Research received over 1,500 responses from its SmartPanelists about their perceptions of life insurance (LI) coverage. From this data, we discovered consumer attitudes towards purchasing LI and found common patterns among those who do and don’t choose coverage. Below is an overview of some key findings.
Education and marriage
Of the 1,151 people (71%) who indicated that they do carry life insurance, most tend to have higher levels of education (a four-year college degree or higher) and are married than the 475 (29%) who are uninsured.
College graduates have higher employment rates than those who have attained less than a bachelor’s degree. This increased financial stability, paired with a husband or wife to lessen the individual economic burden of life insurance, helps explain this trend among those with coverage.
Major barriers to purchase
For the 475 respondents remaining uninsured (29%), many cited expenses and cost as a major barrier to purchasing life insurance. These expenses included mortgage/rent and employment situations. Others felt purchasing LI as unnecessary for the time being.
Economic trends of the past 2 years—namely instability and lack of growth that lead to higher levels of unemployment—help explain these attitudes. People directly affected by the slowed economy are uncertain about spending their limited income on an expense like life insurance. Other financial burdens, like student loans, mortgages, and credit card debt, remain at the forefront.
Furture purchase likelihood
Of those currently uninsured, 51% felt as though life insurance was not necessary in the future, especially when household income is greater than $75,000.
This attitude likely results from younger consumers’ lack of knowledge about life insurance, how it ties into overall future financial plans, and the benefits it offers.
Additionally, a lack of trust in financial institutions and their agents to fulfill their promises and give objective advice contributes to the overall feeling that life insurance is simply not a necessity, according to Deloitte’s 2013 Life Insurance and Annuity Industry Outlook.
How age comes into it
Finally, age is a major factor in likelihood of purchase. Of those respondents who indicated they will not purchase life insurance in the future who belong to Generation Y (which consists of adults ages 18 to 34), 50% said they do not see the value in life insurance. On the other hand, the Silent Generation (ages 77 and up) and Baby Boomers (ages 56 to 76) have made other plans to secure their future, such as a 401K, IRA or CD, for example.
With those likely to purchase life insurance, the differences in attitudes between generations is apparent as well. Generation X and Y feel it is an important part of future planning, but not so much as Baby Boomers. At 74% Baby Boomers describe life insurance as a necessary expense.
There is one thing all generations appear to agree upon, however: they don’t think LI is affordable yet at this point in their lives.
Overall, insurers must be aware of the many factors that influence attitudes towards life insurance coverage, including the state of the economy, marital status, education, and differing generational beliefs.
Whether it’s showing members of Generation Y the inherent value and benefits of LI or gaining consumer trust, understanding the above trends will help insurers succeed in the current market.
by Philip Atkins, director of consulting services
The competitive landscape in the grocery industry just keeps getting stiffer.
Why? One important factor is that shoppers don’t rely on a single store anymore for all their needs. Instead, they are buying specialty items at one store, and more basic ones at another. (Read this article from the StarTribune that inspired this post.)
Additionally, “value” has taken on a new meaning for them. Not only is the price of groceries taken into account, but so is time and gas money.
This evolving trend in shopping patterns has impacted grocers in a number of ways: individual stores’ share of market has decreased, offerings on the shelves have greatly expanded, and slashing prices to draw in more consumers has become commonplace.
Is it even possible for a grocer to differentiate itself given the circumstances? Perhaps. But an even smarter move is determining why consumers shop the way they do.
As a grocer, what would you want to know about your customer when she walks into a competitor’s store? Maybe questions such as: “What did you buy?” or “Why did you shop at that store instead of mine?” and “What will make you consider my store instead next time?”
Now, wouldn’t you appreciate getting these answers right there on the spot, while she is still making her purchase decision?
Grocery and mobile: the new frontier
Fortunately, you can—and surveys conducted through mobile market research are the most effective way of reaching these on-the-go shoppers, who more than ever are using their Smartphones for grocery shopping, planning and researching.
In the FGI Labs, we have been experimenting with a mobile grocery app that would provide grocers with invaluable insights about shoppers’ habits, which in turn help increase customer loyalty and basket size.
One way this works is recruiting a group of your customers who are ready, willing and able to use their smartphones to take surveys. Next, this group of participants would download the FGI mobile app and enable geo-tracking. This way, the app is alerted when customers walk into a competitor’s store, and can immediately ‘ping’ shoppers a request to complete a survey that asks what they were shopping for, why they chose this store versus yours, and what you can do to change their minds next time.
As seen from the above illustration, gaining a deeper understanding of shoppers’ reasoning behind their behavior can benefit grocers immensely.
But surveying these customers on their smartphones while they are actually strolling down the aisles searching for products will reveal on-the-spot responses that won’t require troublesome recall.
Now that’s a good deal.
All of us here at FGI Research hope you and yours have a very happy and safe holiday weekend!
Editor's note: Our short Q&As introduce you to the hardworking individuals that crunch the numbers, make the connections, and change the industry. This month, meet Gandy Dorsainvil, Staff Accountant.
Where is your hometown?
What do you do at FGI?
I’m the Staff Accountant for our finance department.
Why do you work at FGI?
First of all, it’s an honor and a privilege to work for one of the best companies in the Triangle as well as the State of North Carolina.
In addition, it’s the flexibility that the company offers in terms of schedule, the great people, the relationship with our bosses… I need to stop here or the list might get too long!
One thing not many people know about me is…
That I’m a people person who’s very slow to react and quick to overcome.
If you could be a superhero what would your powers be?
Changing people’s lives one day at the time, fighting for equal rights, making the world a better place for everyone, regardless of your nationality, gender, or background.
One of my hidden talents is…
by Sarah Bishop, community manager
In the 2013 GRIT report, online insight communities sat at the top of the list for emerging technologies that market research clients actually want to use, beating out mobile surveys, social media mining, and text analytics for the honor. (A summary of this is illustrated nicely in an infographic.)
Given how personalized most other services (Amazon) and social avenues (Pandora, Pinterest) have become, I didn’t find the results altogether that surprising, especially when taken in the context of concept or product testing. Online market research communities (aka insight communities or MROCs) provide a more personal avenue to communicating and interacting with your customers than any of the other technologies on the list.
Part of this intimacy is due to the nature of the community platform. Customers are given quick polls, discussion prompts, pictures to mark up, or a chance to upload a video of their own (to name a few of the common tasks). All these touches inspire participants to share their true opinions about a concept to you (a captive audience).
The results, taken together or by looking at each participant, are more compelling and revelatory than a quant “like or don’t like” scale from 1 to 5 because it provides the “why” behind their choices and actions (or lack thereof). It becomes a new and often unexpected way to look at your topic; you see it as they do, in detail.
And as an added bonus, you get juicy keywords, targeted marketing messages, and more all in one package.
Quick case studies: insight communities in use
To further illustrate the benefits of MROCs in concept testing, I wanted to share a couple of short case studies. At the close of each of their communities, the companies came away with both short- and long-term recommendations for their concepts from engaged (and now, even more loyal) customers.
Case 1: A manufacturer, which offers a wide variety of products under several brand names, wanted to launch a new product under their private label and test the consumer acceptance of a number of potential product concepts in the home goods space. Their community revealed purchase likelihood, price point sensitivity, marketing believability, uniqueness of each concept, and brand implications.
Case 2: A national grocery chain was reconsidering their digital landscape and wanted to get feedback on their web elements, future ideas, and apps. In their community, they asked customers to react to screen shots, overall layout, usability, and future concepts. They were also able to discern from participants what customers actually wanted from a grocery store’s online presence, giving the company a broader sense of topics and products they might not have thought of before.
Case 3: A CPG company used an insight community as a vehicle to gather feedback from an in-home product usage test, the second part of the 3-part process. First, customers were surveyed online to determine which segment they belonged in, current feelings and likelihood to purchase this type of product, and of course, whether they would be willing to participate in further research. Those that opted in received the product at their homes. Then in the community, participants kept a daily journal of product usage as well as a running tally of their general thoughts about it, types of goods like it, and the brand in general. Lastly, participants were sent a similar online survey to the first to determine what variables might have changed after their in-home experience.