Mystery Shopping Analytics – Out-Executing the Competition

Analytics refers to the gathering and interpreting of data in order to make better business decisions and optimize business processes. In mystery shopping, the most common use of analytics revolves around trying to understand how various interpersonal experience attributes influence customer loyalty measures. Correlations can be determined between loyalty and simple attributes from extending a thank you all the way to product knowledge. How important is it that the customer is thanked? How important is it that the associate smiles when greeting the customer? Answers to these questions allow organizations to more effectively allocate resources and focus on the factors that will bring about the greatest return.

In order to draw confident conclusions on various relationships with these target outcomes using mystery shopping data, it’s critical that the mystery shopping program has three elements:

1. Survey Design: A properly designed mystery shopping survey (analysis friendly).
2. Sample Size: A large enough sample size.
3. Skill: A skilled analyst who understands the client’s business and market research analysis.

Survey Design – Three Critical Design Objectives for Analytics

Survey design plays a critical role in enabling effective analysis of mystery shopping data. Each survey should be developed to include three critical design objectives:

1. Measure the shopper’s overall experience rating against an organizations loyalty measures such as likeliness to return or recommend the store to friends and family.
2. Measure key driver attributes known to influence customer loyalty.
3. Measure variations in performance on key driver attributes over time.

Overall Experience Rating – Net Promoter Score, Loyalty Three, Customer Satisfaction Index
To understand the influence of various experience attributes (e.g. greeting, helpfulness, attitude, etc.) on loyalty, the survey form must include questions relating to overall satisfaction or probable future behavior of the shopper based on the experience. An organization’s customer experience metric (Loyalty Three, Net Promoter Score, Customer Satisfaction Index) might encompass questions such as:

• Based on this experience, how likely would you be to return to this location again?
• If you were in the market for this product, how likely would you be to return to this location?
• Based on this experience, how likely would you be to recommend this location to family and / or friends?

Once this information is gathered, shoppers can be divided into segments and analytics can begin. Identifying the differences and similarities between the experiences of shoppers against the anchors of the Net Promoter Score, Loyalty Three, or Customer Satisfaction Index will reveal attributes that drive motivation to return / recommend and those that do not.

Measuring Key Driver Attributes and Capturing Variations in Performance
A high score doesn’t always equate to a shopper’s intent to return. For example, the associate scored a 95% on the mystery shopping form, but the shopper indicated that she was somewhat unlikely to return again based on the experience with the associate. When this occurs, there are typically two possible causes.

The first is that the survey form simply doesn’t include questions that measure key drivers of loyalty / advocacy (e.g. helpfulness, friendliness, knowledge). Instead, it may be very “compliance” heavy, measuring attributes that don’t influence the customer one way or another.

The second potential cause is that the questions designed to measure these attributes fail to capture varying levels of performance.

• Did the associate greet you? Yes / No
• Did the associate ask questions about your needs? Yes / No
• Did the associate make a recommendation? Yes / No
• Did the associate thank you? Yes / No

Using questions and response options like those above means a great greet, an okay greet and a poor greet are all grouped together as a simple “Yes.” A thank you that makes someone feel valued as a customer and one that seems insincere and scripted are both grouped together. As a result, it’s impossible to know how often great greets are occurring. There is also no way to conduct analytics to determine the influence of a great greet on the overall experience.

Sample Size Considerations

It is important to understand the influence of sample size on the accuracy and validity of analytics. For more information on this, see T&A Consultores CEO Marcelo Tarica and Service Evaluation Concepts CEO Arcadio Roselli’s article titled “Sample Size Calculation in Mystery Shopping Programs.”

Analytic Skill / Capabilities

The final component is finding the right resource to perform the analytics. With an adequate sample size and the right data in hand, a skilled analyst should be poised to engage in high value analytics that can drive decision-making. A skilled analyst will uncover relationships between the key driver attributes and loyalty measures. To tell the story of how performance impacts customer experience, the analyst must consider which statistics to use and how to convey the information in the most effective manner.

Statistical Caution and Top-Box
Using the right statistic can make or break the impact of survey data. If a mean of five is reported, the distribution of responses could have been all fives or half tens and half zeros-either way the mean is five. To assume these two scenarios are equivalent is an obvious misuse of the statistical mean as a productive metric. This is where choosing your statistical measure is vital. One common measure is the top-box, the percentage of surveyed customers who assign an attribute the highest rating; this is in contrast to the bottom-box, the percentage who assign the lowest rating. Depending how the attribute is correlated with the overall loyalty measure or outcome (or dependent variable) we can understand where to focus managerial attention.

Keep it Simple
Most managers don’t have hours to spend interpreting data. To proactively counter this time constraint, even the most complex information must be presented in a simple and visually stimulating fashion. Using cutting-edge statistical analysis programs can aid an analyst in creating thriving dashboards that are both relevant and simple.

Article courtesy of MysteryShoppingLive.com
Authors: Mike Jennings and Nick Vanderheyden

Reconstructing the image of Mystery Shopping

Few years back, a woman took a job in a superstore retail outlet. Every morning of her job started with a fear of mystery shopping. At every duty, of every customer she was scared. She was always too conscious – and the result was job dissatisfaction. However, this was the time when the idea of mystery shopping was sprouting its leaves. Many years have passed since those days and today mystery shopping is a lot different an issue.

Today, mystery shopping is no longer that scary, anonymous, voodoo like process which was totally phantom and even employees could never know of the outcomes. The programs available these days are much more interactive. The famous author Len Barry even suggested the phrase Virtual Shoppers to be used instead of Mystery Shoppers. According to him- mystery shopping the very word has a negative air about it- which creates a terrifying image among the employees.

Mystery shopping program can be made an effective strategy if coupled with proper explanation of the results and adequate rewards, reinforcements as well as training programs followed by it. However, the ultimate results of mystery shopping can be ripped only if the management: top and middle actively takes part in implementing the programs. A change has to be indoctrinated in the organizational culture- the frontline employees are to be incorporated in the strategic improvement.

• Building support for mystery shopping at management level:

It is crucial to get the support of the management in the mystery shopping program. How to get it? For the best shot you need to have a well-designed plan, a measurement system that is thorough and quantifiable, a recurring procedure, well defied objective and proper activation plan basing on the outcomes of the shopping.

As stated by Al Goldsmith, they portray the mystery shopping procedures to the clients as well as to themselves- as an integral part of the overall improvement procedure which is a factor of simultaneous sessions of measurement and according actions. For example, training sessions are designed and implemented basing on the outcomes of a typical shopping session. After the implementation the shopping agency goes on re-shopping to measure the effects again.

Such cycle of cause and effect should be elaborated to the management beforehand so that they are convinced. However, if the client is new in the field-relevant case studies can be shown to the client in order to make the claims credible. Sometimes rewards and incentives play important roles as well.

• Building support for mystery shopping at frontline employees level:

It is always prescribed to business entities to check on their level of employee satisfaction level if they find their customers consistently dissatisfied. A dissatisfied employee will always loath the job and the customers s/he is supposed to handle. Now there are much serious implications of this scenario. When a customer is handled badly- the company loses not only one customer- but rather a stream of potential as well as present customers. For, a dissatisfied or angry customer will surely tell others about his/her experience and this will affect the goodwill of the company hideously. The simplest real life example of this can be the restaurant scenario-

Suppose a waiter behaves badly with a customer. That customer along with his/her peers is hardly going to visit again. So, this is established that employee satisfaction is to be established in order to achieve maximum customer advocacy.

With today’s well designed mystery shopping programs- a retailer has nothing to worry about. Rather these days they look forward to it as they have incentive opportunities associated with it. In pursuit of achieving success in service marketing it is necessary to incorporate the culture of excellent quality assurance in the organizational culture. However, it’s very likely to be encountered with resistance within the organization at the verge of introducing a new culture. But it is also necessary that it is done- because in a service organization the contact between the frontline employees and the consumers is obvious and hence the culture of communication plays a vital role in assuring quality service.

• Building long term effect:

Build a team- yield the long term fruits. You can start by sharing the effects, the results, such as: the increase in sales, customer loyalties can be notified to the employees. When you are sharing such info it creates a sense of belongingness among the employees- they feel they are a part of a strong team which is destined to achieve the organizational objective. They find themselves as important players in the big picture.

As a company goes through a cycle of mystery shopping activities like: shopping and then training based on the results then shopping again, the company starts getting some tangible outputs like growth of revenue, bigger customer base etc. However, to make this step real effective it is important to have credible shopping results- and therefore it’s important to appoint skilled shoppers.

• To what extent the positive spin is to be promoted?

It has been established so far that there is chances of far better results if mystery shopping can be portrayed positively. Now to what level positivity should be portrayed? Is there any chance that too much positivity will end in negligence of the employees? So far a lot of research works have been done on this but no such results were obtained. It has been found out that the more mystery shopping has been made friendlier the more enthused the employees are to embrace it. The results of a positively imaged mystery shopping have always shown perpetual improvement.

While it’s true that, mystery shopping should be made interactive for the employees for better results- however, the best results cannot be ripped until or unless the employees put into action the trainings they have received and venture for better shopping score.
Article courtesy of MysteryShoppingLive.com

How to design an effective Mystery Shopping Program

While an organization is about to establish a new mystery shopping program to be incorporated within its system to evaluate performance efficiency- most of the times, serious confusions arise. This is due to the fact that, a good number of the people working in organizations are not clearly aware of the mystery shopping process. Being a new idea in the industry- there hasn’t been much scope for the establishment and practice of rigid rules regarding mystery shopping.

Now what would you do if you are a manager and want to integrate a mystery shopping program in your organization’s system? Surely, the feedbacks or references from the different departments aren’t going to be much helpful. They will focus on their individual needs and their agenda will be myopic to their scope only. As a result, you- the manager will always be in a to and fro run between the causes and the effects. However, the good news is managing a mystery shopping program need not to be that a hassle if you just can clarify some basic questions and can follow some necessary steps.

The burning questions!

To start with you need to be clear of your objectives and the best way to do this is answer the following simple questions:

• What result is to be gained from the program?
• What information is to be gained from this program in terms of operations?
• How will the top management accept it?
• What expectations does the top management have?
• How will the other functional departments of the organization be affected from the program?
• How the other departments need to be integrated in the system?

The Blueprint:

• The first thing to determine is the standards and factors- upon which the performances will be evaluated. If there is any operations manual or any other form of formal guideline within the organization that can direct the activities it is prescribed to follow.
• Most of the mystery shopper agencies have a generic format designed by them. However, many companies come up with their versions of compatible formats. The organization/manager should decide on which format to follow * While planning on the evaluation format- not all the factors should be given equal weight- therefore the factors should be assigned appropriate weights as per their importance.
• The next thing to consider is the key areas where the customers have repetitively complained, and also what complains has been given most. These issues need to be reflected in the evaluation.
• Now, the focus should be turned towards the format of the evaluation paper. The ranking, scoring methods, the length etc have to be decided.
• For evaluating more than one location- the frequency need to be calculated- i.e. how many times will each location be evaluated? Hence also this need to be clarified that, whether the evaluation will be done for a specific sample or for the whole population?
• If the evaluation process requires shopping/ expenditure of money- then the organization needs to decide on the mode of payment.
• What will be the duration of each shopping sessions?
• Will the shopper have to return the products/service that has been purchased?
• Will the same shopper be allowed in a location more than once?
• What other tools should be incorporated so that the subjectivity of the survey can be lessened?

The decision of the Administration of the system:

• Which functional department will be delegated the authority to manage the internal affairs with the mystery shopping? Is it marketing? Quality Assurance? Or Operations?
• Can a single employee be delegated the authority instead of a whole department? If yes, then what criteria should the person have to provide liaison between the internal affairs and the evaluation?
• Hence an opportunity can be assessed- in some agencies they provide personnel to administer the process- any organization finding it worthy can outsource the process.
• How the program can be linked to several HR issues- such as, compensations, bonuses, performance appraisals, motivational programs etc. It also needs to be decided whether the employees should be given monetary/non monetary bonuses.
• The format of the presentation of the evaluation is important as well. The manager/organization needs to decide before hand of the format of the report- it can be a summary, an analysis, numbers- according to the requirement.
• What profile of the shoppers is required to carry out the evaluation and that will compliment the profiles of the target customers
• Will the shopper be allowed to get recognized by the operations? What back up strategy should the organization have if the shopper gets exposed anyhow?
• What limiting factors does the organization have which might inhibit a shopper’s visit, e.g. business hours.

The Usage of the System:

• How will the information retrieved be analyzed and put into action?
• How can the program be included in the marketing tools?
• Several other organizational tools like motivational training, corrective process, incentive etc might be needed for the program to yield the best effect
• How will the problems identified be handled?

It’s true, designing a mystery shopping program requires much extensive research of the organizational needs. As a manager, you might find the numbers of questions to be answered pretty scary. But the fact is, as influential an effective campaign can be- a faulty one can be equally devastating. So do ponder and plan nicely!

Article courtesy of MysteryShoppingLive.com

How to Get Better Results from Mystery Shopping

In this era of communication, where a million ways exist to transfer vast amounts of information in the blink of an eye, clients always demand results as quickly as possible from their service providers. The field of mystery shopping is no exception to this, and it is very true that the internet and its various facets have changed the act of presenting reports on mystery shopping into a staggeringly fast one. But here we must stop to think: is speed more important than quality? Is the quality of the reports being sacrificed for greater speed?

In order for a mystery shopping system to be perfect, it must be designed suitably, and each module of the program must be properly implemented. It should incorporate a range of specific objectives, a form for the evaluation of the mystery shoppers (with appropriate fields), suitable shopping personnel, clear instructions, a well-planned schedule and timely presentation of reports. Presenting the report is what we are focusing on here. It incorporates everything from communication between the client company, the mystery shoppers and the mystery shopping service.

If the objectives of your mystery shopping program are to obtain a benchmark of employee performance and instigate motivation among your front-line employees, you must make sure that the employees are ready to accept the reports as facts. The client companies are very serious about having high-quality reports delivered on time. However, at times, quality and speed seem to become conflicting priorities, and in order to get the best possible report in the shortest amount of time, you need to iron out certain issues.

Validation of reports: Proper validation must be carried out for the post-visit reports of each mystery shopper. It can be done simply using a receipt from a cash register, which usually contains all relevant information, such as date and time or purchase, as well as the list of items purchased. Unless a report is validated, it will not be taken into account by any major shopping company. It is completely worth waiting for a copy of the receipt, as it can clear a lot of confusion both in the short run and the long run.

The value of the program: If your company aims to motivate the employees to provide good customer service to your patrons all the time, you must keep them on their toes so that they never ‘let their guard down’. Otherwise, their alertness may decrease over time. You can let them know about the shopping program, but never let them get any wind about when it is actually going to take place. If they get to learn its scheduled time, there is a good chance that they will remain alert only until that date, and begin to slack off after that. Many employees choose to take advantage of their employer in this way, and sometimes they do not even bother to hide it from customers. In order to keep your customers satisfied at any given time, you must keep your employees acting their best whenever they are working. On the other hand, if you do not tell them at all when they are going to be tested, they may start slacking off anyway, so it is best to subject them to ’surprise testing’.

The quality of the program: Validating the program for factual accuracy is just one step of the quality control of the whole mystery shopping system. If you want a well-done report, you need to be ready to spend. The lower-end companies only provide basic services such as checking whether all the necessary fields on the forms have been filled up. On the other hand, the higher-end companies provide you with complete reports of high quality by manually reviewing and checking all the individual reports for errors and consistent scoring patterns before compiling the final report. In such cases, reports which do not comply with the required criteria are rejected by default and replacements are ordered for them by sending in more mystery shoppers. While this obviously takes a bit more time than it takes to prepare a report that undergoes only the most rudimentary of screenings, the extra time is well worth it because of the substantially higher quality, since it will help you to structure your future decisions better, in particular the ones which you are making based on the contents of the report.

Field-level reporting: The importance of this particular factor cannot be overstressed. Quickly bringing about change to your front-line employees’ performance requires that the client company allows reports to be delivered directly to the field level rather than to corporate offices. While this requires proper validation and screening, they nonetheless make the whole process a lot less efficient by freeing the core management team from the hassle of having to deal with reports on the individual selling crew, leaving it to the local management instead. Not only does the local management know their employees better in personal terms, but they can also make sure of the validity of the reports far more easily than the core management can. It causes fewer questions to be raised about the repots, and allows the management to always keep the employees on their toes, since they would never be sure about when the next mystery shopper will pay a visit.

The cost does matter. In certain cases, you may find that it is more expensive to go for speedily done reports, since places which do not rely on quick-service systems (e.g. furniture dealers) and encounter fewer customers per day require a new shopper to be sent for every ‘mission’ in order to avoid recognition by the employees – a noticeably more expensive procedure. However, the internet certainly has made things easier by speeding up the data collection and validation process greatly, and it saves the most time when delivering the report, since the communication is more or less instantaneous compared to postal deliveries. Even after making allowances for receipt-based validations, a vast majority of reports can be dispatched to the clients within two days of the date of shopping.

Article courtesy of MysteryShoppingLive.com

Inside Marketing: Can You Really Measure Customer Experience?

Mystery shopping programs help direct staff training and measure how CUs stack up against competitors

By John Swinburn
Executive Director of MSPA

A senior citizen walked into his Dallas credit union to discuss his soon-to-mature share certificate. Interest rates had dropped, and he was looking for a better way to invest his money.

The teller directed him to a CU member service rep to learn more about his options, noting that possibly an annuity or a conservative mutual fund would better suit his needs.

Unbeknownst to the teller, this senior citizen was more than what met the eye. He was a mystery shopper, evaluating the customer experience. He was taking note of such things as whether the teller introduced him to the MSR, or just pointed and sent him away, and whether the teller restated his inquiry to the consultant, or left him, the valued member, to re-tell his story.

Credit unions are using mystery shoppers right now to monitor and improve everything from customer service and cross-selling techniques to compliance and fair lending practices and how well they measure up to the competition.

Often used as a tool to identify and continually improve customer service, mystery shopping can have a significant impact on every aspect of the customer experience, which ultimately drives the bottom line.

The impact of mystery shopping on credit unions has changed significantly over the years.

“Before, we had to explain mystery shopping,” says Judi Hess, owner of Customer Perspectives(TM) in Hooksett, NH. “Now that the discipline has proven its value to the industry, we need to explain new ideas for maximizing your program.”

Hess explains that an effective mystery shopping program approaches customer service improvement from several angles. The same shopping program can be used to measure and fine-tune training initiatives and highlight specific results with individual employees. Results could also be part of an incentive program on the individual or branch level.

Training and Measurement

$1.6 billion Travis Credit Union in Vacaville, Calif., uses mystery shopping on regularly to monitor all member touch points; including teller shops, loan shops, phone shops and new account shops. Management uses the data to pinpoint specific areas that may need improvement.

Travis CU’s January 2007 mystery shop reports were showing a score of just 62 percent for employees asking questions and listening for cues of how the credit union could better serve members. The CU implemented a training program that taught employees what types of phrases to listen for and what kinds of questions to ask. In a year’s time, the same question received an 82 percent score. During the same time frame, Travis CU had 8,014 direct new members compared to 7,842 in 2006.

“If employees aren’t listening and asking questions, they’re missing opportunities,” said Renee DeSantis, president, Game Film Consultants, a mystery shopping firm in Austin, Texas. “That’s where the value of mystery shopping is very evident. It clearly shows where to focus your training efforts.”

In its shop, $98 million Premier Federal Credit Union in Greensboro, N.C., was looking to improve its cross-selling. It leveraged mystery shopping results to pinpoint specific areas where employees could use additional training.

“Realizing that providing top-level service to members includes offering a variety of products to meet their needs, we began to focus on a ‘needs-based’ selling culture,” says CUES member Lori Thompson, executive vice president of Premier FCU. “Our mystery shopping program was a key factor in being sure employees were on the right track.”

When the mystery shopping program began, Premier FCU employees achieved a 52.8 percent score on closing skills, or asking questions to see where additional products may benefit members. Relying heavily on mystery shopping scores and training provided by its mystery shopping provider (Customer 1st, Greensboro, N.C.), Premier FCU was able to increase its closing score to 95.6 percent in just two years.

Measurements and Morale

Premier FCU also implemented an incentive program for strong shop results by publicly recognizing associates for a job well done. Sometimes associates are rewarded with movie tickets or other small but meaningful giveaways to recognize exceptional evaluations. This step can boost the likelihood of positive results, says Carl Philips, director of Customer-1st.

“It’s about catching them doing it right,” says Phillips. “The old adage, ‘You can’t move what you don’t measure,’ applies here. If associates know a mystery shopping program is in place to measure closing standards, then they will be more likely to meet those standards.”

No matter the type of shop, program or initiative, mystery shopping is an excellent opportunity to boost employee morale.

As Phillips pointed out, the simple fact that employees know they’re being evaluated is often an incentive to do good work. Recognizing employees for a job well done or talking through things that didn’t go so well is a form of hands-on training that gets results.

According to Bob Maietta of Service Evaluation Concepts, a credit union in Massachusetts uses its mystery shopping reports at quarterly meetings with loan officers. Team leaders point out the areas of the mystery shopping program that show weakness, taking out the names of employees and discussing ways to fix the problem. More importantly, they also point out the areas where employees do exceptionally well. That simple task alone helped improve mystery shop scores to the level of the credit union’s service standards.

Frank Aloi of ath Power Consulting in Andover, Mass., says mystery shopping can focus a credit union back on the basics of the industry.

“Credit unions were created to deliver the type of member/customer-focused service that banks want to be known for,” Aloi said. “We’re seeing the trend in the industry that mystery shopping is being used as one of the primary mediums to gather customer experience data. Effective shop programs tell management what really is happening out on the front lines.”

In addition to revealing a credit union’s strengths and weaknesses, mystery shopping can also be used to see how well a credit union is measuring up to its competition. Evaluating the same customer touch points at competitive banks and other credit unions allows significant insight into opportunities to gain new members.

“Using the information for coaching and training truly will create change and improve customer service,” says Brian Caldwell, client services manager for IntelliShop in Perrysburg, Ohio. “It’s a simple equation. Better customer service leads to better member satisfaction which equals a better bottom line.”