June 21, 2005
The Mystery of Mystery Shopping
Mystery shopping provides a real cost saving opportunity for most organizations. Simply put, most organizations would be better off not spending any money on mystery shopping because the data they collect is of limited use and can be misleading. Too often, mystery shoppers are easy to identify, are not representative of your customers, and conduct so few shops that the data received from mystery shopping doesn’t allow for detailed trend analysis.
Yet companies continue to spend $15 to $25 per mystery shop to get data that is of limited use and don’t routinely collect data from customers. They put more stock in the evaluation of people who don’t have a relationship with them or significant experience with their products and services rather than asking real customers to identify opportunities for improvement that impact loyalty and future purchases. The focus on mystery shopping rather than collecting real customer data has always baffled me. Does it make sense to you?
If you are unconvinced that mystery shopping has limited usefulness, here are some facts from a real customer situation that illustrate the point . . .
A regional bank with about 50 branches, 1,000 employees (500 of these employees are customer contact employees in the branch) and 130,000 customers uses mystery shopping as their primary means of customer data collection. The mystery shop focuses on Tellers, Customer Service Representatives and Location Appearance. Each branch is shopped at least once a month; larger branches may get shopped twice a month.
Here are some numbers for you to consider:
· With 50 branches getting shopped once a month (one Teller and one CSR get evaluated per shop), the minimum number of shops per year is 1,200. If you assume half the branches get shopped twice a month, then the number of shops per year is 1,800.
· With 130,000 customers, if we assume one branch or telephone service encounters per month, that equals 1.56 million transactions per year.
· With 500 CSR and Tellers, each person is getting shopped about once every three months. Over those three months, each person will handle about 780 transactions.
· Mystery shops are 0.11538% of the banks total transaction volume. For this data to be reliable, a good rule of thumb is that you need to collect data on 5% to 10% of total transactions, or 50 to 100 times what this organization is doing!
Still not convinced? Here are some other limitations to mystery shopping:
· You can’t link the data you get to your customer segments.
· The sample size is not large enough to make strategic or tactical changes. The feedback that you give employees based on their shop scores is not representative of their overall performance.
· Your customers have a vested interest in your business success—do mystery shoppers have the same level of interest and do they reflect the expectations of your customer base?
· Mystery shoppers can’t tell you what changes can be made to get more of their business or how likely they are to remain as customers. Without this data, it is impossible to identify which performance expectations actually drive customer behavior.
Mystery shopping, in and of itself, is of limited value. If you have a mystery shopping program in place and you cannot get rid of it, we recommend that you scale it back to focus on evaluating tangibles like physical appearance and traffic flows. These items have less variability than delivering the customer experience.
In high transaction volume businesses it is risky to drive strategic business decisions based on data extrapolated from individuals who are not customers, may not be representative of your key customer segments, and represent a very small proportion of customer touches to drive strategic business decisions. We believe that mystery shopping dollars will have a greater impact if used to measure overall customer satisfaction and loyalty of key customer segments on a consistent basis and check satisfaction with customer transactions over time.
| Poll | ||
|
Posted by Greg Robinson at 11:13 AM | Comments (7)
