What is an Average Customer Experience?

Most organizations collect and report on metrics that are not descriptive of their processes. Some of you may have noticed that most metrics that are reported are the “average” or the “median”. Most people do not understand elementary statistics and their application to business. Here is the truth of the matter: Your customers do not feel the average — they feel the variation.

An inside-out view of the business is based on average or mean-based measures of our recent past. Customers don’t judge us on averages, they feel the variance in each transaction, each product we ship, each user interface we build, each online process we create, each interaction we have on the phone, each corresondence we have through email or a letter, and every other process that touches the customer in some way or form — online or offline.

Customers value consistent, predictable business processes that deliver world-class levels of quality. They feel the difference, not the average.

A few examples

For qualitative measures, such as “taste” or “texture” of a consumer good, we do not think in terms of an “average” taste. Instead, we as customers, think in terms of “oh, that tasted sweeter the last time I bought it” or “dish #2 from restaurant x tasted creamier this time than last.” We feel the difference, not the average. Yet, even qualitative measure rolled-up in company meetings report on the average or mean.
For quantitative measures, we feel the variation, not the average. For example, (a)”The last time I ordered from Amazon, I received that package in 4 business days; but, on other times I receive my order usually after 6 days on standard shipping” or (b)”Yesterday when I went to Taco Bell during lunch hour, I was in and out of there within 10 minutes, but on other days I’m there for my whole lunch hour” or (c)”Since web x.0 company changed their user interface, it now takes me at least 3 more minutes to do what I used to do with the old user interface.”

These are some examples of quantitative measures. Yet, most likely, these types of metrics are reported as an average to upper management. Example (a) is a time-based measure, where an expectation was made from prior experience on deliver times; example (b) was also a time-based measure based on prior experience on server times; and example (c) is an example of a task-based usability test based on time, yet this metric will most likely be reported as an average such as “80% of of our population that follow this path complete this task with an average time of 245 seconds.”It is a problem when our measures do not reflect how our customers are really feeling. Here’s a real-world example:

I was in charge of a click-to-ship process where the Marketing group made a customer promise that orders would be delivered within 10 business days. This is a great marketing tactic, but what does the data say about the current process capability?

After we collected some data, we showed that the data could be approximated by a normal distribution — this was the first step. Then, we looked at the distribution to find the Mean and Standard Deviation (which helps us know the variation the customer’s might be feeling). We discovered that, on average, this process was capable of delivering within ~7 days. But, what was the variation in this process? The Standard Deviation helps us approximate the “pain” that our customer’s might be feeling from this process. Based on the data we collected, we show that 68% of all orders will be delivered within ~2.96 days and ~12.16 days.

This data shows us that, some of the time, we will not make the customer promise of 10 business days, contrary to the Marketing Department’s promise. Moreover, there was a disparate gap from the Mean, which shows us that there is a lot of variation in this process and helps to describe what the customers of this company might be feeling.

After understanding what the data showed us and better understand the process capability, my goal was then to “shift closer to the mean”, which is a technical way of saying that we want more of our orders to come closer to being delivered around ~7 days. Other technical terms are “shift the mean”, or “closing the standard deviation”. These are terms in basic business statistics that I’ve discovered a lot of business people are completely clueless about.

The results

After better understanding the process and identifying ways in which it can be improved, we implemented some process improvement with our suppliers, internal processes, and also with shipping vendors. After some time, we reduced the variation by 30%, bringing the delivery times closer to ~7 days. The next steps would be to continue on that process where the curve is more centered around the mean, which would be a picture showing that, indeed, most orders are being delivered within ~7 days.

Conclusion

Make sure that your metrics reflect what your customers are feeling. The “average” is an inadequate measure and is not descriptive of what the customer is feeling. There are other measures that basic business statistics makes available to us that will help us (1) understand the customer better, (2) understand better where we can improve (3) identify ways in which we can further delight the customer.