Metrics: How to Improve Key Business Results Part 15

Managers often pick the flashiest looking tools, and are disappointed by their inadequacy. I want to help you avoid "choosing poorly." The fear, uncertainty, and doubt around all metric programs stem from the misuse and abuse of the measures, in turn causing harm to those who are asked to provide the data. At times, this is a result of misguided or malicious management practices. Because I am an optimist and believe fully in the greater good, I believe that the misuse of metrics is rarely due to intentional actions. Most of the time, the abuse of metrics occurs because leaders are misled by any one of the deficient management techniques that use metrics improp-erly. These include the following: The use of stretch goals.

Measures as goals.

Targets and thresholds.

Incentive programs based on measures.

These techniques may seem quite innocent and proper. There are authors, experts, and consultants who promote each, if not all, of them. Even successful CEOs will applaud the use of these "tools" for improvement. Do not be fooled: these are misuses of metrics, and any success attained by these paths is temporary.

Let's look at each briefly, and then introduce a simpler solution.

The Use of Stretch Goals.

Using goals to strive for success is a good thing. The misuse of this concept by creating "stretch goals" and using measures as goals is where leaders fall astray. Goals in themselves can be a great tool. Vision, mission, and goals are a potent combination for steering an organization forward. But the misuse of goals can cause an organization's efforts to crash and burn.

Invariably, stretch goals are not independently identified for self-improvement. In business I find that managers think it's a good tool for motivating the workforce. Achievable goals aren't enough. "Stretch" goals are the latest silver bullet. The leader sets stretch goals for a department or staff member-and voila! The goal is attained. The leader looks brilliant, the department/ worker is applauded, and everyone is happy. Except that in most cases, the stretch goal was dictated (rather than collaboratively agreed upon) and the workers have no ownership. No ownership means less pride in the accomplishment.

The stretch goal is typically a big task to be achieved. This is fine if it ends there. But the school of thought is to upgrade the first stretch goal with a tougher, bigger stretch goal the next year. And so on, and so on. The workforce eventually catches on to this.

I can clearly hear the orator in the coliseum announcing, "Let the games begin!"

The manager keeps pushing to see how much he can get out of the workforce. The workforce wisely realizes how late they have to achieve the goal within the required time period (finishing early would mean a new stretch goal before the year is out-better to deliver on time than early). Stretch goals do not belong to the organization-and they definitely do not belong to the workforce. No, these goals are solely owned by management.

Another unintended consequence of stretch goals is that other work will suffer. Work that may have been performed to an exemplary level is reduced to "good enough" since it is not a part of the stretch goals. It is hard to stay focused on the big picture when mini-goals are set up as the standard of excellence.

Measures as Goals.

Another common problem lies in the misuse of the measures of success (MoS) as the goal itself, instead of focusing on the identified requirement. Sometimes this becomes a slight nuance in meaning. If your task is to sell five thousand units of the new product are you measuring to the goal or to the MoS?

If you said the goal, you'd be mistaken. A simple test is, if you sold one more (or one less) than the "goal," would you be happier or not? How about ten more (or ten less)? One hundred? If it matters, then you're using the measure of success as the goal.

Where this becomes important is when the workforce needs to understand the purpose, the big picture, the mission and the vision of the organization. When you treat the worker as a partner in improvement, you don't attempt to manipulate (motivate) them by a.s.signing "stretch goals" or by having them chase the measures instead of achieving the goal.

The true goal may be to improve the bottom line. Increasing sales is one possible task toward that end, and the idea of selling any particular number of any particular item would be a subtask. The number of items sold would const.i.tute a measure of success.

When you chase the MoS you lose focus. You may reach the measure while failing to achieve the guiding goal. For example, you may sell five thousand units of the new item, but sales could drop significantly in the other product lines. This would not end up helping you achieve the goal.

Another way to look at it is to remember that any measure of success (or full blown metric) is only an indicator. Therefore, you should not celebrate the meeting of a MoS since it is not the goal; it is only an indicator of possibly reaching the goal.

Even when measures of success are properly used, managers occasionally make them tools for abuse by setting targets and thresholds.

Targets and Thresholds.

Targets, like stretch goals, have the nasty habit of being re-written each time they are achieved. If you have a target, you can bet that it will be reached at the eleventh (if not final) hour. Again, there's no benefit to the workforce in achieving the target earlier than asked for since the most common response (after the small celebration and reward) is a new target to reach.

Targets (when misused) aren't moved as a result of logical, data-driven decisions. They are moved based on their achievement, or lack thereof. This means that the new target is set based only on how well you met the previous one-again using a poor indicator as the "truth." The workforce again will come to understand the rules of the game quickly. If they reach the target early, the reward is usually the same as if they reach it at the last possible moment. Some managers reward last-second heroics more than they reward early achievement. Early success is seen as the manager's underestimation of the capabilities of the workers. The target wasn't "tough" enough, so why highly reward for reaching it?

And the games continue.

It behooves the workers to barely reach the target. The manager sees himself as an astute judge of the workers' capabilities and a shrewd motivator. The workforce is seen as hardworking and able to be encouraged. Everyone is happy.

Thresholds are the same as targets, but from a negative point of view. Instead of setting a bar higher each time and trying to reach it, the threshold sets the minimum acceptable performance and challenges the workforce to stay above it throughout.

Again, the workforce is smarter than the manipulators believe. They will stay above this minimum, but barely. If they exceed it too significantly, the minimum will be raised until they fail, at which time they will incur the unjust wrath of management for being incompetent.

The stories of workers telling new hires to "slow down" because the over energetic rookie will make them look bad is not a myth. Workforces learn the rules of the game and quickly adapt to the motivational tools used by poor managers.

You're probably wondering what the problem is with clearly articulating the acceptable levels of performance. It would seem to be good communication to make this clear to the workforce. And you'd be correct. But, when targets and thresholds are misused, they never stabilize-they constantly move in reaction to their achievement. As the saying goes, it's hard to hit a moving target.

Incentive Programs Based on Measures.

The last misstep I'll cover is building incentives around measures. Again, as with stretch goals and using measures of success as the goal-the emphasis (and reward) is placed on the wrong thing. There was an Olympic weight lifter who was given a large monetary reward for every weight-lifting record he broke. Not surprisingly, he broke his own world records incrementally-with the smallest increase in weight allowed to count as a new record. He was able to break world records many times. This was probably not exactly what the establishment was trying to motivate with their incentive program. But since the measure was the focus, rather than the goal, the results were naturally off center.

More enterprises than you can imagine have gone out of business due to misplaced focus. This absence of focus includes forgetting who the customer is, forgetting why the organization exists, and forgetting the goals of the organization. Focusing on individual acclaim rather than organizational success, on incentives rather than overall excellence, on measures instead of the goals, and on thresholds, targets, and incentives around measures-are misaligned with what is important.

Expectations.

The use of expectations is not a conceptual or theoretical breakthrough. It is not a silver bullet. It, too, can be abused and misused.

Expectations help you stay focused on what is important. They provide the ultimate context for your metrics-based fully on the customers' point of view.

Expectations provide the ultimate context for your metrics-based fully on the customers' point of view.

Expectations are a clear description of what the customer expects from your service or product. Of course, you can use expectations for any of the views in the Answer Key, but we're focused on effectiveness, so we'll stick to the first quadrant.

The following are the questions that you will ask: What level of service does the customer expect?

What is the quality of product that the customer expects?

Will exceeding these expectations benefit the organization?

Will failing to meet these expectations hurt the organization?

Why does the customer have these expectations?

With expectations, we start with the a.s.sumption that the customer has a range of expectations that you will provide. Let's look at the service center/help desk again as an example.

When a customer calls the service center seeking advice, he has some general expectations. For example, when I buy a product online and consequently need to get help from a service center, my expectations are rather low. My first expectation is that I won't be able to easily find a phone number to call on the company's web site. I am rarely disappointed-it seems not matter what link, icon, or b.u.t.ton I select, I am given more opportunities to buy something. After a diligent search and more keystrokes (and clicks) than I want to count, I find the phone number. After dialing it, I expect that I won't get to speak to a living person until I spend at least five minutes navigating the auto-response call system. Of all the choices provided, (1 for new service, 2 for extending your service, 3 for adding a service, etc.) none of them will be, "speak to a service representative." So, without listening to all the choices, I press 0. Then, I get the "sorry that is an invalid choice, please try again." So I press 0 again and before the same recording can apologize for my mistake again, I press zero once more and, yes, I finally get a living, breathing representative to answer my call!

Now, my expectations change: how long it will take for a technician to actually engage me in conversation? Once I've explained my problem to the technician, how long will it take him to a.s.sist me? How well will the technician's solution work? These are all expectations that will help the organization define the measures used for performance.

If you're measuring time to respond and time to resolve-two natural measures for a help desk, then you will need to understand expectations from the customer's point of view, not management's.

If you simply look at the measures without expectations, you will quickly realize something is missing. Figure 8-1 shows this.

Figure 8-1. Cases resolved in less than 8 work hours Figure 8-1 is a chart showing the percentage of help-desk cases resolved within eight hours. Does the chart tell a good story or a bad one? Are the results in March acceptable? Are the results in August acceptable? Is the overall percentage for the calendar year (CY) within acceptable parameters?

Without expectations, we don't know if the story is a good one or a bad one.

When you gather the expectations, you'll need to determine what "meets expectations" means. This should almost always be a range, and not a single value. In the scenario of "time to resolve" we may have different expectations based on different criteria-like the level of the problem. If the customer's issue is a simple one, the expectations are for a quick resolution. If the issue is a complicated one, then the expectations won't be as stringent.

Let's continue with a simple problem: you need a pa.s.sword reset. How long should this take? My expectations are that I should have it reset within an hour from whenever I actually get to speak to a living person. I expect that I will have to prove who I am-the proper "owner" of the account that I have lost the pa.s.sword to. Once I've answered the required questions, I expect to receive an e-mail (another security measure) with a new pa.s.sword. While most times I get this e-mail within minutes, I'm willing to accept up to an hour. So, my expectations range from five minutes to one hour.

Those are my expectations. Meeting these expectations would make me a satisfied customer. I don't require more, and I would be dissatisfied with less (if it took more than an hour, I would likely call the help desk back). If the resolution took less than five minutes, I'd be happy. Happy and surprised. But I would not expect it to happen the next time. And more importantly, I wouldn't be dissatisfied if it took longer (but still within my hour time frame). If I received the e-mail before I hung up with the technician, I'd probably even remark on the speed. Again, it wouldn't change my expectations, but I'd be happy about the unexpected. I'd actually wonder how it was possible.

This is the other essential difference with expectations-they are not changed easily or readily. When an expectation is met, the customer is satisfied. When it is not met, they are dissatisfied. When they are exceeded, it is truly a surprise. A happy surprise, but a surprise nonetheless. The customer doesn't raise her expectations every year, or if her expectations are constantly met. Meeting expectations doesn't necessitate their change-it affirms them.

Let's see what happens when we add expectations to the chart. Figure 8-2 shows the same data as in Figure 8-1, but with expectations added.

Figure 8-2. Adding expectations The dotted and dashed lines indicate the direction the expectations flow. Expectations are met when they remain within the boundaries. Anomalies above or below the range of expectations are worthy of investigation.

How do you determine the customers' expectations if your customers are a large, diverse group? For example, in the case of international companies like Amazon.com, McDonalds, or Ford Motor Company? Even if it were possible to ask each customer about personal expectations, everyone would not come to a consensus. Therefore, it is important for you to come to a decision about the general expectations of the customer base.

From the organization's point of view, it is important to meet customers' expectations. If the organization cannot do so, it has to change so that it can. This is the foundation for a Service/Product Health metrics program. When the organization fails to meet expectations, further investigation should take place to see if the cause can be avoided in the future. Most organizations strive to do this already. Unfortunately, these same organizations forget to celebrate the times they meet expectations-and instead expend their energies punishing themselves for the few times they fail to meet expectations. These organizations also celebrate and reward times when they exceed expectations. But, if the expectations are correctly identified, any deviation-whether below (failure to meet) or above (exceeding)-should be investigated.

If there's a failure to meet expectations, determine if the occurrence can be prevented in the future. Perhaps a change in process or procedure can eliminate the occurrence.

If results exceed expectations, determine if the occurrence can be replicated-within manageable costs. Since exceeding expectations is not normal, chances are that something else was sacrificed to achieve this level of service or product quality. Either a different service was neglected, extra time was required, or extra expenses were invested. If this is not true, it would not be an anomaly, it would be the norm. And it would be within the customers' expectations.

There is a small nuance differentiating targets and the upper limit of expectations. If you consistently exceed expectations, then either your determination of what the customer expects is not ambitious enough or your service/product quality levels are such that the customer will truly expect more from you.

The scenario I gave about my expectations in re-setting my pa.s.sword was true. But if I were to discuss my expectations of a third-party delivery for a product I bought online vs. my expectations for Amazon.com's Prime sales (two-day, free delivery), they would be extremely different. As would be my expectations if I paid for overnight delivery vs. standard shipping. Overall though, whenever I see that a product I purchased is supplied by Amazon (vs. a third-party choice), I have higher expectations of the speed of delivery and the quality of the packaging.

The same will happen with your customers if you consistently exceed expectations. The question is: can you afford to do so consistently? The best-case scenario is that you are exceeding expectations because of a change to your processes or procedures. If your continuous process improvement efforts (Total Quality Management, Kaizen, or Six Sigma, for example) are successful, you may be delivering better and faster than before, without degrading other services or increasing costs.

If this were a "target," management would look at even one occurrence in which you exceeded the target and demand that you do so increasingly often each year. Or it may raise the target the next year.

When you exceed expectations, it is as much an anomaly as when you fail to meet them. Granted, the customer is not dissatisfied, but you will still need to investigate the causes, because your organization may not be able to afford these anomalies. Therefore, don't automatically applaud them as a good thing. Leadership must be as curious about the causes behind exceeding expectations as about failing to meet them. Granted, exceeding suggests that you are not running the business into the ground (Efficiency Measures) or that you are not causing poor customer experiences in other service or product areas. But this is why you cannot afford to push for or reward exceeding expectations.

Meeting expectations is truly the preference.

A more thorough representation of the measure would make the expectations band stand out. And don't use the cla.s.sic green is good and red is bad style. Figure 8-3 presents a clearer picture of the expectations for cases to be resolved within eight work hours.

Figure 8-3. Neutrally colored expectations.

Discovering Expectations.

Many times when I'm helping a department define their metrics, the managers don't know the customers' expectations. I usually push and prod with questions like, "What would make your customers dissatisfied?" Or, "What would make them take notice?" I'll use their customer satisfaction surveys (remember, almost everyone already has these in place) to see what customers have said about speed, availability, or accuracy.

Normally, using a little investigative tenacity, I can create an initial range for the customers' expectations. Once I've a.n.a.lyzed enough data, and considered anything usable from satisfaction surveys, the norm is easily identified.

This norm provides insight to the stability of the processes and to the level of service normally provided. I use this as a guide to work with the department to create expectations. This is necessary because usually the customer base is just too large to query a complete representative body. After I've determined the norm, I ask the department if it seems right.

Figure 8-4 shows an example of data-call it trouble calls resolved within eight hours or less-collected over three calendar years.

Note: If you don't have historic data, you may not be able to conduct this exercise until you have collected enough data to compare to.

Figure 8-4. Sample data When there is data for multiple years and several data points, it becomes easier to determine expectations. In the case of Figure 8-4, I would ask the customer if Year 1 was a "normal" year. Usually, the answer is that it was a below-par year for them. The poor results, compared to other years, were usually caused by a change in process, leadership, or system. Regardless of the reasons, I only want to know if, what took place during that year could be considered normal.

I then check Year 2 and Year 3. Chances are that these years were more normal and the department felt good about its overall performance during these periods.

Using only Years 2 and 3, I work with the department to see what the data is showing as "normal." This is seen in Figure 8-5. I disregard Year 1, the "abnormal" year. Even if it had been an exemplary year-with results consistently close to 90 percent, I would remove it because that, too, would not be "normal."

Figure 8-5. Two-year sample At this point, I again ask the department to use their collective memory and tell me if the lows and highs were abnormal. Rather than try to get them to set a range, I just ask them about the extremes.

"In August of Year 2, you were below 75 percent. Was that normal?" If the answer is no, I will ask about March of that year.

"It seems that March is traditionally a 'bad' month-is that right? Were the past two Marches abnormal for any reason?"

I will ask the same questions about August of Year 3, in which the unit reached its highest numbers, and then work my way down. You can also use statistical a.n.a.lysis for this purpose, but I find using the charts of the data much less intimidating to the department. Also, statistics give the impression that creating the range is not up to the department.

It is important to give the unit ownership of both the metric and the team's performance. By having the members of the department collaboratively determine the customers' expectations, you gain benefits from the start, as follows: Ownership of both the processes and team performance levels A common understanding of what the customers expect An open discussion of the previous highs and lows-without negative or positive connotations. The department learns to view metrics as input, rather than drivers of consequence.