Metrics: How to Improve Key Business Results Part 6

"And what would you have done if you didn't sell them all? What if you sold only half as many as you ordered?"

"I don't know. Probably order only three-quarters next year and try harder to sell them. Or maybe find a different fundraiser?"

"John, one of the things I encourage clients to do is determine how they will use the answers to their questions before they collect any data. So, I would have asked you-what will you do if the answer is 'very successful?'"

"Tremendously successful!" He said with a smile.

"Tremendously successful. The point is, if you told me that you'd double your order I would help you get to your real question-which sounds like it's 'How many coupon books should I buy next year' with the possible answer being anything from none-drop the program-to doubling the order."

"What's the difference?"

"Well, if we go with the new question, you'd look at your data differently, and you'd also collect some other data, like: Who made your sales?

Who bought the coupons?

Why did you sell out?

How many requests did you have that you couldn't satisfy?

How did you market the program?

"I hear what you're saying, but I don't see how any of that would make a difference."

"I'll give you a simple scenario. What if I told you that you sold half of your tickets in one day? A week before your deadline, you sold all the tickets you had left, all in one day."

He stopped and thought a bit.

"Half?"

"Yes."

"Hmm. I might have to know why we sold so many in such a short span of time."

"Good. And if I told you that they were all sold by one person?"

"Wow! I'd have to give that person a big thank-you card. Perhaps we could give them an award?"

I gave him a look that only a parent or princ.i.p.al could understand.

"I'm jumping to conclusions again?" he offered.

I nodded.

"I guess I'd want to know how that happened."

"Good. Let's say you find out that someone, an anonymous donor knew that you had only sold half of your goal after two-and-a-half months into your program. Let's say that the donor decided to help you out by buying all of your remaining coupon books."

"What did they do with all of them? That's a lot of books! Wait, I should care, right? I mean that's important, isn't it?"

"Actually, I'm glad you want to know. That means you're not jumping to conclusions and you're starting to think of the measures properly. They should be something that you use to help you think. Good job."

He smiled.

"The donor gave them away to people like a secret Santa. He just gave them out like candy. Now, what's more important though is-what if you double your order next year?"

"Well...I guess if the donor doesn't decide to bail us out, especially since we could have three times as many coupon books left at the same time in the program... Oww. It could be a catastrophe."

Now I smiled. "So, let's talk about your real information needs, what you really want to know, and we can build a metric if need be."

Designing metrics, like almost every organizational development activity involves much more than the actual task. You get a lot more out of the effort to design a metric then just the metric. And sometimes you don't end up with the expected result (a metric) at all, but you always end up with something useful.

Planning a Good Metric.

Where to Begin.

In Why Organizations Struggle So Hard to Improve So Little, my coauthors and I addressed the need for structure and rigor in doc.u.menting work with metrics. More than any other organizational development effort, metrics require meticulous care. Excellent attention to detail is needed-not only in the information you use within the metric (remember the risks of human involvement), but also with the process involved.

In this chapter, we'll cover identifying the many possible components of a metric development plan and doc.u.menting the metric development plan so that it becomes a tool for not only the creation of the metric, but a tool for using it effectively.

The Metric Development Plan.

We already discussed the building blocks of the metric. If we envision metrics as a house, the root question can logically signify the foundation of the structure. Without a solid foundation, the house may not last (ask the Three Pigs). Of course, if the structure isn't intended to be a long-term home, then the foundation becomes much less important. We also discussed the load-bearing wall (metric), the two-by-fours (information), the drywall (measures), and the nails (data). We haven't yet discussed the various other items that go into building a house or the components that go into turning a structure into a home.

While a metric is designed to answer a root question, and uses information, measures, and data to give it substance, metrics also require other components to make it a useful tool. These components are included in a metric development plan.

The Components of a Metric Development Plan.

A metric development plan captures all of the components of the metric-data, measures, information, pictures, and of course, the root question. The plan also doc.u.ments how these components are collected, a.n.a.lyzed, and reported. The development plan includes timetables, information on who owns the data, and how the information will be stored and shared. Like most plans, it gives guidance from the beginning (designing and creating the metric) to the end (reporting and using the metric). The plan may also include when the metric will no longer be necessary.

The metric development plan provides a game plan for making metrics well-defined, useful, and manageable. At a minimum, it should include the following: A purpose statement An explanation of how it will be used An explanation of how it won't be used A list of the customers of the metrics Schedules a.n.a.lysis Visuals or "a picture for the rest of us"

A narrative The plan not only helps in the creation of the metric, but it also provides guidance for the maintenance and final disposition.

If it's worth doing, it's worth planning to do it right.

The Purpose Statement.

Is the purpose statement the same as your root question? The answer is, "maybe." You will doc.u.ment the purpose statement in your Development Plan when you identify the root question as shown in Figure 3-1.

Figure 3-1. Purpose Statement If you have a well-formed root question and you have dug as deep as you can, your root question may very well contain the purpose statement. Consider the following example: Root Question: "How well are we providing customer support using online chat?"

Purpose Statement: "To ensure that we are providing world-cla.s.s customer support using online chat."

Not much of a distinction between the two. It does provide a clearer requirement. It gives the underlying reasons for the question, and therefore the metric. The purpose of the metric is usually larger than providing insights or an answer to your root question. There is usually a central purpose to the question being asked. This purpose allows you to pull more than one metric together under an overarching requirement.

Let's take my example of customer support through chat. If we kept asking "why," we may find that there is an underlying need to improve overall customer service/support to the point that the customer support team is a key component to the overall organization's success. The requirement is to make the customer support a selling point to customers. It should be one of the reasons the customer becomes a loyal, repeat promoter of the organization.

This underlying purpose will give us a much clearer guide for the metric. It will also allow us to identify other metrics needed, if you are ready to do so. You may have to settle for working with the question you currently have and get to the bigger-picture needs in the future. But it's always best to have the big picture-it allows you to keep the end in mind while working on parts of the picture.

I've been working lately with numerous clients in another area of organizational development that I find extremely rewarding-vision setting. Much like the work we did to identify the root question, vision setting requires getting to the underlying need or want. In vision setting, I do not advocate a wordy statement. When we write a statement, we often tend to use flowery, multi-syllabic words to describe something that should be extremely simple. Your purpose statement should not be crafted for beauty-you're not going to frame it. It should be edited for clarity. Remember to define every word and clause in the statement.

Your root question is the foundation of the metric and the purpose statement. If you've identified a good root question, the question and your purpose may be one and the same.

When doc.u.menting the metric development plan, I make a point to capture both the root question and the purpose. If they are synonymous, no harm is done. If they are not, then I gain more insight into what the metric is really all about.

Allow me to give you an actual example. The executive director of a center for women and I were working on a possible metric and our conversation went as follows: "I want to know how many women come through the door each day," she said. This seemed a valid question, but not a fully vetted one.

"Why? What will you do with the number?"

"I need to show our board of directors our worth."

I asked, "Will they know your worth based on the number of people who come through your door?"

She thought for a moment, seeing my point. "No. Our worth is based on the number of women we help."

I said, "There may be other ways to measure your worth, so before we go into actual measures, let's look deeper at your question. Why do you want to prove your worth to your board?"

"They don't think I'm doing a good job."

"How do you know that they think that?" I asked. "Have they told you so?"

"No, of course not. But they are asking me for ways to measure my performance."

"Is evaluating your performance part of their mandate?"

She said, "Well, yes. I guess so."

"OK, then. Let's a.s.sume that they are only asking for input from you so that they can carry out their responsibilities. Perhaps a good question to start with is 'How well is the executive director performing her duties?' "

"Yes, that sounds OK."

"So, what are your job responsibilities, exactly?"

From this starting point, I was able to determine what the position of executive director meant. We could have rolled it all up into a pretty purpose statement like, "To ensure the care center's day-to-day operations are executed in a timely, efficient, and effective manner." This would be plaque worthy, but not useful for defining the metric in answer to the question of how well the director was performing. No, I needed a breakdown of the tasks and responsibilities that made up the executive director's position.

With a direct question like "what are your job responsibilities?" you can identify the proper measures to provide a meaningful answer. As with any requirements-gathering exercise, you have to get the real need, the root need.

When I've worked with clients who need to answer questions from above (in their organizational chain of command, not heaven), many times we don't get to a "good" root question-we're just not afforded the luxury due to the time involved or the ways in which we're asked to produce data. Even in that case, I cannot overstress the importance of trying. Push the envelope as far as you feel comfortable. Ask for the "why." You will do a much better job of designing the metric, collecting, and a.n.a.lyzing the data if you understand "why."

Figure 3-2 shows the three major things to capture during the collection of data, measures and information.

Figure 3-2. The "How" and "Who"

How Information Will Be Used.

Along with the root question and purpose, you should articulate clearly how the metric will be used. This part of the metric development plan provides a key tool in helping overcome the fear people have of providing data. It will also help with the fear, uncertainty, and doubt people have toward the way the data will be used. Again, if you have a well-formed, clear, and foundational root question and purpose statement, this should be easy. While it may be easy to define how the metric will be used, that doesn't make the definition obvious. You need to ask the question directly: "How will you use the metric?" Your goal is to try and get the most direct answer. The more direct the question, the higher likelihood that you'll get a direct answer.

Vague answer: "To improve our processes."

Direct answer: "To measure how the changes we implemented affect the process."

More specific answer: "To measure how the changes we implemented affect the process and allow for course corrections."

If public speaking is one of the greatest fears, the use of personal data might be a close second. Not just any numbers and values-but data that can be used to hurt an individual or create negative public perception of our organization. We imagine the worst, it's in our nature. So when we are asked to provide data, especially data that we believe reflects in any way upon ourselves or our departments, tremendous fear is created.

When we don't know the reason, we imagine the worst possible scenarios. It's our nature.

The greatest basis for fear is found in data that people believe addresses them at a personal level. This is why I recommend starting metrics programs at the effectiveness level.

When the data you are collecting (and later a.n.a.lyzing and reporting on) can be indicators about an individual, the fear factor becomes exponentially greater. It doesn't even matter if you plan to use the data at an aggregate level, never looking at the individual. If the data could be used at the individual level, the fear is warranted.

Time to Resolve can be a good effectiveness measure, used to improve overall customer service. The purpose may be simply to achieve better customer service and, therefore, satisfaction. But if you fail to communicate this purpose, the root question, and how you will use the answers-the individuals providing the data will imagine the worst.

Be forewarned. Even if you have an automated system to collect the data (for example, the day and time the incident was opened and closed), the ones opening and closing the case in that system are still providing the data.

If the staff learns that you are gathering data on resolution speeds, they will "hear" that you are collecting data on how long it takes them to resolve the case. Not how long it takes the team or the organization to resolve most cases, but instead how long it takes each of them individually to do the work. And, if you are collecting data on an individual's performance, the individual in question will imagine all of the worst possible scenarios for how you will use that data.