Find out what improvement really looks like

Data – it’s not what you think it is – part 3 – When an improvement really isn’t an improvement

You’ve been working hard at improving part of your business. You’ve implemented a new system, trained your staff, or spent money on a new consulting firm. You are excited to see the improvement, and you are looking forward to seeing the data right away.

So what do you see? What are you expecting to see?

Sure – everyone knows that if you are looking at profits – up is better than down. More customers are better than fewer. And a downward trend in costs is more favorable than an upward one.

But if all you are doing is looking and reacting to a change in direction, you might be putting your company at risk.

Don’t react until you know

Things to know:

1) If you are looking at your data for the first time, then go back to two of my blogs that will help set the stage: (Part 1 – Why we need to measure, Part 2 – the two mistakes with averages).

2) If you have people involved in your process, and most of us do, then there will be random variation in your data. Simply put, your data will go up or down simply because people are not machines.

Think of it this way: if you were measuring your own personal happiness, today will be better or worse than yesterday. Even if you locked yourself in your house and had no interaction with others for two days, our moods, actions, behaviors, and emotions change on a regular basis. Therefore, your productivity/output/data will be slightly different every day.

In any human system, you should expect your data to go up or down each time that you measure. This doesn’t mean that things are getting better or worse, necessarily.

The important thing to tell yourself is NOT to react just because things are up or down. Take a deep breath and move on to….

3) Know what true change really looks like.

Now that you’re not reacting to random changes in a human system, what does change really look like? To know that, you need to know what your process typically looks like.

For now, put away your metric that measures the average and forget your trend lines. As discussed before, they can be deceiving. Instead, make a simple line graph which shows your metric over time.

Most likely, you’ll see a graph that has a series of ups and downs. Again – that’s normal. What you want to look for is the area of the graph that comprises roughly 95% of all of the data points. Then draw a horizontal line at the top of that 95% and one at the bottom. (There are easy mathematical ways to figure where this exactly is on your graph, but you can even eye-ball it.) Everything in between those lines are where your process lives today.

The big finish

Now – here’s the fun part. If you are CHANGING your process, you expect the result to improve, right? How do you know if things are improving? It’s not because that one measure is higher than the other. It’s not because you have 2, 3, or 4 data points going up in sequence (randomness like that happens every day). It’s not because things FEEL different.

REAL change shows itself when the data shows that a different result is coming from your hard work. In our case, when the data starts to move outside of your 95% area on a more consistent basis. Not just once or twice (remember – 5% of the time with your old process, things went outside of this as well), but a significant number of times.

And what if your data points stay within the 95% area? The answer is easy: either the new process isn’t being followed by your staff (for help, first start here: Every business owner must know THIS one thing to be successful), or the change you just implemented doesn’t work. It’s that simple!

When you change your process… your data will change too! But you have to know what to look for. Businesses around the world are fooled into thinking things have improved when they really haven’t.

With a little discipline and practice, you’ll never be fooled again.

For help with these issues and to get other tips, visit While you are there, check out and subscribe to our blog for FREE weekly business tips.

This is the third of a 4 part blog series on data, its problems, and how to use it to get more value out of your business. It will be followed up with our very first podcast – Process Pitches – where we will be inviting Six Sigma/Lean/data experts and business leaders to share some of their stories from the business world about the struggles leaders have understanding data to its fullest.

Start Now