Painting the Wall

Tom Van Vleck

When I was a QA manager, there was a person I didn't hire. I interviewed him on the phone; he told me he'd been doing QA on a new microcomputer's operating system.

"How did that go?" I asked. "Oh, I found lots of bugs. Tons of them," he said. "What percentage of the system did you test?" I asked. "As much as we had time for." I tried again. "Of the lines of code in your system, how many were exercised by your tests?" "Well... we didn't think of it that way." "Did you have a test plan?" "No, there was no time for that."

Here was a fellow who'd been told to paint a wall. What he'd done was to stick his brush in the paint and start painting right away. Some of the wall right in front of him had five or six coats of paint, and other patches right nearby had no paint at all. He had no idea how big the wall was, or whether he had enough paint, or what needed painting the most. Because he wasn't working systematically, he couldn't say how much he had done or how much longer it would take to finish.

Suppose two people do the same job. One does it in a systematic way, according to a plan, with visible progress against the plan. The other appears to make no progress at all until the last moment, and then comes up with a finished result. Even if they take the same time and produce the same quality result, which one would you rather work with?

Always know what you're trying to optimize.

If we measure something, we can optimize it. There is a risk here: if the only thing we can measure is the wrong thing, we'll end up optimizing the wrong thing. But if we don't know what we want to optimize, our time will almost surely be wasted.