Nasty Question

Tom Van Vleck

I have a favorite question to ask of people trying to sell me software. I let them go on & on about features, functions, benefits. Then I smile and say, "What do you do to make your product high quality?" I've gotten some interesting answers. "Oh, the programmers are very sincere. They work really hard." or "After we write the code we run a lot of tests we found somewhere." or my favorite one, "Well, you see, I am Swiss, and I am very precise."

When I get an answer like this I smile more, and say, "Yes, but what exactly do you do to make your product high quality?" Some people still don't get it. I asked one prospective supplier my question, and they answered with a lot of charts about the bug rate in their shipped products. They were very proud that as the product matured, the bug rate went down. Another prospective supplier had a better answer: they talked about their systematic testing, generated from an assertion language and validated by a coverage tool. But these are both about how they recover from quality problems, not about how they put quality in.

If anybody asks me this nasty question, I want to be ready with a good solid answer. Ideally, everybody in my team should know the answer, believe in it, contribute to it, and be able to evangelize it. Our answer should be true, and way beyond what the competition can claim. It should be obvious to the user that our quality is phenomenal. We should do four things:

Commit to zero defects
Proceed systematically
Check everything
Improve continuously

Zero defects: a friend was telling me about the landscaping he was having done. The landscapers had made some mistakes, and he said, "I paid for a perfect job. They'll have to fix it." The customer expects a perfect job from us. Even in teams that already have wonderful commitment to quality, some folks are scared of promising zero defects... but that's what we expect from others. And any tiny defect can blow the reputation of the whole system, in the user's eyes. (This is the cockroach theory: if you see a roach in a restaurant, you don't say "There's the roach in this place." You say, "Let's get out of here, this place is infested.")

Proceed systematically: Architecture, procedures, well-reviewed designs, high-level languages and discipline for using them, plans for thorough testing, and measurement of results are all evidence for a development process that's under control. In addition, the team should have documented methods for design, defect prevention, and inline testing, and tools to mechanize these.

Check everything: Every work product should be checked somehow. Most teams review designs, review source code, and test compiled code. The principle can be applied to other things we do for a "client" anywhere that quality might suffer. If this principle becomes part of our culture, each step can concentrate on preventing or removing its own defects, counting on clean input from the prior step.

Improve continuously: If we do all the above, it won't be enough. We need to learn from our mistakes, look for better ways, question authority. Our methods and processes have to keep evolving to meet changing conditions.

I think a story like this will not only work as good advertising, it will also reduce development time, cost and risk.