原创 Using design metrics

George Dinwiddie. a good friend, maintains a blog that focuses on software development. One of his postings particularly got my attention.

In that blog post, George discusses the balance between people and process. He faults organizations that find the most productive developer, and then tries to clone him across the team by duplicating whatever processes he uses.

I agree that even though this approach is seductive to some managers it's doomed to failure. But there's a more fundamental problem: without productivity metrics it's impossible to know if the team is getting better or worse.

In my travels I find few firmware groups that measure anything related to performance. Or quality. There are lots of vague statements about "improving productivity/quality," and it's not unusual to hear a dictate from on-high to build a "best-in-class" development team. But those statements are meaningless without metrics that establish both where the group is now, and the desired outcome.

Some teams will make a change, perhaps adopting a new agile approach, and then loudly broadcast their successes. But without numbers I'm skeptical. Is the result real? Measurable? Is it a momentary Hawthorne blip?

Developers will sometimes report a strong feeling that "things are better." That's swell. But it's not engineering. Engineering is the use of science, technology, skilled people, process, a body of knowledge – and measurements – to solve a problem.

Engineers use metrics to gauge a parameter. The meter reads 2.5k ohms when nominal is 2k. That signal's rise time is twice what we designed for. The ISR runs in 83µsec, yet the interrupt comes every 70. We promised to deliver the product for $23 in quantities of 100k, and so have to engineer two bucks out of the cost of goods.

Some groups get it right. I was accosted at the ESC by a VP who experienced a 40% schedule improvement by the use of code inspections and standards. He had baseline data to compare against. That's real engineering. When a developer or team lead reports a "sense" or a "feeling" that things are better, that's not an engineering assessment. It's religion.

Metrics are tricky. People are smart and will do what is needed to maximize their return. I remember instituting productivity goals in an electronics manufacturing facility. The workers started cranking out more gear, but quality slumped. Adding a metric for that caused inventory to skyrocket as the techs just tossed broken boards in a pile, rather than repair them.

Metrics are even more difficult in software engineering. Like an impressionistic painting they yield a fuzzy view.

But data with error bands is far superior to no data at all.