标签: programmers

The U.S. Declaration of Independence says "All are created equal." That's very true, but we don't all stay equal. Or, to paraphrase Animal Farm's pig Napoleon, all people are equal, but some people are more equal than others.   I'm not talking about politics here, though that might be an interesting topic for another day. I'm talking about variances in people's abilities to produce a product. More specifically, a high-quality software product.   Over the years, I've known a lot of software folk – some good, some bad, some mediocre. I've known a few who were scarily good, almost to the point of inspired, Twilight Zone class performance. I've seen a few who might have been better as cab drivers.   No, wait. That's an insult to cab drivers.   Over the years, I've seen a lot of software – some good, some bad, some ugly. Most of it mediocre at best.   No, wait. According to Defense Department studies of the 1970's, most of the software was never delivered, or was delivered but never worked. It's most of the delivered, working software that was mediocre.   The question at hand today is: Is there a correlation between people and their products? Does quality software come from quality people, or can you get quality software from mediocre people? There are experts today who assert confidently that you can, if only we put enough procedures and methods in place.   I'm not so sure.. They've been saying that for a good 40 years, but I don't see much evidence of it.   What is quality software? Before we talk further about the relationship between software and the people who build it, we'd better define what we mean by quality software.   Many years ago, JD Hildebrand, then managing editor of Computer Language magazine, invited me to give a paper at the magazine's conference. The paper was supposed to address software quality and how to get it.   It was a timely topic; the term was being widely bandied about at the time. Like user-friendliness, modularity, or flexibility, everyone claimed their software had it, but few bothered to define it or back up their claims.   It was familiar ground for me. A little earlier, my colleague Joe Philipose and I had done a similar paper on the term "User-Friendly Software." ( "Toward a Friendly Environment," Proc. Second Annual Phoenix Conference, IEEE Computer Society Press, Silver Spring, MD, March 14-16, 1983, pp.527-533 )   We noted that the adjective, "friendly," was usually reserved for human interactions. So, we reasoned, user-friendly software should leave us with the same feeling we'd get from a friendly human being. Think of it as the ultimate Turing test.   I took a similar approach in defining software quality. I asked myself, what attributes do we see in other, more tangible products, that cause us to see them as having that elusive attribute known as quality? I looked at – among other things – cars, pianos, violins, cameras, and stereo equipment.   Think of the Rolls Royce, the Mercedes Benz, the Steinway piano, the Stradivarius. Nobody would argue that these products exude quality, though I do recall one member of the audience who was vocally offended that I didn't include her Camry.   Parenthetically, I should note that she had a valid point. The Rolls is a quality product, no question about it. It's also bloody expensive. Her Camry probably didn't give back much in terms of feeling luxurious, but it provided quality at a reasonable price. That's important.   There are those who claim, sometimes stridently, that quality software is software that meets its requirement specification. That's a demonstrably inadequate definition. The specification for a car might say, among other things, "transport a driver and three passengers over a distance of 300 miles." But a Ford Model A sedan can do that. A King Midget (Google it) could carry two.   It's not enough to require that a car be able to negotiate, say, a highway cloverleaf at the posted maximum safe speed. We should also expect that, if you exceed that speed. the wheels don't fly off. In fact, any definition of quality should include the behavior of a product when pushed well beyond its design limits. Squealing of tires is Ok. Turning turtle is not.   Longevity is an issue with consumer products. The Mercedes has it, and Mercedes from the 1930's often bring auction prices over $1,000,000. So, clearly, do the Steinway and the Strad.   In the end, I came up with a handful of attributes that, to me, defined a quality product. They included the ff: - Greatly exceed specified performance limits - Respond rationally when pushed well beyond those limits - Never, ever fail, even when given improper inputs - Last.   This is clearly not an exhaustive list, but you get the picture.   Continuing with my paper, the next question was, how does one build a quality product?   As I thought about all the analogies – cars, pianos, etc. – I realized that a lot had to do with the people doing the building, and the way they felt about it. When you look underneath a Steinway, you see these exquisite wooden joints, made up from the highest quality woods. I don't know this for a fact, but I strongly suspect that you would find the same kinds of exquisite joints, even in places that are completely invisible to the customer.   Why are those joints there, and built with such care? They're there, I think, because the fellow who crafted the joints wasn't just an assembly-line worker, working at union scale. He was a craftsman, an artisan. An artisan doesn't create a beautiful joint to impress the buyer, or even his boss. He creates it because it's the right thing to do. He takes pride in his work, and gets job satisfaction from doing it right.   How are you going to put THAT in a requirements spec?   The software crisis Unless you're over 50, you probably don't remember the Software Crisis that rocked the U.S. Defense Department (DoD) in the 70's. It was based on two trends.   First, someone looked into the trend of using software in DoD weapons systems, and was alarmed at how fast it was increasing. By 1980, they said, software acquisitions would be 90% of DoD's entire procurement budget.   Someone else looked into the quality of the software in existing DoD programs, and was appalled. More than 50% of software procured by the DoD was never delivered at all. Another large chunk was delivered, but didn't work. Yet another chunk worked, but was so difficult to use that it was abandoned.   In the end, only about 3% of DoD-procured software was actually delivered, worked, and was put to routine use.   The other side of the Software Crisis coin was the pool of available talent. With more software, you need more people to build it. And the universities weren't turning out nearly enough to fill the demand.   Worse yet, as I noted earlier, not all programmers are created equal. Statistical distributions being what they are, the chances are good that most of the new programmers would not be superstars. Most would be mediocre. The greater demand for programers would likely mean hiring more of the mediocre folk.   So here was the Software Crisis in a nutshell: DoD was going to need more and more software, with much higher quality than it had got so far. To produce this software, it was going to need more programmers – more than the universities could produce. And since they were going to have to drain the labor pool to get those people, the quality of the people was likely to go down, not up.   There seemed to be only one solution to the Software Crisis: you have to get quality software from mediocre people.   Is that even possible? Aye, that's the question.   The Methodology Many groups in academia and industry looked into this problem, and did studies aimed at improving both software productivity and quality. The solution, they declared, was the methodology. More specifically, THEIR methodology.   The term "methodology" is a $10 word for "method." It makes you sound more authoritative when you say it. The idea is simply to define a set of practices that can, if followed, give you a better chance of achieving that elusive thing called quality, even when the people are mediocre at best. As with the assembly lines, make the process of building software so regimented, so tightly controlled, that any idiot can follow it and be productive.   Many groups got research grants to develop and refine their particular methodologies. Some of the pilot projects showed really significant improvements in software productivity and reduced error rate – as much as a 10:1 improvement.   Unfortunately, these performances weren't always realized when used by others. It seems that the people participating in the pilot program – usually graduate students – were enthusiastic rooters for the methodology, so they tended to work harder, and be more creative, than the average mediocre programmer.   Even so, most organizations adopted one methodology or another, and used them in their software programs. We've been using such methodologies to this very day.   One software guru put it this way: Imagine you're starting at a new company, and your boss comes by and plops a 500-page document on your desk. He says, "This is our company methodology. Read it and study it; it's what we'll be using on this project."   Recalling that the purpose of a software methodology is to get quality software out of mediocre people, what is the manager's message? Isn't it that he thinks you're one of the mediocre ones?   Now, don't get me wrong. I am not arguing against methodologies. I am by no means advocating that we go back to the bad old days of ad hoc development, programmers as rugged individualists, and completely disorganized processes. I'm just asking the question: Can even the best, most carefully crafted and religiously followed methodology produce quality software from mediocre people?   I'm asserting that it can't. I'm further asserting that the more stringently we enforce the methodology, the more likely we are to get a mediocre product.   Is that Ok? Is "good enough" good enough? Maybe it is. But it's still mediocrity.   Quality Assurance Many organizations have independent departments responsible for Software Quality Assurance (SQA). I've worked with a few of them. I've chaired quite a few design reviews, and I've managed an SQA department. An independent SQA department is justified in order to relieve the software developers from conflicting interests.   Most software developers want their software to be better than mediocre, but they're also faced with tight schedules, loosely defined specs, and critical managers. In the end, they may be focused more on keeping the boss of their backs, than insuring quality. An independent QA group can be better divorced from production schedules.   I've had mixed results from the QA folk. In one company (Honeywell), the QA group was absolutely wonderful. They saved my buns more than once. For one thing, they were experts at reading and understanding the various specifications, and the things we had to do to meet the contractual requirements on schedule.   Even more important, they were good software engineers in their own right. They actually desk-checked every line of our software, and pointed out many errors that the team members had missed. The team members didn't always appreciate what they viewed as "snooping" into their software, but I sure appreciated it.   Not all SQA groups are so helpful. In all too many cases, the head QA guy is more bureaucrat than engineer, and often doesn't even know programming. He's more focused on checking off boxes in a questionnaire. Did you have the review on schedule? Did the following questions get asked? Have all the action items been completed? If he can say "yes" to all such questions, he can check the little boxes and declare the software to have quality.   What it really has, at best, is mediocrity. More about this in my next blog..    

It won't come as a surprise to any practitioner of the software art that we tend to be an ornery and opinionated sort. That's been true throughout the history of the discipline and was especially true in the early days. We early software folk tended to be "rugged individualists" with very strong, albeit wildly differing, notions about how software should be written. There were few if any established methodologies or ground rules at the time. Even Kernighan's and Plauger's seminal book, Elements of Programming Style , was still in the future.   Today, academic disciplines and studies, textbooks, formal methodologies, company training courses, programming guidelines, and peer attitudes may have dampened the wildest excursions of our individualism but not eliminated them.   Like all creative people, software folk often have ideas of their own as to how things should be done. But ideas come in all flavors: some are good, some are brilliant, and some are crushingly, embarrassingly bad. In the end, the trick is not to have only good ideas (that's not possible) and definitely not to blindly follow the ideas of some vaunted authority. The trick, rather, is to be able to discern the good ideas from the bad ones, reject the bad ones, and adopt the good ones as your own.   For reasons I don't fully understand and therefore can't explain, I've often found my own ideas to be out of step with those of my colleagues. These differences led to debates, ranging from polite discussion to out-and-out food fights. In time, I came to accept the battles as part of the profession and contented myself with the observation that "my side" often carried the day.   But I never anticipated that I'd be having to fight the same tedious battles, generation after generation, ad infinitum. Just when I think I've gotten out of the debates, they pull me back in. Just when I think a given issue has been settled for all time, along comes a new generation of programmers, sporting the same tired old idea. Some of the worst ideas seem to enjoy eternal life, enjoying rebirth and returning to plague me, like Count Dracula rising from his coffin.   Today, I'd like to tell you about two of the more persistent and pernicious of the Bad Ideas.   "It's Too Inefficient" Most programmers are perfectly willing to agree, in principle, that notions like modularity, encapsulation, information hiding, and software reuse are good ideas, leading to more reliable and maintainable software. Yet all too often, these ideas are honored mostly in the breach. Old Fortran programmers like me well remember the days when most Fortran programs were written in bowl-of-spaghetti (BOS) fashion, with little if any modularity. Although even the earliest compilers, and assemblers before them, supported callable subroutines with passed parameters, many programmers chose not to use them.   I got this one right. From the get-go, I was writing small subroutines with passed parameters. David Parnas had nothing on me. Don't get me wrong: I claim no 20/20 prescience here. I used the modular style for two good reasons: First, it's the way I was taught. The fellow who taught me Fortran didn't show me any other way. He had me writing small subroutines and functions for him to use in a large simulation, and he showed me how to write in the style that he wanted. Black boxes he wanted, black boxes he got, and my programming style was set forever.   When I looked at software created by others, I was pretty dismayed to find that the BOS model predominated. Their Fortran programs would go on for page after page, with nary a single CALL or RETURN, and therefore no parameter lists (for that matter, no comments, either). Just large numbers of GOTO's.   We were doing scientific programming at the time, simulating space trajectories, and therefore making heavy use of vector and matrix math. I used my subroutines, which weren't all that different from the ones you've seen in my C++ vector/math package.   Most of my colleagues used the same algorithms, but not the idea of modularity. Instead, they coded them in line. Where I might write:      Call Cross(a, b, c)   They'd write:      c(1) = a(2)*b(3)—a(3)*b(2);       c(2) = a(3)*b(1)—a(1)*b(3);       c(3) = a(1)*b(2)—a(2)*b(3);   At each place where they needed another cross product, they'd code the same three structures, only with different variables names. If they needed multiple cross products, they'd code the same three lines again, carefully duplicating the index patterns.   Which brings me to the second reason I liked modularity: It kept me from making stupid errors. Each time you copy that three-line algorithm for the cross product, you risk getting either one of the indices, or one of the argument names, wrong. The error can often be hard to track down (did you spot the error in the lines above?). Using a black-box subroutine, I could be pretty certain that if it worked right on the first cross product, it would also work right for the next one.   More than once, I'd mention to a colleague, "You know, you could just call a subroutine for that. I have one you can use, if you like."   The answer was always the same:   "No, I couldn't do that. It would be too inefficient."   Edsger Dijkstra pointed out that code efficiency depends far more on using the right algorithm than on tricky code. But if it's speed you want, and you don't care if the answer is right, I can give you speed out the gazoo. For me, at least, correctness and ease of programming trumps raw performance any day.   Once, a programmer was assigned to adapt one of my simulations to a new computer. After working on it for awhile, he came to me almost livid in anger. My coding, he asserted, was far too inefficient. I had subroutine calls nested four or five layers deep, and he thought this was unconscionable. "Every time you call a subroutine," he warned, "you waste 180µs."   I looked at my watch for a time, then said, "I can wait."   He was not amused.   Ironically enough, he later wrote an attitude simulation of his own. He used the "efficient," BOS model. But he made a mistake, inverting a matrix in the highest-speed loop, even though it was a constant. For this and other reasons, my "inefficient" program was more than twice as fast. Now, one could argue that the two issues aren't connected—that the math mistake didn't have anything to with his coding style. I tend to suspect, though, that if he'd stuck to a modular style, his coding and testing tasks have been easier, perhaps easy enough so he'd have spotted the mistake.   Over time, people learned that modularity was good, spaghetti bad. And as computer mainframes got blazing speed, execution time was not such an issue. But the efficiency notion came back to life in spades, with the advent of microprocessors. With their (initially) slow clock speeds and software floating point, efficiency was again an issue. Even (or especially) when using assembly language, programmers tended to the BOS model. So did (and do) many programmers of embedded systems.   The notion persists to this day. Only a year ago a colleague, whose intelligence I value, showed me a Matlab program he had written. It used many long modules and had many of the "in-line cross product" sorts of constructs. I ran a McCabe metric on it, and got a cyclomatic number around 27 (something like 3-5 is better). I gently asked my friend why he hadn't used a more modular style. He replied,   "I thought about it, but that would have been too inefficient."   Now, you have to take the time to get your arms around the situation. Matlab is an interpreted language. It's not going to use the computer efficiently, no matter what you do. We don't write programs in Matlab when we want speed; we use it when we want the convenience, ease-of-use, and GUI environment of the interpreted language. Granted, Matlab programs can be compiled into faster executables for production, but my friend wasn't using that feature. So efficiency should have been the last thing on his list of needs.