标签: programmer

I read a great deal of code. The vast majority is in C with some C++ and a bit of assembly sprinkled in. Some of it is brilliantly-written, clearly by developers who are craftspeople of the highest order. Some are, well, not so nice. I sometimes wonder what the developers were thinking.   One pet peeve is with the comment header at the top of each module. A great variety of styles are used, and some are better than others.   The worst comment headers are the ones that aren’t there, or the ones that are completely wrong.   On one safety-critical project every module did have a header. But they were obviously block-copied. There’s nothing wrong with that, except in this case dozens of files had identical headers. A module containing functions for the ADC, for instance, had a header documenting (well, with a mere line or two) some entirely different chunk of code. A huge percentage of these headers had identical dates and authors. Either that person was a code-cranker of monumental skill and speed, or he was completely careless. Incorrect comments are worse than none at all.   Oddly, each header was many tens of lines long, but most of these were blank lines. Probably the intent was to fill in the blank spaces with appropriate comments, but no one ever bothered.   I can’t stand many headers used in open-source projects. Here’s an example from Linux:   The first real line describes the module, which is exactly the right way to start a header. That’s what someone wants to know when opening the file. A copyright notice and the author’s name is appropriate. But the rest is lawyerly crap. No one reads this stuff, and you’re left paging through the words to find where the real documentation starts. Though it may be important to have this for legal reasons, it gets in the way. It should be at the very end of the file where it will be read just as carefully as when it’s annoyingly at the beginning.   In too many cases the small print goes on for far longer than this one. Do we really need to stumble over it all? In every single module? Is it fair to make perhaps hundreds of others waste their time with it?   Also, note that the disclaimer about the warranty is itself embedded in the GPL 2. Adding it to the header is redundant. And the GPL 2 is about 3000 words; either this header is woefully incomplete or should be replaced by a one-sentence reference to the license.   Here’s another extract from Linux. There’s a brief reference to the license. I like this as it provides legal cover while not cluttering the file. But it doesn’t reference which version of the GPL applies (GPL 2 was released in 1991, long before the date in the file). Now as to content. In the previous example a line does mention what’s in the file. But what’s a governor? Some sort of general description is needed as others, many perhaps far less familiar with the code than the authors, will be digging through this. Remember that many people want a very fast high-level view of what is in the module – just a sense of how it fits into the big picture. Perhaps something like “The governor uses a pair of weighted sheres to ensure Linux’s triple-expansion steam engine maintains a constant speed.”   Freescale’s MQX headers all start with this line: /**HEADER******************************************************************** And end with: *END*********************************************************************/   Do those comments add useful information? The asterisks might be nice to block off the comments from the rest of the code, but HEADER and END are an affectation.   FreeRTOS is very well commented and is a joy to read. But each module starts with this:     This is a sales pitch. It’s not documentation. In fact, there’s not a word about what the module does. But, like Marco Rubio’s 25-second verbal tic, did you notice that 1 tab == 4 spaces?   I have heard Ada programmers wonder if C users have an aversion to typing. While that’s an unfair generalization there is a common pattern of excessive briefness, to the point of taciturnity. It’s important to be expressive, to get all of the important information on the page. Doxygen provides pretty good tools for this: the header starts with a brief, one-line description, but that is fleshed out thereafter. The one-liner is the headline; the rest is the story.   Documentation can be carried to an extreme. One module I read had a header 2000 lines long - the code was merely 200. Sometimes it makes sense to point to external documentation instead.   Every header should include: - A brief description of the file - A detailed description of the file - Author’s name - The date of first release - The developer’s name, date and a description of each revision - Perhaps a line or two about licensing   I prefer using the /* comment rather than double slashes. This: /* Comments Comments Comments */ is better than: // Comments // Comments // Comments … because the former eases making free-form changes to the comments. Having to reformat double slashes when just adding a few words to a sentence makes one less inclined to maintain the comments.   A project I looked at recently had no comment headers at all. None. Modules were expressively named “b.c” and functions were little better. The entire program was written by one individual, and presumably the thinking was that, since he would maintain it, comments were superfluous. As always happens 1 , he moved on to another company and the poor sods left had to deal with the mess. I think a role of management is demanding high quality work from their people, and doing enough of an audit to ensure that the quality is there. It can be hard to assess the quality of software, but it’s pretty easy for someone with basic software skills to look at source and notice a dearth of comments or inconsistent styles.   I’ve been told by smart developers that they don’t want to take the time to write descriptive, careful comments. But if you can’t type pretty quickly in this day then programming is probably not for you. And if you can’t carefully formulate your intention in English, odds are you can’t in code.   The comments are a love letter to yourself and your successors. They are every bit as important as getting that interrupt handler right.   Note 1: Always. The only valid assumption when documenting the code is that someone else, someone who knows far less than you do about the problem, will be left trying to make sense of your work. It’s up to you whether he thinks “Joe was a master craftsman!” or “that **%()($# Joe! I hope he rots in hell with the Perl programmers!”

It has been said that humanity can be divided into two basic personality types --  the Apollonian, whose existential statement is “I think, therefore I am” and the Dionysian, who says “I feel, therefore I am.” This, of course, is simplistic – there are many other types. One that I identify with is the Hephaestian, whose statement is: “I make, therefore I am.”   Hephaestians are the makers, those who ask, “How can you solve a problem if you don't have one?” We take delight in the process of conceptualization continually tied to realization. We are continuously drawn to the realization of physical structures, or of non-physical structures that have a life in the physical universe more than in the abstract intellect.   We do not claim ancestry of the natural philosophers and scientists, who for millennia could not answer the question, “what will come of this?” Our lineage is from the engineers who, always working with imperfect tools and inadequate mathematics, managed to build the infrastructure to make possible the civilization of the moment.       It has only been in the last century or so that two groups have come together, conflated in the popular mind. As part of growing up in this cultural phenomenon many Hephaestians have followed the lead of Apollonians and disparaged sport, which they see as concerned only with the life of the body, unlike our presumed cultivation of the life of the mind. In my elite high school, populated with high-functioning nerds, the athletes held a low social rank.   I maintained this viewpoint until only a few decades ago, when I came to the realization that I had been engaged in sport all along! The parallels were complete – both the jocks and I lived for the exercise and expansion of personal or team capability and the experience and display of virtuosity. Reward was almost completely internal – one could accept the plaudits of others but this could never overshadow the knowledge of obstacles not yet overcome.   Last year I lectured in Russia and heard a presentation by a Russian sociologist who had studied the culture of the Hephestians in the Soviet Union where entire “academic cities” were built to nurture them. These young elite programmers and engineers, as it turned out, were enthusiastic physical sportsmen, confirming the thesis that I was explaining in my lectures – that the phenomenon of hacking was in fact another manifestation of sport.   Why does one go through years of development, consisting mostly of failure, pursuing the elusive goal of performance beyond one's personal best, when it would mostly be misunderstood by those who had never been there, and when the reward would be only the opportunity to try again to do better? Material reward is tied to such success only loosely -- both types of athletes must adapt their economic life to this pursuit of virtuosity.   We do it because we love to, and we do not yet have an answer to the question of the source of this love. Everywhere on Earth people pursue this same quest, one that will never end, and find fulfillment in the process. This is a mystery.   Go to the Maker Faire and you will see not only the high-tech gizmos that young technologists are developing. Look further into the corners of the exhibition halls and you will find table after table of crafters working in fabric, string, plastic, cardboard and wood. They have always been there, a majority being women. Walk among them, handle their products, and talk with them – and see the world being continuously reborn.   Our mystery is confronted every day in this way not in cloisters but on kitchen tables and basement workbenches by people everywhere continuing to rise to the challenges they pose themselves. For this reason I cannot despair of humanity regardless of the negative evidence. It cannot be stopped.   Lee Felsenstein is an electrical engineer who lives in Palo Alto, Calif.

MISRA C is most likely the most popular of all firmware standards. Originally targeting the automotive market, its audience now includes pretty much every industry that builds embedded systems. The last version appeared in 2004, but a few weeks ago MISRA 2012 hit the street. Among other improvements it now handles C99 (as well as C90). The new version is huge, at 236 pages about double the size of the previous standard. A few more rules were added, and the wording of many of the others changed. MISRA 2012 has better explanations of the rationale behind each rule. Now there are rules as well as directives. The latter are guidelines which may require access to other resources to prove compliance. An example is Directive 3.1: "All code shall be traceable to documented requirements." Rules can be tested just by examining the code. The CERT standard took a strong stand against code that can have unwanted side effects (like careless use of the increment and decrement operators). MISRA 2004 had some rules about these issues, but the new version is much more explicit. But some of the rules and directives leave me scratching my head. Directive 4.1 says "Run-time failures shall be minimized." What does that mean? How can one measure it? What does "minimize" mean, quantitatively? This rule almost leaves one thinking that it's an advisory against the use of C, which places the burden of taking care of run-time problems entirely in the programmer's lap. The guidelines suggest adding code to deal with problems and using static analyzers, both great ideas. But I can't see how one can claim conformance with the Directive without making some sort of formal argument rather like a safety case. And that's quite an expensive proposition. 5.2.1 goes further, and says, among other things, that developers have to demonstrate that run-time errors have been avoided. "Run-time error" is not defined, although examples (like arithmetic overflow) are given. Is code that doesn't implement a requirement properly a run-time error? Absent a proof-checking language like SPARK, this seems more a wish than a command. 5.5 is probably wise but will be problematic for non-greenfield projects that require adherence to MISRA. To claim compliance one must assure, among other things, that all of the C code in the project meets MISRA 2012. Using Linux? Forget putting a MISRA badge on the project. Is there legacy code (and most projects use lots)? Well, I'd advise using MISRA for the new code, but you won't get compliance points. I'm a strong advocate of MISRA. No one (well, with the possible exception of those who crafted the standard) likes all of the rules, but most of them make a lot of sense. MISRA is one way to get a firmware standard in place fast, one that has plenty of street cred. If you don't have regulatory compliance issues, overcome objections to the few rules you don't like by subsetting it. Happily a lot of automatic compliance checkers exist, and LDRA has already announced support for the 2012 version. And, their tool lets you pick and chose which rules to check. There's more info here: http://misra.org.uk/ .  

Let me first clarify, this book is NOT about hacking in the modern sense of finding out weaknesses in a computer system and exploiting them. In the early days of computers, a hacker was someone who delighted in subtle programming tricks and small algorithms that could be used to make their computer code "tighter" and more efficient. Hacker's Delight really is a great book. Just to give you a feel for it, it's the first book of this type that I've ever seen on Amazon in which every reviewer (and there are a bunch of them) has awarded five stars. On the one hand, the tricks in this book are becoming less relevant as computational resources – in the form of memory and performance – increase. Many of today's programmers work at a high level of abstraction and are conceptually a long way from the "machine". Also, in many cases, a slight inefficiency in the code is of less importance than getting the application out of the door in a timely manner. On the other hand, if you are working on embedded systems with limited performance and memory resources, and if you are working "close to the metal", then the tips and tricks in this book are for you. Most of the time the author assumes a 32bit machine and describes his tips and tricks in a generic computer algebra; more complex tricks are presented in C; but all of these techniques could be easily adapted to machines of different word lengths (8bits, 16bits, 64bits, etc.) and/or implemented in assembly language if required. As a simple tempting taster, let's consider the very first page of Chapter 2: Basics (Chapter 1 is the introduction where the author defines things like the notations to be used and the syntax of his computer algebra). Note that in the following discussions: – x      = Arithmetic negation ~x      = Bitwise NOT (ones complement) x y = Bitwise AND x | y    = Bitwise OR x ^ y   = Bitwise Exclusive OR OK, the first part of the first page reads as follows: Manipulating Rightmost Bits Use the following formula to turn off the rightmost 1bit in a word, producing 0 if none (e.g., 01011000 = 01010000): x (x – 1) This may be used to determine if an unsigned integer is a power of 2 or is a 0; apply the formula followed by a 0-test on the result. Similarly, the following formula can be used to test if an unsigned integer is of the form 2 n – 1 (including all 0 or all 1's): x (x + 1) Use the following formula to isolate the rightmost 1bit, producing 0 if none (e.g., 01011000 = 00001000): x (–x) Use the following formula to isolate the rightmost 0bit, producing 0 if none (e.g., 10100111 = 00001000): ~x (x + 1) Use one of the following formulas to form a mask that identifies the trailing 0's, producing all 1's if x = 0 (e.g., 01011000 = 00000111): ~x (x – 1) , or ~(x | –x) , or (x –x) – 1 The first formula has some instruction-level parallelism. I haven't finished yet. Remember that we are still on the very first page of the main body of the book. At the bottom of the page is one more formula to form a mask that identifies the rightmost 1bit and the trailing 0's, producing all 1's if x = 0 (e.g., 01011000 = 00001111). This formula is as simple as its companions shown above ... I'll leave it as a little teaser for you to play with. As one reviewer on Amazon said "If you approach optimisation as a puzzle, wherein the solution is its own reward, this book is indeed a compendium of delights." I totally agree, and I highly recommend this book to anyone who is interested in knowing the "tricks of the trade." This is of especially interest to embedded software developers working with limited resources, but it is also of interest to any software developer who takes satisfaction in making his or her code as "tight" and efficient as possible.  

The U.S. Declaration of Independence says "All are created equal." That's very true, but we don't all stay equal. Or, to paraphrase Animal Farm's pig Napoleon, all people are equal, but some people are more equal than others.   I'm not talking about politics here, though that might be an interesting topic for another day. I'm talking about variances in people's abilities to produce a product. More specifically, a high-quality software product.   Over the years, I've known a lot of software folk – some good, some bad, some mediocre. I've known a few who were scarily good, almost to the point of inspired, Twilight Zone class performance. I've seen a few who might have been better as cab drivers.   No, wait. That's an insult to cab drivers.   Over the years, I've seen a lot of software – some good, some bad, some ugly. Most of it mediocre at best.   No, wait. According to Defense Department studies of the 1970's, most of the software was never delivered, or was delivered but never worked. It's most of the delivered, working software that was mediocre.   The question at hand today is: Is there a correlation between people and their products? Does quality software come from quality people, or can you get quality software from mediocre people? There are experts today who assert confidently that you can, if only we put enough procedures and methods in place.   I'm not so sure.. They've been saying that for a good 40 years, but I don't see much evidence of it.   What is quality software? Before we talk further about the relationship between software and the people who build it, we'd better define what we mean by quality software.   Many years ago, JD Hildebrand, then managing editor of Computer Language magazine, invited me to give a paper at the magazine's conference. The paper was supposed to address software quality and how to get it.   It was a timely topic; the term was being widely bandied about at the time. Like user-friendliness, modularity, or flexibility, everyone claimed their software had it, but few bothered to define it or back up their claims.   It was familiar ground for me. A little earlier, my colleague Joe Philipose and I had done a similar paper on the term "User-Friendly Software." ( "Toward a Friendly Environment," Proc. Second Annual Phoenix Conference, IEEE Computer Society Press, Silver Spring, MD, March 14-16, 1983, pp.527-533 )   We noted that the adjective, "friendly," was usually reserved for human interactions. So, we reasoned, user-friendly software should leave us with the same feeling we'd get from a friendly human being. Think of it as the ultimate Turing test.   I took a similar approach in defining software quality. I asked myself, what attributes do we see in other, more tangible products, that cause us to see them as having that elusive attribute known as quality? I looked at – among other things – cars, pianos, violins, cameras, and stereo equipment.   Think of the Rolls Royce, the Mercedes Benz, the Steinway piano, the Stradivarius. Nobody would argue that these products exude quality, though I do recall one member of the audience who was vocally offended that I didn't include her Camry.   Parenthetically, I should note that she had a valid point. The Rolls is a quality product, no question about it. It's also bloody expensive. Her Camry probably didn't give back much in terms of feeling luxurious, but it provided quality at a reasonable price. That's important.   There are those who claim, sometimes stridently, that quality software is software that meets its requirement specification. That's a demonstrably inadequate definition. The specification for a car might say, among other things, "transport a driver and three passengers over a distance of 300 miles." But a Ford Model A sedan can do that. A King Midget (Google it) could carry two.   It's not enough to require that a car be able to negotiate, say, a highway cloverleaf at the posted maximum safe speed. We should also expect that, if you exceed that speed. the wheels don't fly off. In fact, any definition of quality should include the behavior of a product when pushed well beyond its design limits. Squealing of tires is Ok. Turning turtle is not.   Longevity is an issue with consumer products. The Mercedes has it, and Mercedes from the 1930's often bring auction prices over $1,000,000. So, clearly, do the Steinway and the Strad.   In the end, I came up with a handful of attributes that, to me, defined a quality product. They included the ff: - Greatly exceed specified performance limits - Respond rationally when pushed well beyond those limits - Never, ever fail, even when given improper inputs - Last.   This is clearly not an exhaustive list, but you get the picture.   Continuing with my paper, the next question was, how does one build a quality product?   As I thought about all the analogies – cars, pianos, etc. – I realized that a lot had to do with the people doing the building, and the way they felt about it. When you look underneath a Steinway, you see these exquisite wooden joints, made up from the highest quality woods. I don't know this for a fact, but I strongly suspect that you would find the same kinds of exquisite joints, even in places that are completely invisible to the customer.   Why are those joints there, and built with such care? They're there, I think, because the fellow who crafted the joints wasn't just an assembly-line worker, working at union scale. He was a craftsman, an artisan. An artisan doesn't create a beautiful joint to impress the buyer, or even his boss. He creates it because it's the right thing to do. He takes pride in his work, and gets job satisfaction from doing it right.   How are you going to put THAT in a requirements spec?   The software crisis Unless you're over 50, you probably don't remember the Software Crisis that rocked the U.S. Defense Department (DoD) in the 70's. It was based on two trends.   First, someone looked into the trend of using software in DoD weapons systems, and was alarmed at how fast it was increasing. By 1980, they said, software acquisitions would be 90% of DoD's entire procurement budget.   Someone else looked into the quality of the software in existing DoD programs, and was appalled. More than 50% of software procured by the DoD was never delivered at all. Another large chunk was delivered, but didn't work. Yet another chunk worked, but was so difficult to use that it was abandoned.   In the end, only about 3% of DoD-procured software was actually delivered, worked, and was put to routine use.   The other side of the Software Crisis coin was the pool of available talent. With more software, you need more people to build it. And the universities weren't turning out nearly enough to fill the demand.   Worse yet, as I noted earlier, not all programmers are created equal. Statistical distributions being what they are, the chances are good that most of the new programmers would not be superstars. Most would be mediocre. The greater demand for programers would likely mean hiring more of the mediocre folk.   So here was the Software Crisis in a nutshell: DoD was going to need more and more software, with much higher quality than it had got so far. To produce this software, it was going to need more programmers – more than the universities could produce. And since they were going to have to drain the labor pool to get those people, the quality of the people was likely to go down, not up.   There seemed to be only one solution to the Software Crisis: you have to get quality software from mediocre people.   Is that even possible? Aye, that's the question.   The Methodology Many groups in academia and industry looked into this problem, and did studies aimed at improving both software productivity and quality. The solution, they declared, was the methodology. More specifically, THEIR methodology.   The term "methodology" is a $10 word for "method." It makes you sound more authoritative when you say it. The idea is simply to define a set of practices that can, if followed, give you a better chance of achieving that elusive thing called quality, even when the people are mediocre at best. As with the assembly lines, make the process of building software so regimented, so tightly controlled, that any idiot can follow it and be productive.   Many groups got research grants to develop and refine their particular methodologies. Some of the pilot projects showed really significant improvements in software productivity and reduced error rate – as much as a 10:1 improvement.   Unfortunately, these performances weren't always realized when used by others. It seems that the people participating in the pilot program – usually graduate students – were enthusiastic rooters for the methodology, so they tended to work harder, and be more creative, than the average mediocre programmer.   Even so, most organizations adopted one methodology or another, and used them in their software programs. We've been using such methodologies to this very day.   One software guru put it this way: Imagine you're starting at a new company, and your boss comes by and plops a 500-page document on your desk. He says, "This is our company methodology. Read it and study it; it's what we'll be using on this project."   Recalling that the purpose of a software methodology is to get quality software out of mediocre people, what is the manager's message? Isn't it that he thinks you're one of the mediocre ones?   Now, don't get me wrong. I am not arguing against methodologies. I am by no means advocating that we go back to the bad old days of ad hoc development, programmers as rugged individualists, and completely disorganized processes. I'm just asking the question: Can even the best, most carefully crafted and religiously followed methodology produce quality software from mediocre people?   I'm asserting that it can't. I'm further asserting that the more stringently we enforce the methodology, the more likely we are to get a mediocre product.   Is that Ok? Is "good enough" good enough? Maybe it is. But it's still mediocrity.   Quality Assurance Many organizations have independent departments responsible for Software Quality Assurance (SQA). I've worked with a few of them. I've chaired quite a few design reviews, and I've managed an SQA department. An independent SQA department is justified in order to relieve the software developers from conflicting interests.   Most software developers want their software to be better than mediocre, but they're also faced with tight schedules, loosely defined specs, and critical managers. In the end, they may be focused more on keeping the boss of their backs, than insuring quality. An independent QA group can be better divorced from production schedules.   I've had mixed results from the QA folk. In one company (Honeywell), the QA group was absolutely wonderful. They saved my buns more than once. For one thing, they were experts at reading and understanding the various specifications, and the things we had to do to meet the contractual requirements on schedule.   Even more important, they were good software engineers in their own right. They actually desk-checked every line of our software, and pointed out many errors that the team members had missed. The team members didn't always appreciate what they viewed as "snooping" into their software, but I sure appreciated it.   Not all SQA groups are so helpful. In all too many cases, the head QA guy is more bureaucrat than engineer, and often doesn't even know programming. He's more focused on checking off boxes in a questionnaire. Did you have the review on schedule? Did the following questions get asked? Have all the action items been completed? If he can say "yes" to all such questions, he can check the little boxes and declare the software to have quality.   What it really has, at best, is mediocrity. More about this in my next blog..