The US National Security Agency has released a case study showing how to develop zero-defect code in a cost-effective manner. Diomidis D. Spinellis has taken a closer look at the claims and the code released for this project. At first when I read through the article I agree with most his points, but it gets quite interesting when I read the comments.
The things we consider important when living in our business-entrepriecey-systems-world might not be feasible when you’re doing realtime-secure-flightsystems etc. If you are working under the paradigm of formal verification, loops might be a bad thing.
I don’t know much about this stuff, and I am quite content not needing to think about it. That said I really try to strive for zero-defect, maintainable code, it just doesn’t seem worth it doing formal verification on most systems I work on. I should look into it though to learn a little about it without picking it up. I think working in this fashion will probably block all attempts to be agile.
2 replies on “Report on zero-defect code”
I seem to have touched some raw nerves with this blog entry. However, as somebody pointed in another comment, my concerns are orthogonal to those of the study. Maintainability (which includes appropriate naming and correct formatting), flexibility, and developer productivity are not optional attributes of software development. In most environments they are as important as zero defects. If a development method doesn’t address them (and I think I’ve shown that this study falls short in this respect), then people who can choose will use the more functional and up-to-date software, even if it is full of bugs and vulnerabilities. The end result is the nightmare of millions of Windows zombie machines we’re currently facing.
In environments where people aren’t allowed to choose they will actively try to circumvent dated and spartan zero-defect software, using their own laptops and iPhones to get their job done, again at the cost of their organization’s overall security and reliability.
Good points Diomidis. I havn’t read the study so I really can’t discuss that. If the study havn’t considered the longterm effects on development like maintainability it would be a serious weakness. It might even say that they secrifice maintainability (that kind of development really has to), but that should be concious choice.
I really believe in long term software development with solutions that has to evelove and change over time. But I don’t know enough about formal validation or secure/realtime systems to know that it’s not worth it there.