Software Quality, Defects and Requirements
Steve Rakitin of Software Quality Consulting provides an assessment of the state of the software quality assurance profession in his latest newsletter. He notes the following regarding software bugs:
Today the best most highly skilled software developers inject an average of one defect for every 8 lines of code they write. [2] The single most common reason software engineers inject defects has been and is still poorly written requirements.
We also have anecdotal information suggesting that through verification activities such as peer reviews and static analysis and validation testing, we typically find about 95% of these injected defects. The end result is:
released software has, on average, a defect density in the range of 5-6 defects per thousand lines of code (KLOC).
So if we look at a typical software-based system that has one million lines of code, here’s what we’d expect to find:
Defects injected: using 1 defect /8 lines of code = ~120,000
Defects removed: assuming 95% found = 114,000
Defects remaining: (defects injected - defects removed) = 6,000
To put this in perspective, 2010 model-year cars will have about 100 million LOC. Given the above, there could be as many as 600,000 defects that remain in the software that controls your car.
Imagine the potential safety impact of that many defects in a complex medical device system -- or a system of systems.
More interesting, is the primary cause of these software defects: poor requirements. In my experience, PCDI or medical device connectivity applications are particularly susceptible to requirements related software defects. The reasons for this are pretty straight forward.
Unlike generating requirements for the embedded systems (the conventional stand alone "black box" medical device) for which manufacturers have many years experience, requirements generation for connectivity is a new challenge. This is further exacerbated by the fact that requirements gathering for an embedded device can be quite different from capturing connectivity requirements. Different kinds of methods and tools are frequently used because the workflow automation and systems integration required for connectivity are fundamentally different from the "knobology" and operation of the embedded device.
Over the years, I've received numerous calls from manufacturers asking me to, help them "finish" a project. A good engineer, if provided with adequate requirements, can complete a product design. After exploring the situation, the problem almost always starts with, or comes down to, a lack of requirements. Those projects become exercises in capturing the missing requirements, and training the client on how to capture connectivity requirements in the future.
