Steve Rakitin of Software Quality Consulting provides an assessment of the state of the software quality assurance profession in his latest newsletter. He notes the following regarding software bugs:
Today the best most highly skilled software developers inject an average of one defect for every 8 lines of code they write. [2] The single most common reason software engineers inject defects has been and is still poorly written requirements.
We also have anecdotal information suggesting that through verification activities such as peer reviews and static analysis and validation testing, we typically find about 95% of these injected defects. The end result is:
released software has, on average, a defect density in the range of 5-6 defects per thousand lines of code (KLOC).
So if we look at a typical software-based system that has one million lines of code, here’s what we’d expect to find:
Defects injected: using 1 defect /8 lines of code = ~120,000
Defects removed: assuming 95% found = 114,000
Defects remaining: (defects injected - defects removed) = 6,000
To put this in perspective, 2010 model-year cars will have about 100 million LOC. Given the above, there could be as many as 600,000 defects that remain in the software that controls your car.
Imagine the potential safety impact of that many defects in a complex medical device system -- or a system of systems.
More interesting, is the primary cause of these software defects: poor requirements. In my experience, PCDI or medical device connectivity applications are particularly susceptible to requirements related software defects. The reasons for this are pretty straight forward.
Unlike generating requirements for the embedded systems (the conventional stand alone "black box" medical device) for which manufacturers have many years experience, requirements generation for connectivity is a new challenge. This is further exacerbated by the fact that requirements gathering for an embedded device can be quite different from capturing connectivity requirements. Different kinds of methods and tools are frequently used because the workflow automation and systems integration required for connectivity are fundamentally different from the "knobology" and operation of the embedded device.
Over the years, I've received numerous calls from manufacturers asking me to, help them "finish" a project. A good engineer, if provided with adequate requirements, can complete a product design. After exploring the situation, the problem almost always starts with, or comes down to, a lack of requirements. Those projects become exercises in capturing the missing requirements, and training the client on how to capture connectivity requirements in the future.
Between recent regulatory problems (here, here and here) and lagging the market with pump connectivity, Baxter's inked a deal with SIGMA (press release). In the deal, Baxter pays SIGMA $100 million for exclusive distribution rights world wide and a 40% equity stake in SIGMA and the option to acquire the rest of the company in the future. "Baxter may make additional payments of up to $130 million for the exercise of its option to purchase the remaining portion of the company as well as for the achievement of certain R&D, regulatory, and commercial milestones, according to the release."
It's interesting to note the 3 categories of milestones. Given that SIGMA is way ahead of Baxter from a connectivity perspective, the fact that SIGMA has R&D milestones (that one assumes are required to remain competitive) indicates the increasing importance of workflow automation in purchase decisions, and the difficulty of developing that same set of features. It's not clear how Baxter's regulatory issues, which should be mostly behind them, play into the deal -- but a separate product development process and good manufacturing practices would appear to provide a regulatory buffer from any lingering issues Baxter faces. I'm assuming commercial milestones are sales commitments on Baxter's part, and perhaps manufacturing capacity commitments for SIGMA.
Greg Stett, a clinical analyst at MD Buyline asked the hard question:
When we asked Baxter about what their plan will be down the road if they were to completely acquire SIGMA, more specifically if the two product lines would be integrated in any way, they officially said, “Baxter will continue to develop our next generation products, remediate the COLLEAGUE volumetric infusion pump, and establish an agreement with the FDA on a path forward for re-commercializing within the U.S. Baxter remains committed to the COLLEAGUE volumetric infusion pump platform, and development of the next release of the COLLEAGUE volumetric infusion pump will continue as planned.” So, at this time, past COLLEAGUE issues will not slow down future development of the new generation COLLEAGUE products.
A number of questions come to mind:
The criteria are not consistent with those used by CCHIT to certify EHRs. They are outcomes-oriented (CCHIT’s are feature, structure and process-oriented), and they do not require that any particular technology (such as the client-server applications used by the legacy vendors who sit on the board of CCHIT) be used to achieve the results.
Subsequently, HHS announced that it planned to assume responsibility for deciding which EHR systems qualified for bonus payouts under Medicare, and shortly thereafter, ONC’s HIT Policy Committee said that it planned to recommend that several entities should certify EHR systems.
In its announcement, the Committee envisioned the establishment of 10 to 12 such agencies.
The upshot of these moves by the Federal government are that (1)CCHIT no longer decides what criteria will be used to certify EHRs, and (2)its days as the exclusive provider of EHR certification services are numbered.
But as mentioned above, ONC won’t sign-off on its “meaningful use” criteria until the spring of 2010. In effect, CCHIT is asking EHR vendors to gamble that CCHIT can, like a fortune teller at a carnival, predict what those final recommendations will be.
"Choose the risk you want to take," CCHIT Chairman Mark Leavitt recently challenged vendors. "Go ahead now (with a CCHIT review) and have an extra year to implement, with a small risk that there will be some gap in which EHR systems would have to be updated to receive final certification…" or risk the consequences of sitting on your hands, he presumably would add.
Never mind that the latest information is in the public domain, and that any vendor can compare it against their current capabilities and development plans! Disregard the fact that CCHIT has no track-record in promulgating outcomes-oriented certification criteria or in certifying against them!
What is CCHIT charging for its fortune-telling expertise? According to Government HealthIT, a HIMSS sponsored publication, prices for modular certification begin at $6,000 for up to 2 modules. They rise to $24,000 for up to 20 modules and to $33,000 for more than 20.
As always, fees for CCHIT’s comprehensive certification are $37,000 for ambulatory systems and $49,000 for hospital systems. Annual renewal costs are $9,000 for each.
That’s chump-change for the legacy vendors whose top executives sit on the board of CCHIT, but it guarantees nothing.
A recent example of this came from two leading suppliers in the medical sector; Philips and Motion Computing. They have both recently introduced wireless medical tablets, powered by Intel's Mobile Clinical Assistant platform (MCA). This class of device uses WiFi to communicate and features a digital camera, along with both barcode and RFID readers, for scanning medicine labels and identifying patients by their wrist straps. They will also likely feature Bluetooth, to capture patients' vital signs.
So will the introduction of open Bluetooth and ZigBee profiles stimulate growth, and if so how? Gee believes that interoperability, and therefore profiles, will be essential, but other things need to be in place. For instance, to overcome the situation of proprietary equipment interfacing with equipment based on open standards, there needs to be some form of common data exchange, a kind of 'http for medical devices'.
The industry is rapidly approaching a point where an independent test and certification organization is required to provide verification testing of connectivity and interoperability (or semantic or syntactic interoperability) across multiple manufacturer's products and systems.
an initiative aimed to develop wireless medical monitoring systems, or body sensor networks (BSN), which would replace the traditional tangle of bedside cables used to capture a patient’s vital signs. GE’s vision for the systems would enable wireless monitoring from anywhere in the hospital—or even remotely from home.
Proposed Frequency Band: "identified the 2360-2400 MHz band as the preferred frequency band based on engineering studies showing that MBANS devices can successfully coexist with incumbent operators and users." I would love to see that coexistence data. I was told by David Davenport, with GE Global Research that, spectrum just outside 2.4 GHz was desired because it would enable the use of off the shelf 2.4 GHz components, with only minimal modifications.
Permitted Operations and Eligibility: "the proposed rules limit the type of devices that will be permitted to operate within the spectrum to those involved in the monitoring, diagnosing or treatment of a patient." No big deal here.
Authorized Locations: While only medical applications may use the spectrum, users "will be fully mobile, both inside and outside of healthcare facilities, without restriction."
Spectrum Sharing Requirements: Here's the big issue -- "The proposed rules do not require the use of any specific standard protocol but GEHC anticipates that industry standardization efforts may occur. Currently, the proposed rules only require that all devices implement basic contention-based mechanisms to allow predictable and fair access to the spectrum." The same vague promises of working out an industry standard after the fact were make when WMTS was created. Nothing ever came of that, and there were very serious coexistence problems with both GE's and Philip's WMTS the first several years they were on the market. This is a recipe for the creation of vendor specific proprietary systems. This is a great product strategy for GE (and Philips), but not so good for smaller competitors with not so large R&D budgets.
Emission Limits: This is low powered stuff, "MBANS devices be permitted to have fundamental emissions of up to 0 dBm EIRP for the proposed maximum 1 MHz emission bandwidth."