5 steps to effective medical device defect reporting

"Defect rejected - not reproducible” - we have all experienced the gnawing feeling of closing an unreproducible defect. We suspect that the error is still in there somewhere, but without being able to reproduce it, we are fumbling in the dark. In a Medical Device context, where a considerable risk is potentially lurking in this evasive defect, we end up in a very uncomfortable spot.

To minimize these kinds of risks, it is therefore of utmost importance that defect reports contain the information necessary for timely reproduction. 

"We are all interested in getting defects fixed as soon as possible. However, more often than not, defect reproduction takes more time than the actual fixing. The tester has a major role in providing the information necessary to reproduce a defect. However, he can only excel in this role if the underlying concepts of providing accurate diagnostic information are in place, to begin with.", claims Mr.  Michael Stegmann, Chief Executive Director in the Stegmann Innovation Team. 

Mr. Stegmann, having an extensive background in Medical Device Verification and Validation, singles out five critical software aspects that have a decisive effect on effective defect reporting.

Versioning of the software under test

"If the software I test is not unambiguously identifiable, the chance that the development team will be able to reproduce the defect decreases significantly. "V1.0.0.0" or "found in current trunk" will not help anyone. Sending patched dll:s back and forth is a risky practice that eventually will make your environment completely opaque. Therefore, a clear version schema of the software tested needs to be in place from day one." 

Nightly builds with versioned Installers

"Half-baked deployments from various developers will invariably deteriorate your test environment. In the end, you will simply not know what you are testing. Better to go with a centralized build system that automatically creates versioned Installers. 

In that way, all testers have a common source of versioned, predictable deployments that also correctly cleans out all files and dependencies during an uninstall. Setting up a build server does not have to be complicated or expensive. There exist many examples of free and low-priced options in the market."

 Logs, logs, logs

"Logs are one of the main diagnostic tools for finding and fixing defects. When the software developer wants to reproduce the defect, he will certainly ask for the logs. This, of course, presupposes that there exist logs in the first place.

Characteristics of a good logging concept include:

• Logging shall be easy to apply in the code
• Log messages shall have timestamps
• The Log files shall be humanly readable on any computer
• Message severity categories to enable quick visual identification of abnormal entries
• In multi-threaded applications, the logging shall be tread-safe
• The logs shall be stored in an unambiguous and accessible location 
• The log files shall be of a manageable size for viewing and sending, 500kB is a good approximation
• Log message metadata that enables a Log Viewing Application to efficiently filter large amounts of data"

 An Error Handling Concept

"Error handling includes the anticipation, detection, resolution, and communication of errors. When it comes to effective defect tracking the last word, communication, is the most important. If the developer deals with an error situation, seeming ever so improbable, a clear message of what, why, and where the problem occurred, communicated both in logs and on-screen, will go a long way.

Details on the error context, stack traces, and probable causes shall be communicated in a way that is detectable and collectible for the tester. Only then can the information serve as accurate feedback to the developer."

Automatic Collection of Diagnostic information

"Collecting test evidence often includes the finding and bundling of log files, screenshots, PC Information, event log files, configuration file, crash dumps, etc. This is often a tedious and time consuming activity. Automating this process can greatly speed up the verification process and also make sure that the evidence collected is uniform and according to expectations. 

Integrating such a capability in the software will have the additional benefit of making it easier for users and field service engineers to collect and send diagnostic information back to the company in case a defect has slipped through the verification net."

"These aspects shall be designed and implemented in due time before verification starts. In a medical device development environment where risk management is a central factor, a solid foundation for collecting valid and accurate diagnostic information is a requirement, not an option.”

Request a live demo and let us show you how Aligned Elements can help you with your reporting