Is a Qt-based UI reliable enough to be used in a medical device? [closed]

Question

I work for a small company developing a complex medical device with a rich UI. We are currently at the early stages of design. The application is targeted for Windows (desktop only) and preferably should be written only in C++.

After some research done we tend to choose Qt to develop the UI. It seems to answer all our needs, namely a modern-looking and highly responsive UI can be developed, the development is rather fast (after getting familiar), memory usage is somehow reasonable, free for commercial use (bonus for us).

My question is: is it reliable enough for a medical device? We absolutely can't accept any crash in the middle of an examination. I understand that first of all it depends of course on the quality of code we write but still I'd like to know if anyone has encountered any mysterious crash-related problems that were particularly hard to resolve. Especially when using QML that is a scripting language and it can naturally result in errors hard to predict and explain.

The cost of encountering such an issue in production will be very high for us, so we extremely need a right decision to be made before we go for any specific package. If you know any other Qt-related problem that could arise in our particular context (I admit that it was impossible to do a very extensive package testing), I'll highly appreciate mentioning it too.

Sweaty

So you intend to bet the farm on advice from strangers whose qualifications to provide such advice are completely unknown to you ? Take my advice, get better advice. – Pasteurization

will nokia not pay you when qt crashes an air breathing machine and the patient dies? perhaps you can find some deal with them. for example, if the cost is hugher than 100k euros, they will pay. – Calabro

I guess every highly complex UI toolkit like Qt is has its pitfalls. You just have to know and avoid them by performing enough tests. It won't be different if you develop your own UI. – Trivet

I'd definitely stay away from QML for now. This is still a relatively new technology and it's probably not as stable as the rest of the library. – Lovable

@HighPerformanceMark i'm not really trying to rely on strangers' advice, just trying to learn something from more experienced people. This site has a good reputation so I expect people that feel able to give a valuable answer to do it, and I'll take into account their opinion. Especially if they have a good reputation - like yours. Anyway, I can see your point and I'll take your advice seriously. Thanks! – Sweaty

@JohannesSchaub-litb I appreciate your humor :) Hope you'll never get a chance to meet our qt-based air-breathing machine in the real life ;) – Sweaty

Here is another similar question: #9795608 – Shumate

Qt is used in a wide range of major commercial apps, including by Adobe and others, so if you know what you are doing, it can be a great environment. But bad programmers will always screw up a nice thing. – Sabba

Answer 1

In my opinion Qt is stable enough if you follow their coding style. I would also buy support from Digia and use a stable version of the library.

The thing comes down to:

• evaluating the risks introduced by the usage of Qt (or any other GUI library)
• act defensively against those risks (introduce new requirements and tests that cover the risks, e.g. run the GUI as a separate process that communicates with the core via TCP or whatever)
• evaluate the risk of a crash in the application and document the procedures to be followed when that scenario occurs.

In my experience with the certifications of medical devices, a loud crash of the device is preferable to a silent and erroneous operational one. When in doubt, ask the certification body that is following your case.

Also, have a look at the standards (e.g. 60601-1-4 or whatever is used now).

Usage of Qt in medical apps: http://qt.nokia.com/qt-in-use/qt-in-medical/

Medardas

Relevant standard is IEC 62304 for software embedded in Medical Devices (60601 is for "medical electrical equipment", not directly related to software). – Notice

Answer 2

I would consider the basic tenets of high-reliability engineering sufficient. With those, you can use Qt. Now, Ambroz Bizjak mentions quite a few "troublesome" scenario's. They're irrelevant, when you follow the basic rules.

So, what are these rules? They're not very hard. Identify the bits that can fail, and do those at times when failure is not critical. For instance, Ambroz has a good point about window deletion. Don't do that in the middle of an examination. Dump the ojects in a defered deletion queue, and make sure they don't interfere with operations (i.e. objects must have a passive state. E.g. for widgets, that includes being invisible). Similarly, create all objects that you might need (including all possible dialog windows) before you start the examination.

You can nicely summarize this as

1. Preparation that can fail
2. Things that may not fail
3. Cleanup that can fail

I would avoid QML for the reasons already mentioned. The scheme above highlights why it's so problematic. You can't move all the suspect QML steps to the preparation phase.

Regenerate

My point was also, wouldn't you prefer if you could just delete that window, and trust the framework to do its job correctly? When writing a reliable system, I certainly would. – Cambridge

I'd also prefer systems where new would never throw. But as an engineer, one of the core tasks of my job is dealing with the unavoidable imperfectness of the world around me. – Regenerate

This is not the same at all. The design problems I have exposed actually have solutions and there are probably better alternatives to Qt, but every single computer has a finite amount of memory. And running out of memory is not that hard to avoid - make sure there are no memory leaks, and add some safe upper limit on how much stuff you load. E.g. if this is an image viewer, add a limit that you never open more than N images at once, and that no single image is larger than XxY. – Cambridge

@AmbrozBizjak : sure, I'd love to trust the framework to be 100% bug free... but back in the real world, you have to accept there is nothing that is totally bug, defect and side-effect free. – Postbox

Answer 3

I'm assuming you're not developing a safety-critical device, because you are running Windows and the Windows license agreement has a few things to say about that. So your question is really, "We are making a consumer product that must be as stable as possible or we will look really, really bad".

Personally, I could suggest using C#, because it has excellent tooling under windows and is significantly easier and safer to develop with (Garbage collectors are your friend for stability, if not performance) and slightly more convenient to unit test. Use C++ or C++/CLR for any performance critical sections, but there's no reason to use such a complex and potentially hazardous language for constructing your GUI.

Evaporate

First, thanks for taking your time to answer! I think you rephrased my question perfectly. Second, we ARE developing a safety-critical device, could you please elaborate on those Windows issues you mentioned? A link will also be of a great help. – Sweaty

Another question regarding C# - do you mean going to WPF? I've heard some rumors that MS doesn't plan to invest in it anymore. – Sweaty

Writing in a safe language (e.g. unexpected exceptions instead of complete crashes) does not make your program any more reliable. What does make it more reliable is thinking about it (proving facts about it), and testing it. And these things are easier in some languages than others. In fact, C seems to be widely used for safety critical systems. This question might be relevent: #243887 – Cambridge

@AmbrozBizjak: it removes whole classes of subtle failures by generating exceptions instead of allowing corruption caused by buffer overflows, and the like. It provides a substantial number of means to check and assert invariants and many tools are available to assist reasoning about your software. However, my point was made with the assumption that the OP was not writing a safety critical system. Safety critical systems tend to require real-time performance guarantees that neither windows nor the CLR nor C# can offer; C is a good choice in that situation. – Evaporate

@user1526557: if you are writing a safety critical system that could result in injury or death if there's an issue at run time, discussions over GUI technologies are singularly inappropriate and irrelevant. Stop right now and seek some professsional advice! I don't think that Stack Overflow will necessarily help you pass FDA certification. But as regards WPF; yes, it is on its way out. Windows Forms is a workable alternative whilst you wait for a suitable replacement (eg. WinRT). – Evaporate

@Evaporate yes you are correct here. I was just trying to say that you don't strictly need those features to write correct code. For "unmanaged" code, there are also tools like Valgrind which detect many of such cases. – Cambridge

@Rook: You're mistaken about the buffer overflow problem. In safety-critical code, the consequences of a buffer overflow are unacceptable, period. You therefore ensure in the development process that they do not happen. As a result, it is fully irrelevant what happens after them. Cosmic rays become more important, as they do happen. – Regenerate

I have to clarify regarding the safety-critical issues: all hardware related code that could possibly cause an injury runs on a separate (different) machine, that only communicates with a machine running GUI. GUI provides an examination setup and displays the results. Still, GUI failure will cause a very bad user experience that we can't afford. – Sweaty

@MSalters: nicely said. In fact, I find buffer overflows one of the easiest bugs to avoid, if only you care about it. – Cambridge

Right, so my original rephrasing, "We are making a consumer product that must be as stable as possible or we will look really, really bad", was accurate. Please don't confuse the issue by saying it is safety critical! I will repeat my original C# suggestion. It is supported by the OS vendor and is a mature platform with excellent tooling. I would definitely choose it over Qt and C++ for a Windows-only application. – Evaporate

@MSalters: perhaps you missed the bit where I said "Not a safety critical system". And also missed it the second time, right underneath where I talked about buffer overflows. Here it is again: my statement was made with the assumption that the OP was not dealing with a safety critical system. Safety critical systems were not considered when formulating my answer. No safety critical systems were harmed in the typing of this response. – Evaporate

@Evaporate Thanks! Sorry for confusing I caused. – Sweaty

@Evaporate "highly reliable systems" are not too different from safety-critical systems. Many of the same approaches are useful, though they may not be applied to the same extent or with the same formality. – Cambridge

@AmbrozBizjak: nonetheless, I posit that the number of ways in which you can induce subtle application failure modes, and the number of ways in which you can impair your reasoning about a program both increase dramatically when you use pointer arithmetic and manual memory management. Buffer overflows may be easy to avoid, but as they are one of the most common programming mistakes ensuring that they are not possible to create at all removes that particular class of issues from your code. Do you disagree? – Evaporate

@Rook: Yes, I disagree. I have no problem reasoning about a program with pointers, and I think that manual memory management actually helps you write correct code, because it forces you to think, in parcicular about the lifetime of objects in the system, which you need to understand, manual or automatic memory management. Of course it all depends on the overall design of the program; if the design is poor and components are not documented well, working with pointers can of course get very ugly. – Cambridge

@AmbrozBizjak: I think at this point we will have to agree to disagree. Presumably you also feel that having a strong type system is also no benefit, because entirely dynamic, weak typing forces you to think about the actual objects you are passing around? – Evaporate

Answer 4

From my expereience with Qt, it is a bad choice when reliability is important. For example, many crashes are caused by seemingly valid code but which did something in some specific context, that some part of Qt did not expect. Here I list some problems I've encountered with Qt regarding writing correct code:

• Deleting QObjects (read: anything) is a very painful procedure. If you delete it from a signal being emitted by this object (read: half of the time), your program will likely crash, because the code emitting the signal will not care to check if the object still exists after you signal handler returns and may continue using it.

• If you look more closely, the suggested workaround is to use QObject::deleteLater() which will make the event loop delete the object sometime, when it's safe. While this may seem acceptable, consider that during that time the object still exists and can emit signals. This introduces unnecessary intermediate states, which you have to handle, and this will introduce bugs; likely ones that happen rarely and in very specific circumstances (read: not while you're testing it). Anyway, this page discusses some problems with QObject deletion.

• Other classes in Qt have similar deletion-related problems. For example, QGraphicsScene::removeItem() may cause a crash if you remove the item from its scene (not delete it) within its mousePressEvent handler. And this does not seem to be documented anywhere.

• Qt classes are full of convenience features mixed with the core set of features, making it hard to figure out how the class actually works and how to properly use it. For example, QProcess, beside the stateChanged() signal which is the only one needed, has signals like error(), finished() and started(), whose semantics and proper handling is not entirely clear.

• Many abstract interfaces are badly designed and poorly defined. For example the class QIODevice is used for both reading and writing and both blocking and asynchronous I/O. A class is free to chose which subset of this to implement. This violates the whole idea of abstract interfaces, which is that anything that implements the interface works in a defined and consistent way.

• There is no really good and uniform way to handle events. There are two kinds of events in Qt: QEvents, for which callbacks are installed by reimplementing virtual functions, and signals. Why? Anyway, both kinds of events are in the end just about calling functions (directly, forget about queued signals here). This means that various modules, can only communicate by calling each other's functions. As has been seen, this is problematic because when a module calls another module, that other module could really have done anything, and it may be hard to consider all possible scenarios at every callback site (as we've seen, Qt doesn't, and instead crashes).

  This problem can be completely solved by adding a feature to the event loop which allows for much easier communication and synchronization across modules. I describe this design in this question.

To sum up, when aiming for maximum reliability, you do not want such design issues in the framework you base the program on, and you do not want to workaround problems with it on almost every step. Even if you properly work around all problems you could think of, how do you know there aren't more? I think you should just try writing some completely correct code in Qt (that is, read all the documentation, and think on every step what you're doing and how the framework will react). After some time, ask yourself: do you trust the framework, or do you feel like it's trying to trick you?

Cambridge

The page on object deletion you link to has information regarding thread safety which only applies in Qt versions prior to 4.0 (which was released 7 years ago). Qt is certainly not without bugs, but in general I find it much more reliable than you describe it here (QML aside, which last time I used it was absolutely riddled with memory leaks and other issues). – Sexuality

@AmbrozBizjak Thanks for taking your time to answer and for the information provided. Could you please describe which kind of applications have you mostly developed with Qt? And which C++ UI toolkit you can consider really reliable? – Sweaty

@DanMilburn sure, some bugs have been fixed, but the general problems I pointed out still stand. The removeItem() crash happens in Qt 4.7 I think. – Cambridge

@DanMilburn The question is - do you think it is reliable enough comparing to any other toolkit? And regarding QML I feel it is still very useful when it comes to rapid UI prototyping. – Sweaty

@Sweaty I haven't been doing anything too big. Some simple stuff with QGraphicsView and widgets, no QML or anything. But it's so much more problematic that I've encountered all these problems even simple programs. – Cambridge

@DanMilburn: I don't know about other toolkits much. They may as well all be non-reliable, having been designed for desktops and other systems where reliability is not the top concern (again, I don't know). I'm not aware of any tooklit designed for safety-critical systems or which has reliability as a primary goal. I was just commenting Qt, which is what the question is about. – Cambridge

@Sweaty It is reliable enough for the applications I've used it for. I don't have much experience with other toolkits, but I've used Qt for everything from QML phone applications to very large desktop GUI apps and it's been reliable enough. – Sexuality

@Sweaty rapid and safety-critical don't mix together well. Actually, they don't mix at all. – Cambridge

Note of these problems sound like they are unique to Qt. X11 in particular is such a baroque idea that I would avoid it without further thought; Win32 (GDI) is euqally tricky. – Regenerate

@Regenerate I believe you; it's a sad state. But it's not an excuse for why Qt can have such problems (or any particular framework) :) – Cambridge