Python coding standard for Safety Critical Applications
Asked Answered
G

3

5

Coming from C/C++ background, I am aware of coding standards that apply for Safety Critical applications (like the classic trio Medical-Automotive-Aerospace) in the context of embedded systems , such as MISRA, SEI CERT, Barr etc.

Skipping the question if it should or if it is applicable as a language, I want to create Python applications for embedded systems that -even vaguely- follow some safety standard, but couldn't find any by searching, except from generic Python coding standards (like PEP8)

Is there a Python coding guideline that specificallly apply to safety-critical systems ?

Granddaddy answered 22/10, 2021 at 8:35 Comment(5)
While it's surely possible to use Python for safety critical apps, it wouldn't be my first choice. Perhaps that's the reason you can't find any such standards.Backbone
"Skipping the question if it should or if it is applicable as a language" no, this is a critical question, you can't just dismiss it because you want to use python, your main question can't be answered because python must not be used for such applications. I would love to be able to use python for everything but in some situations it's simply not possible, and this is one of those situations.Trotta
@Shinratensei Perhaps was not clear from the post, I am not going to use Python really for safety-critical systems, I am trying to level-up the robustness of my systems, by following ( but not totally implementing ) a possible Python safety standard. That's the deal with "skipping the question" hypothesisGranddaddy
Then there is nothing besides the PEP8, although that doesn't really improve the robustness of your software, it just attempts to remove ambiguity and bad practises but it's far from perfect.Trotta
Check it: pamirsevincel.substack.com/p/… "FEATURE: Concerns with using Python in Machine Learning Flight Critical Applications"Elect
G
4

Top layer safety standards for "functional safety" like IEC 61508 (industrial), ISO 26262 (automotive) or DO-178 (aerospace) etc come with a software part (for example IEC 61508-3), where they list a number of suitable programming languages. These are exclusively old languages proven in use for a long time, where all flaws and poorly-defined behavior is regarded as well-known and execution can be regarded as predictable.

In practice, for the highest safety levels it means that you are pretty much restricted to C with safe subset (MISRA C) or Ada with safe subset (SPARK). A bunch of other old languages like Modula-2, Pascal and Fortran are also mentioned, but the tool support for these in the context of modern safety MCUs is non-existent. As is support for Python for such MCUs.

Languages like Python and C++ are not even mentioned for the lowest safety levels, so between the lines they are dismissed as entirely unsuitable. Even less so than pure assembler, which is actually mentioned as something that may used for the lower safety levels.

Golden answered 2/2, 2022 at 8:46 Comment(9)
(Similarly, MISRA-C:2012 with TC frowns at pretty much all new features introduced in C11, to the point where I think they are being needlessly conservative. At least it covers C99 though. But notably C99 wasn't supported until as late as 2012. "Proven in use" is a recurring term in safety-related contexts, even though it's very subjective.)Golden
Thanks for the insight @Lundin, now I understand the mentioned mindset. My intention was to make my Python SW more robust, rather than actually implementing Python in real-world Safety-Critical systems. It seems like sticking to PEP8 coding standard is good enoughGranddaddy
@Granddaddy No matter the context, applying any coding standard is usually a big improvement of software quality. Ideally it would be enforced with peer reviews and static code analysis too.Golden
Misra C++University
@artlessnoise Yes it's been around since 2008. However, I haven't seen any safety standard suggest that it's a suitable programming language. I only have access to older versions of the functional safety standards though. Given all the changes from C++11 and beyond, I think it is even less likely that C++ is considered suitable now, than before C++11.Golden
Uhm... about C++... see DO-332... JSF Coding Standard, and MISRA C++... Consider updating answer.... cheers...Elect
@Golden - see Table C.1 in 61508-7 and check your answer again for C++ #12 indicates HR for conditions being taken for safety critical... same with DO-332... Annex G goes into further details... down vote until corrected...Elect
@Elect OK so maybe C++ was added in a newer version of the standard. However, the language is extremely unsuitable for safety-critical/deterministic applications due to the massive amounts of poorly-defined behavior in the language. They even have a whole ISO working group trying to figure out just how much of C++ that's poorly defined - it seems no living human has managed to compile a list. With that in mind, recommending C++ for such applications is just utter madness. Any safety standard doing so is a safety hazard in itself.Golden
@Golden - see DO-332 (2011) and Table C.1 in 61508-7 (2017)... and JSF C++ coding standard (2005) (stroustrup.com/JSF-AV-rules.pdf) from Bjarne Stroustrup, i.e. one of the main creators of C++ - note he is still living... in the safety community this is established precedent if the guidance provided is followed... peace.. check out those not-new references... please provide references for your statements above as that will help to pursue them further as not aware of such sources... thanks again... cheers.Elect
U
2

It is possible depending on the safety case and governing standards body to use Python in a safety critical system.

Technical - Availability and Real time

However, for continuous safety, there are always time demands. For example an anti-lock breaking system must always be ready to perform. Ie, high availability. It will also have timing guarantees. It is no good if the anti-skid mechanism engages after a fish tail has started. This would be a real-time gaurentee.

Many higher level languages such as python include garbage collection. If the garbage collector is not incremental or controllable (when the garbage collection happens), it is impossible to fulfill the timing guarantees. It is difficult to have timing demands meant in Python.

Some systems are not continuous, such as a Covid assay (do I have Covid-19 yes/no). It is more important to be reliable, meaning do I get the correct results all the time.

Standards

As Lundin alludes, some standards are prescriptive such as the Automotive standards (ISO-26262) and the base standard IEC 61508. That is they give a list of ways to achieve safety.

Some standards such as IEC 62304 (medical software) are goal oriented, but allow the use of the prescriptive IEC 61508. A safety case must be made through technical arguments that Python was a good technology choice for the use case. This can be very difficult, so defaulting to the prescriptive standard is the norm.

Frankly, I believe that Rust would probably be a better choice than Python for a wider variety of cases. Some regulatory bodies allow you to have your safety case previewed before large scale development gets underway. If you have a goal oriented standard, it would be very prudent to get some acceptance of your argument for the language from an auditing body.

Reality

Entrenched languages will have tools and pre-certification. For example you can get TUV-Sud certified compilers. Not only the code, but all aspects that are used in development must be analyzed for a safety case. This include static checkers, revision control tools, code review tools, CIT systems, etc. Depending on the 'safety level', you many need all of these elements to be certified. Ie, Level C pace maker or ASIL-4 automotive component. If the language was never used in a safety standard before, it can be difficult to find certified tools, an OS or run-time libraries.

As well, it can be fairly expensive to make the arguments that a new technology is safe. This means there will be added cost to the company that initially undertakes this exercise. A product that succeeds, makes it to market and helps to save lives is better than an intellectual exercise that fails.

University answered 20/3, 2022 at 15:20 Comment(0)
M
2

It will be hard to find guidance for Python in safety critical software development because there are too many things that make Python no option for really critical applications. You could never entrust a piece of Python code your life. See a few reasons why:

The memory model in Python is much different then the one in languages for embedded software. Python uses massively the Heap, while safety critical software normally uses global data and the Stack. For dynamic memory it is just too complicated to perform a safety assessment. You have phenomenon like memory fragmentation, garbage collection, memory leaks. The algorithms in Python, that prevents these things, have never been qualified.

In Python you normally don't care for dependencies, cyclic dependencies. As everything is interpreted at runtime, it just works. But safety critical applications normally require that you have a Top-Down design with abstraction levels and dependencies in a tree structure. Because this is, what the V-Model wants. That is the reason why Python code most of the times is not compliant to safety standards.

Of course in Python it is as easy as in languages lika Ada or C to write good and certifiable code that follows certain design principles. But it is also as easy as writing code that does not follow good design principles. That means it is harder to convince the authorities that you follow the right principles. With "right" principles I mean that the authorities want to see what you consider as your principles. You have to define that. With Python you would start from an empty paper. And then you have to convince the authorities that these are really appropriate principles and that you are compliant to them. Most teams would not even have the skills to define their own rules. The certification liaison is then a lottery. Example of such principles, you can see here or here

For example in Python it is normally not a problem that Low-Level functions call High-Level functions. You can just do that. But statically typed langauges would not compile in this case. They force you to establish Dependencies in a stricter way. In practice it is extremely hard to establish such principles and convince developers of them if the compiler does not enforce this. For example the Dependency Inversion Principle, which you do often in Object Oriented Design is not something, that is easy and obvious for developers. It is hard to establish that mindset, especially in teams that have more an Embedded Software or hardware background.

Python developers prefer a defensive style of programming, so that the code is immune against adverse unexpected situations. But in safety critical code all situations must be taken into account in advance. Defensive programming prevents the detection of design errors early in the development.

For things like lambda expressions and closures it is hard to use test coverage metrics. But these are required in safety critical software development. You at least need to cover each Python statement once because without static typing, testing is extreme important. Otherwise you would not reliably identify type errors.

I understand that you don't expect that level of safety and that you just want to have certain (the best possible) level of safety. It would indeed be interesting to have guidance what features of Python can be considered as safe and what programming techniques are recommended. Unfortunately there seems to be no guidance available, yet. Perhaps people are worried too much about the run time environment and the concepts of Python not been mature enough or adecuate for safety critical applications.

A good book for reading is "Developing Safety-Critical Software - A Practical Guide for Aviation Software and DO-178C Compliance" by Leanna Rierson. It gives very interesting insights about what you have to care when developing safety critical software. It covers even things like Object Oriented Programming. When it comes to C# or Java it is said, that these languages are just not mature enough. And it can be supposed that the same is the case for Python.

I end this with a practical example. In safety critical applications the program must be deterministic. The program behaviour must be already known and predictable at "certification time". But see this code:

def print_message(msg):
  print(msg)
  
def print_in_capital_letters(msg):
  print(msg.upper())
  
print_message("Hello")

if i_want_capital_letters:
  print_message = print_in_capital_letters
  
print_message("Hello")

It is hard to make clear statements about the behaviour of print_message. It depends on the value of i_want_capital_letters at runtime. Such dynamic behaviour is hard to bring in line with certification requirements. The certification authorities want to hear what the behaviour of the code actually IS and not that it depends on something, that is comming from the user. That sort of flexibility is probably a reason why Python became so popular. When trying to avoid such features, it makes probably no sense to use Python, at all.

The problem with the example is not so much, that the program behaviour changes due to user input. It is just not obvious, that the bahaviour of print_message is permanently changed. There is nothing like a status variable that makes this change visible. It is even hard to find out, what the behaviour in certain moment in time is.

(You may argue, that this example is already an exotic use case, that does not happen so often. But this is actually the way how things for Unit Testing purposes are isolated. All things that do not belong to the "unit under test" are "stubbed" away using exactly this mechanism of the example. But the valid question of the certification authorities is then, if you have really tested the operational code or some manipulated code. For programming languages that use static typing, the type system provides "some" sort of confidence, that a stubbing mechanism does not change the semantic of the program, while in Python many things can be changed at runtime.)

A final show stopper for critical applications is the runtime system of Python. Safety critical software must be 100% reliable. That includes each Assembler instruction that is ever executed. If you compile a piece of C code, normally you get in Assembler quite exactly what you have written in that C code. Projects with a high level of safety requirements even have a process that verifies that the Assembler code fully traces to the C code. If you don't use any libraries there is no software pieces between your C code an the operating system or bare metal. All this means, the verification (e.g. testing) of your running Assembler code (and your product) can be done by verifying (e.g. testing) the C code. There is nothing (or at least not much) more. In Python this is different. Python is interpreted or at least executed on a basis of bytecode. That means that the Python interpreter is part of your product. But the Python interpreter is an Open Source project, that is completely out of your control. It has never been developed with a focus on safety. It is simply not possible to certify a product if safety was not taken into account from begin on. The Python interpreter would never be certifiable.

Mcnutt answered 24/7, 2022 at 16:0 Comment(7)
Good deal: FEATURE: Concerns with using Python in Machine Learning Flight Critical Applications - pamirsevincel.substack.com/p/…Elect
Interesting source. What they say "use Python to develop [...] but then transform it to a certifiable language i.e. C/C++" is just an idea, not a recommendation. You cannot say, it is just another programming language. If you have a program in Python, that uses all nice features of dynamic programming (which is very normal in Python), it is unclear if this can be expressed appropriately in C++, at all and if this is really a good way of working and if it pays off. It may be even too challenging to conceptualize a sound development process if you have technologies that are so different.Mcnutt
absolutely, grab the paper or contact the authors if you can... for that difficult transition they go into the possible use of Neural Network Exchange Format (NNEF), Open Neural Network Exchange (ONNX), and/or AITemplate to aid in the translation, but indicate it is a risk. However, some organizations seem to be attempting to pursue such transitions. Others are trying to stay with C++ environments all the way through. Interesting paper.Elect
You're not making any sense. "In python you don't normally care for dependencies"? Huh? Just because stuff happens at run time doesn't mean dependencies are not cared for. Implementing a top down design following the V model is just as easy in python as any language. Your example code can be written in C, too. That's just how branching works.Chilopod
@MattEllen Thank you for the comment. I updated this in my answer. I hope, it becomes more clear. But of course my answer is only my humble experience. I don't wanted to say that it is impossible or not recommended to use Python. It is just a bit more complicated at the beginning, because with Python you don't build upon decades of experience in software development for safety critical systems. But if you manage to find your way with Python, then it is clearly a good choice. The only real show stopper in my opinion is the runtime system. But I guess there are alternative for it.Mcnutt
@Mcnutt that's fair. python can be included in VxWorks, for example, which is a real time, safety critical OS, so there are ways. The dynamic typing, JIT compiling and garbage collection certainly present a challenge for real time constraints, but the safety critical parts, at least in medical devices, are covered by the processes of design and risk assessment.Chilopod
Also, I'm not sure I understand what you mean by dependencies.Chilopod

© 2022 - 2024 — McMap. All rights reserved.