How to debug a crash that only occurs on application shutdown? (Delphi)

Question

So, after some recent changes we discovered that one of our oldest applications is sometimes crashing on shutdown. This manifests itself either in the form of "Runtime error 216" messages or in a message from Windows Error Reporting that the application has stopped working. The application is already emitting OutputDebugString-messages at every turn and AFAICT all of our own code gets executed correctly to completion. All destructors are called as are all finalization sections and class destructors, none of which are raising any exceptions.

Also, neither madExcept nor FastMM4's Full Debug Mode seem to have anything to complain about (though that might be a false conclusion because the crash might happen even before those components' own finalization code runs).

So, what would you do? Where would you start?

---

This question is supposed to be more about the general approach to this class of problems than about the specific instance I'm currently facing so I'm deliberately leaving out details. Feel free to ask if you think they might be relevant to the choice of debugging approach and I will add them later.

Liederman

There is a VCL bug that manifests as AVs on shutdown. It's related to Owner destruction of forms. I work around it by explicitly closing forms in the .dpr file after Application.Run. If you let Application do the closing for you there can be problems with stale references. – Lodhia

@David: That sounds interesting. Do you have a QC number for that? However, except for the main form I'm creating all forms in code with the respective parent form as the Owner. Would the issue you're talking about apply in that case as well? – Liederman

if you are relying the destruction of Application to bring everything else down, then the bug I'm talking about could apply. Try putting a MainForm.Free in your .dpr file. I'll look out the QC number. – Lodhia

Answer 1

A runtime error 216 means that you have an Av (access violation) and SysUtils has already stopped translating those errors to exceptions.

First try: Build with debug DCU's and look in the unit system where the error is raised, set a breakpoint there. hopefully you can catch it in the debugger and work from there.

You probably have a memory bug (dangling pointer, null reference etc etc use of s string constant in an already finalized unit) and the best trick is to check the finalizations after sysutils is finalized. You can do this by building WITH debug dcu's, setting the break point to the finalization in sysutils and start stepping through the code until the error occurs.

Azo

Answer 2

Are you using runtime packages? I've seen similar issues before. If you're sharing globals or interfaces across package boundaries, you have to make sure that all references to classes that belong to a certain package get cleaned up before that package is unloaded; otherwise they'll try to make virtual calls into memory that's no longer valid.

Geniculate

Oh yes, not only runtime packages but also COM-DLLs (i.e. plugins) that depend on these packages (as does the host EXE). Don't ask. I wouldn't do it again this way today, but it has been running flawlessly for over 10 years now... I'll double check any references to some of the global singletons though. That does indeed sound promising... – Liederman

"running flawlessly for over 10 years now" What has changed? That's probably how I'd attack this problem. – Lodhia

Touché! ;) That one's almost good enough for an accepted answer. :-P – Liederman

Answer 3

Well the Runtime error 216 are memory issue (Access violation), it seems like you refer to some Object which doesnt exist at that point of time.

Emarcadero writes:

Applications that use the SysUtils class map most runtime errors to Exceptions, which allow your application to resolve the error without terminating

Did you try to set a breackpoint at the finalization section of Sysutils?

I would try it with an Allocation / Memory Profiler, its not sure you will find the error code line but this may show you some parts of your code where Memory issues occure.

Richers

Answer 4

You've likely got a pointer problem. Some event or method is trying to run on an object that doesn't exist anymore.

Vaish

Yes, that's what the "Runtime error 216" more or less implies, but the question was how would you go about tracking that problem down? – Liederman

If you can reproduce it reliably then you can use the debugger (you'll need Debug DCUs) to work out where it happens. Unfortunately I think you are liable to have a pretty hopeless stack trace at the point it raises. – Lodhia

@David: "reliable reproduction" of the issue is still a problem: I'd say it happens about 60% of the time. And I haven't yet been able to determine what the difference between the working and the non-working sessions might be. I have actually already thought I had fixed the problem a couple of times only to have it reoccur a couple of sessions later... – Liederman

Answer 5

"Runtime error 216" is from Windows itself, not the Delphi exception handler. I've found that it is caused by code that runs in initialization and finalization sections of units, which execute before the Delphi exception handler kicks in. Particularly. COM objects that need to unload through finalization code running after the Delphi App has terminated will cause this and similar errors. So check that stuff.

MNG

Aqaba

Not quite: see in unit system: function MapToRunError maps the NT error code STATUS_ACCESS_VIOLATION to ErrCode := 216. – Azo

Thanks for the comment - regardless, this occurs before or after the default Delphi application exception handler has been initialized. Units are loaded based on uses clause before Delphi application is launched, unloaded after application has terminated -initialization and finalization section run upon unit load and unload and so the code running in there isn't handled by the Delphi application exception handler. That's why you get a cryptic 'runtime error 216' or 217 and no exception message 'Eaccess violation.. @ ' etc. – Aqaba

This is not true; the 216 is from the Delphi RTL, after the exception handling mechanism got unhooked. It is the Delphi way of telling there is an AV. The 216 is not form Windows. – Deform

Please see above, as explained. – Aqaba

Answer 6

Like others said: 216 means AV after SysUtils was shutdown. Usually, the only thing that shutdowns after SysUtils (and have chance to raise AV) - is System unit. Specifically: memory manager.

So, run-time error 216 at shutdown usually means memory corruption bug in your application.

That can be very easy to solve - just enable full debug mode in memory manager or use a debugging memory manager. Sometimes, however, it can be very hard to find. But you can start with debug mode of MM first.

See this article.

Dedal

Answer 7

Old thread, but actual problem. My Solution was to add Application.MainForm.Free; to the FormDestroy procedure of the primary / main form to remove the error on closing. Try with "MainForm" instead of your form name. Only this works for me. Maybe this could help someone, too.

Tatter

Just in case ... Which Delphi version are you using? Does it concern all of your applications? Do you see the error every time you shut down? I ask these questions, because I have not seen that problem, so I wonder what is so special with me (or my programs). – Mayer

I use an older version... It's one of the XE Versions. Maybe this could be the problem. In my case there seems to be a problem with an multidimensional array that is causing the problem. (Have copied the project and deleted all parts of the program step by step until the error disappeared.) I think there is an issue with my code. So if I change my code this should work, too, but I did not find another way for the routine to do it better. This project and the solution is ok for me. This was only a simple project for personal use. – Tatter