How to detect "dangling pointers" if "Assigned()" can't do it?

Question

In another question, I found out that the Assigned() function is identical to Pointer <> nil. It has always been my understanding that Assigned() was detecting these dangling pointers, but now I've learned it does not. Dangling Pointers are those which may have been created at one point, but have since been free'd and haven't been assigned to nil yet.

If Assigned() can't detect dangling pointers, then what can? I'd like to check my object to make sure it's really a valid created object before I try to work with it. I don't use FreeAndNil as many recommend, because I like to be direct. I just use SomeObject.Free.

Access Violations are my worst enemy - I do all I can to prevent their appearance.

Lanie

AFAIK, you cannot tell whether a pointer points to a valid object or not. That's why you should always use FreeAndNil - it is quite direct – Distinctly

So my 500+ "Object.Free" lines of code I've typed over the past 5 years isn't the right way? – Lanie

It works but is not the best way precisely because of this problem you're having. They didn't create FreeAndNil for nothing – Distinctly

Well I guess that's where freelance programming gets me - never took school for programming besides some sleazy into to VB class. All my knowledge about Delphi I initially learned from 1 colleague and the rest I've learned on my own. – Lanie

@Seth: Have you been tracking the FreeAndNil threads in the Delphi newsgroups? Just asking, I don't want to reignite that holy war. Again. Here. – Phenice

You don't need schooling (most great programmers I know didn't have any schooling), just read some books by people who are really good – Distinctly

@Phenice no, I don't read the Delphi newsgroups, I wasn't aware it was controversial. I don't see why it would be. – Distinctly

@SethCarnegie Reading books is considered schooling in my eyes :P Google on the other hand is where I found the majority of my expertise. – Lanie

Google is basically mostly people regurgitating what they've learned in books (possibly with some loss of accuracy), so it's basically the same – Distinctly

@Seth: There's a school of thought that sees the use of FreeAndNil as a "code smell" indicative of a design problem. The forums are currently offline or I'd post a link. – Phenice

@SethCarnegie it's not necessarily, it's the difference between 'street smart' and 'book smart'. Think about a mechanic explaining to someone how to do an oil change. Although oil changes are well documented in many books, the knowledge was passed through a third party. That third party is what defines the border between book smart and street smart. Then again I'm just rambling on. – Lanie

@Phenice So then it looks like I just walked right into the middle of some battlegrounds? – Lanie

@Seth: Here's the thread -- Smelly FreeAndNil?. WARNING: Rendering this page will make your browser cry. Reading it will make your head hurt. – Phenice

@Phenice I would read it but the site is down again. Jerry no, there is no battlegrounds here, don't encourage retarded debates by acknowledging them – Distinctly

Answer 1

If you have an object variable in scope and it may or may not be a valid reference, FreeAndNil is what you should be using. That or fixing your code so that your object references are more tightly managed so it's never a question.

Access Violations shouldn't be thought of as an enemy. They're bugs: they mean you made a mistake that needs fixed. (Or that there's a bug in some code you're relying on, but I find most often that I'm the one who screwed up, especially when dealing with the RTL, VCL, or Win32 API.)

Phenice

FreeAndNil is effective when you have a single pointer reference that "owns" the object, but it won't help when you have multiple references to an object. FreeAndNil will set the given pointer to nil, but any other references to the object being freed will become "dangling pointers" just as with normal deallocation. – Salem

Answer 2

It is sometimes possible to detect when the address a pointer points to resides in a memory block that is on the heap's list of freed memory blocks. However, this requires comparing the pointer to potentially every block in the heap's free list which could contain thousands of blocks. So, this is potentially a computationally intensive operation and something you would not want to do frequently except perhaps in a severe diagnostic mode.

This technique only works while the memory block that the pointer used to point to continues to sit in the heap free list. As new objects are allocated from the heap, it is likely that the freed memory block will be removed from the heap free list and put back into active play as the home of a new, different object. The original dangling pointer still points to the same address, but the object living at that address has changed. If the newly allocated object is of the same (or compatible) type as the original object now freed, there is practically no way to know that the pointer originated as a reference to the previous object. In fact, in this very special and rare situation, the dangling pointer will actually work perfectly well. The only observable problem might be if someone notices that the data has changed out from under the pointer unexpectedly.

Unless you are allocating and freeing the same object types over and over again in rapid succession, chances are slim that the new object allocated from that freed memory block will be the same type as the original. When the types of the original and the new object are different, you have a chance of figuring out that the content has changed out from under the pointer. However, to do that you need a way to know the type of the original object that the pointer referred to. In many situations in native compiled applications, the type of the pointer variable itself is not retained at runtime. A pointer is a pointer as far as the CPU is concerned - the hardware knows very little of data types. In a severe diagnostic mode it's conceivable that you could build a lookup table to associate every pointer variable with the type allocated and assigned to it, but this is an enormous task.

That's why Assigned() is not an assertion that the pointer is valid. It just tests that the pointer is not nil.

Why did Borland create the Assigned() function to begin with? To further hide pointerisms from novice and occasional programmers. Function calls are easier to read and understand than pointer operations.

Salem

IIRC, Assigned came to existence to resolve a different issue, when dealing with Event pointers, you really have one variable containing two pointers (one pointing to the method code and other pointing to a object instance of that class), so it's not enough to check for nil, and Assigned do the work in this case. Of course, you @Salem sure knows that better than me, so please correct me if I'm wrong. – Recorder

Yes, Assigned() also took care of testing for null event pointers. – Salem

Allen Bauer blogged about that exact topic: Assigned or not Assigned, that is the question... – Alcina

Answer 3

The bottom line is that you should not be attempting to detect dangling pointers in code. If you are going to refer to pointers after they have been freed, set the pointer to nil when you free it. But the best approach is not to refer to pointers after they have been freed.

So, how do you avoid referring to pointers after they have been freed? There are a couple of common idioms that get you a long way.

• Create objects in a constructor and destroy them in the destructor. Then you simply cannot refer to the pointer before creation or after destruction.
• Use a local variable pointer that is created at the beginning of the function and destroyed as the last act of the function.

One thing I would strongly recommend is to avoid writing if Assigned() tests into your code unless it is expected behaviour that the pointer may not be created. Your code will become hard to read and you will also lose track of whether the pointer being nil is to be expected or is a bug.

Of course we all do make mistakes and leave dangling pointers. Using FreeAndNil is one cheap way to ensure that dangling pointer access is detected. A more effective method is to use FastMM in full debug mode. I cannot recommend this highly enough. If you are not using this wonderful tool, you should start doing so ASAP.

If you find yourself struggling with dangling pointers and you find it hard to work out why then you probably need to refactor the code to fit into one of the two idioms above.

You can draw a parallel with array indexing errors. My advice is not to check in code for validity of index. Instead use range checking and let the tools do the work and keep the code clean. The exception to this is where the input comes from outside your program, e.g. user input.

My parting shot: only ever write if Assigned if it is normal behaviour for the pointer to be nil.

Iver

Answer 4

Use a memory manager, such as FastMM, that provides debugging support, in particular to fill a block of freed memory with a given byte pattern. You can then dereference the pointer to see if it points at a memory block that starts with the byte pattern, or you can let the code run normallly ad raise an AV if it tries to access a freed memory block through a dangling pointer. The AV's reported memory address will usually be either exactly as, or close to, the byte pattern.

Alcina

I love that feature of FastMM4. It's worth noting that this pattern filling of the memory when freeing an object only happens when running in FullDebugMode. – Bacciferous

I guess that would be handy if I were debugging something, but my goal is to have this check inside the final application. – Lanie

Answer 5

Nothing can find a dangling (once valid but then not) pointer. It's your responsibility to either make sure it's set to nil when you free it's content, or to limit the scope of the pointer variable to only be available within the scope it's valid. (The second is the better solution whenever possible.)

Presumption

Answer 6

The core point is that the way how objects are implemented in Delphi has some built-in design drawbacks:

• there is no distinction between an object and a reference to an object. For "normal" variables, say a scalar (like int) or a record, these two use cases can be well told apart - there's either a type Integer or TSomeRec, or a type like PInteger = ^Integer or PSomeRec = ^TSomeRec, which are different types. This may sound like a neglectable technicality, but it isn't: a SomeRec: TSomeRec denotes "this scope is the original owner of that record and controls its lifecycle", while SomeRec: PSomeRec tells "this scope uses a transient reference to some data, but has no control over the record's lifecycle. So, as dumb it may sound, for objects there's virtually no one who has denotedly control over other objects' lifecycles. The result is - surprise - that the lifecycle state of objects may in certain situations be unclear.
• an object reference is just a simple pointer. Basically, that's ok, but the problem is that there's sure a lot of code out there which treats object references as if they were a 32bit or 64bit integer number. So if e.g. Embarcadero wanted to change the implementation of an object reference (and make it not a simple pointer any more), they would break a lot of code.

But if Embarcadero wanted to eliminate dangling object pointers, they would have to redesign Delphi object references:

• when an object is freed, all references to it must be freed, too. This is only possible by double-linking both, i.e. the object instance must carry a list with all of the references to it, that is, all memory addresses where such pointers are (on the lowest level). Upon destruction, that list is traversed, and all those pointers are set to nil
• a little more comfortable solution were that the "one" holding such a reference can register a callback to get informed when a referenced object is destroyed. In code: when I have a reference FSomeObject: TSomeObject I would want to be able to write in e.g. SetSomeObject: FSomeObject.OnDestruction := Self.HandleDestructionOfSomeObject. But then FSomeObject can't be a pointer; instead, it would have to be at least an (advanced) record type

Of course I can implement all that by myself, but that is tedious, and isn't it something that should be addressed by the language itself? They also managed to implement for x in ...

Jailbird