How can I maintain a weak reference on a COM object in C++?

Question

In my application, I'm hooking various functions for creating COM objects (such as CoCreateInstanceEx) to get notified whenever some object is created. I'm keeping track of all created objects in a std::list and I'm iterating over that list to do various things (like checking which OLE objects have been activated).

The issue with this is that right now, whenever adding an IUnknown pointer to my list, I call IUnknown::AddRef on it to make sure that it doesn't get destroyed while I'm tracking it. That's not what I really want though; the lifetime of the object should be as long (or short) as it is without my tracing code, so I'd rather like to maintain a weak reference on the objects. Whenever the last reference to some tracked COM object is removed (and thus the object gets destroyed), I'd like to get notified so that I can update my bookkeeping (e.g. by setting the pointer in my list to NULL).*

What's the best way to do this? Right now, I'm patching the (first) VTable of all created objects so that the calls to IUnknown::Release via the first vtable get notified. However, this won't work for COM interfaces which inherit from multiple interfaces (and thus have multiple vtables), but I'm not sure whether this is really a problem: given the Rules for Implementing QueryInterface, there should always be just one IUnknown returned by IUnknown::QueryInterface, right? So I could do that and then patch that vtable.

Furthermore, this approach is also a bit hairy since it involves creating thunks which generate some code. I only implemented this for 32bit so far. Not a big issue, but still.

I'm really wondering whether there isn't a more elegant way to have a weak reference to a COM object. Does anybody know?

*: The next thing I'll have to solve is making this work correctly in case I have active iterators (I'm using custom iterator objects) traversing the list of COM objects. I may need to keep track of the active iterators and once the last one finished, remove all null pointers from the list. Or something like that.

Vorlage

Have you taken a loot at the Running Object Table (ROT): msdn.microsoft.com/en-us/library/ms695276(VS.85).aspx (it has some notions of weak references). – Uria

Curious, what do you want to use this for? Some sort of debugging? – Plugboard

@Simon Mourier: I was aware of the ROT but I didn't know that it has (some) notion of weak references. Thanks for mentioning that, I'll have a look. – Vorlage

@BrendanMcK: Yes, exactly - I'm tracing the state of various components (including all COM objects) of this software for the sake of debugging. – Vorlage

Answer 1

This isn't an answer as much as a set of issues why this is a really tricky thing to do - I'm putting it in as an answer since there's too much information here than fits in a comment :)

My understanding is that the concept of weak reference just doesn't exist in COM, period. You've got reference counting via IUnknown, and that's the sum total of how COM deals with object lifetime management. Anything beyond that is, strictly speaking, not COM.

(.Net does support the concept, but it's got an actual GC-based memory manager to provide appropriate support, and can treat WeakRef objects differently than regular references in memory. But that's not the case with the very simple world that COM assumes, which is a world of plain memory and pointers, and little more.)

COM specifies that reference counting is per-interface; any COM object is free to do ref counting per object as a convenience, but the upshot is that if you're wrapping an object, you have to assume the most restrictive case. So you cannot assume that any given IUnknown will be used for all addrefs/releases on that object: you'd really need to track each interface separately.

The canonical IUnknown - the one you get back by QI'ing for IUnknown - could be any interface at all - even a dedicated IUnknown that is used only for the purpose of acting as an identity! - so long as the same binary pointer value is returned each time. All other interfaces could be implemented any way; typically the same value is returned each time, but a COM object could legitimately return a new IFoo each time someone QI's for IFoo. Or even keep around a cache of IFoos and return one at random.

...and then you've got aggregation to deal with - basically, COM doesn't have a strong concept of object at all, it's all about interfaces. Objects, in COM, are just a collection of interfaces that happen to share the same canonical IUnknown: they might be implemented as a single C/C++ object behind the scenes, or as a family of related C/C++ objects presenting a facade of a 'single COM object'.

---

Having said all of that, given that:

I'm tracing the state of various components (including all COM objects) of this software for the sake of debugging

Here's an alternate approach that might produce some useful data to debug with.

The idea here is that many implementations of COM objects will return the ref count as the return value to Release() - so if they return 0, then that's a clue that the interface may have been released.

This is not guaranteed, however: as MSDN states:

The method returns the new reference count. This value is intended to be used only for test purposes.

(emphasis added.)

But that's apparently what you're doing here.

So one thing you could do, assuming you own the calling code, is to replace calls with Release() with an inline called MyRelease() or similar that will call release, and if it notices that the return value is 0, then notes that the interface pointer is now possibly freed - removes it from a table, logs it to a file, etc.

One major caveat: keep in mind that COM does not have a concept of weak ref, even if you try to hack something together. Using a COM interface pointer that has not been AddRef()'d is illegal as far as COM is concerned; so if you save away interface pointer values in any sort of list, the only thing you should so with those is treat them as opaque numbers for debugging purposes (eg. log them to a file so you can correlate creates with destroys, or keep track of how many you have outstanding), but do not attempt to use them as actual interface pointers.

Again, keep in mind that nothing requires a COM object to follow the convention of returning the refcount; so be aware that you could see something that looks like a bug but is actually just an implementation of Release just happens to always returns 0 (or rand(), if you're especially unlucky!)

Plugboard

+1 for the insightful response. It makes sense to rather think of interfaces instead of objects (which makes it look reasonable to do refcounting per interface). I guess it's not even safe to assume that the IUnknown I get from QI'ing for IID_IUnknownwonder is the one used for subsequent QueryInterface calls. If it was, I could hook into that and return proxy interfaces all the time which detect Release calls via any supported interface. – Vorlage

Another thing I just noticed after reading your response. The 'Rules for Implementing QueryInterface' page I mentioned in my question says: Interface implementations must maintain a counter of outstanding pointer references to all the interfaces on a given object. You should use an unsigned integer for the counter.. Maybe gaining access to this counter would get me further? – Vorlage

The MyRelease()-based idea is actually what I'm doing right now. I'm calling AddRef() in the beginning and in my Release() hook function I check whether the new refcount is 1. If so, I know that I'm the only one holding a reference to the object, so I release that as well and log the object destruction. The main bug in this is that I'm not guaranteed to get notified of all Release() calls. – Vorlage

COM doesn't say anything about where the interface count lives - in many C++ objects, it's close to being the first variable - but that's a coincidence of C++ object layout, but it could be anywhere. And, as mentioned above, an object doesn't actually have to count on a per-interface basis, most C++ code counts at the object level. The important thing is that client code matches AddRef and Release per interface instance; but the object can track that anyhow and anywhere it wants to. – Plugboard

There may just be some limitations to this overall approach that you just have to live with. By tracking your calls to Release(), you may still be able to create a useful list of "potentially outstanding objects", which can still be useful in tracking down specific issues - it gives you a starting point. On the other hand, trying to make this into a perfect solution by tracking down every last Release() could end up creating a bunch of complex code that introduces subtle bugs and ends up taking longer to get right than any problem it might help you with. – Plugboard

Answer 2

First, you're right that QueryInterface for IUnknown should always return the same pointer; IUnknown is treated as the object's identity IIRC, so needs to be stable.

As for weak pointers, off the top of my head, maybe you could give CoMarshalInterThreadInterfaceInStream a whirl? It is meant to allow you to serialize a reference to a COM object into a stream, then create a new reference to the object on some other thread using the stream. However, if you serialise into a stream and retain the stream as a sort of weak pointer, then unmarshal later on to recover the pointer, you could check whether unmarshalling fails; If so, the object is gone.

Conto

AFAIK, CoMarshakInterThreadInterfaceInStream won't work: it essentially does a preemptive AddRef() on behalf of the receiving thread, so the stream itself ends up containing a ref. (May actually be more than one ref, that's an impl detail; one side effect is that you can't just abandon the stream data if you decide you don't want the object after all - you're responsible for unmarshaling and calling Release on the resulting object to clean it up properly one way or another.) – Plugboard

+1: I like the idea of relying on an error by some function in case a given COM object pointer is invalid. Maybe this approach could be used with some other function... – Vorlage

Oh well. Pity that isn't mentioned on the function's MSDN page. – Conto

@Conto - it's not mentioned explicitly, but it's implicit in how COM's lifetime memory management works. You've got IUnknown - and that's basically it. Suppose you give an interface pointer to a COM support function - how does it know if the object is still there or not? It has no way of knowing, it has to play by the same AddRef/Release rules as everyone else. – Plugboard

@Plugboard - Bingo, I should have put my thinking hat on. I vaguely recall now that the marshal/unmarshal pair creates a proxy/stub pair, and they don't need special knowledge of the original object. – Conto

@Conto - yup; the proxy/stub are usually created in response to the unmarhsaling (can be optimized away if unmarshalled in same apartment as source). Curiously, the reference between the proxy and stub is "weaker", due to the extra level of indirection. You can actually tear down a stub using CoDisconnectObject(pOriginalInterface), causing the stub to Release() the original interface, leaving a proxy that's pointing to an "empty" stub - which will then reject incoming calls with CO_E_DISCONNECTED. – Plugboard

Answer 3

With WinRT IWeakReference was added to enable weak refs to COM objects. Objects created with WRL's RuntimeClass support IWeakReference by default (can be disabled with an option).

you can use IWeakReference in your designs but it means you will need to use at least some WinRT concepts, IInspectable based interface.

Exenterate

Answer 4

I would recommend you to take a look at the https://github.com/forderud/MiniCOM repo that contains a SharedRef wrapper class for non-owning weak references through a IWeakRef interface. This is similar to IWeakReference, but is also compatible with classical IUnknown-based COM.

Claypoole