Calling -retainCount Considered Harmful

Question

Or, Why I Didn't Use retainCount On My Summer Vacation

This post is intended to solicit detailed write-ups about the whys and wherefores of that infamous method, retainCount, in order to consolidate the relevant information floating around SO.*

1. The basics: What are the official reasons to not use retainCount? Is there ever any situation at all when it might be useful? What should be done instead?** Feel free to editorialize.

2. Historical/explanatory: Why does Apple provide this method in the NSObject protocol if it's not intended to be used? Does Apple's code rely on retainCount for some purpose? If so, why isn't it hidden away somewhere?

3. For deeper understanding: What are the reasons that an object may have a different retain count than would be assumed from user code? Can you give any examples*** of standard procedures that framework code might use which cause such a difference? Are there any known cases where the retain count is always different than what a new user might expect?

4. Anything else you think is worth metioning about retainCount?

---

_{* Coders who are new to Objective-C and Cocoa often grapple with, or at least misunderstand, the reference-counting scheme. Tutorial explanations may mention retain counts, which (according to these explanations) go up by one when you call retain, alloc, copy, etc., and down by one when you call release (and at some point in the future when you call autorelease).}

_{A budding Cocoa hacker, Kris, could thus quite easily get the idea that checking an object's retain count would be useful in resolving some memory issues, and, lo and behold, there's a method available on every object called retainCount! Kris calls retainCount on a couple of objects, and this one is too high, and that one's too low, and what the heck is going on?! So Kris makes a post on SO, "What's wrong with my memory management?" and then a swarm of <bold>, <large> letters descend saying "Don't do that! You can't rely on the results.", which is well and good, but our intrepid coder may want a deeper explanation.}

_{I'm hoping that this will turn into an FAQ, a page of good informational essays/lectures from any of our experts who are inclined to write one, that new Cocoa-heads can be pointed to when they wonder about retainCount.}

_{** I don't want to make this too broad, but specific tips from experience or the docs on verifying/debugging retain and release pairings may be appropriate here.}

_{***In dummy code; obviously the general public don't have access to Apple's actual code.}

Ichneumon

I found this fairly recent question: https://mcmap.net/q/193328/-when-to-use-retaincount and Dave DeLong's very useful answer, but I am, as I said, hoping to create a central location for retainCount info (and learn some stuff myself!), specifically with some discussion of the reason for retainCount's existence and examples of its uselessness. It should go without saying, but if you think this is an un-useful dupe, please vote to close and I will delete it! – Ichneumon

Because of this: 'You might override this method in a class to implement your own reference-counting scheme. ' – Quadruplicate

+1 for the Dijkstra reference. – Orlan

@Stefan: That's a great point. I hope you'll consider expanding that into an answer later (however brief). – Ichneumon

@Bavarious: Thanks. It seemed like the obvious move, though. – Ichneumon

@Josh Caswell: Hm, probably not. A Guess: If we work for Apple and we definitely know, when it's absolutely safe to trust the retainCount, we could implement our own GC ref-counting scheme. Since we probably don't know, who else maintains an autorelease pool, we probably can't. Hm... – Quadruplicate

Let's say I'm writing a library and I want to unit test that a particular function does or does not retain its argument. Wouldn't it make sense to use retainCount in the unit test? – Tranquilizer

Yes but only if, during the test, the object passes solely through code that you have access to. If it goes through any Cocoa (or other opaque framework) calls, then you cannot know what the absolute retain count "should be". – Ichneumon

This seems more like a comment than an answer, especially with the question at the end. Can you expand your post? – Ichneumon

Please have a look at this link also -stackoverflow.com/a/4636477 – Reddin

Good foresight asking for the history of why retainCount exists in the first place. – Adamant

Answer 1

The basics: What are the official reasons to not use retainCount?

Autorelease management is the most obvious -- you have no way to be sure how many of the references represented by the retainCount are in a local or external (on a secondary thread, or in another thread's local pool) autorelease pool.

Also, some people have trouble with leaks, and at a higher level reference counting and how autorelease pools work at fundamental levels. They will write a program without (much) regard to proper reference counting, or without learning ref counting properly. This makes their program very difficult to debug, test, and improve -- it's also a very time consuming rectification.

The reason for discouraging its use (at the client level) is twofold:

1. The value may vary for so many reasons. Threading alone is reason enough to never trust it.

2. You still have to implement correct reference counting. retainCount will never save you from imbalanced reference counting.

Is there ever any situation at all when it might be useful?

You could in fact use it in a meaningful way if you wrote your own allocators or reference counting scheme, or if your object lived on one thread and you had access to any and all autorelease pools it could exist in. This also implies you would not share it with any external APIs. The easy way to simulate this is to create a program with one thread, zero autorelease pools, and do your reference counting the 'normal' way. It's unlikely that you'll ever need to solve this problem/write this program for anything other than "academic" reasons.

As a debugging aid: you could use it to verify that the retain count is not unusually high. If you take this approach, be mindful of the implementation variances (some are cited in this post), and don't rely on it. Don't even commit the tests to your SCM repository.

This may be a useful diagnostic in extremely rare circumstances. It can be used to detect:

• Over-retaining: An allocation with a positive imbalance in retain count would not show up as a leak if the allocation is reachable by your program.

• An object which is referenced by many other objects: One illustration of this problem is a (mutable) shared resource or collection which operates in a multithreaded context - frequent access or changes to this resource/collection can introduce a significant bottleneck in your program's execution.

• Autorelease levels: Autoreleasing, autorelease pools, and retain/autorelease cycles all come with a cost. If you need to minimize or reduce memory use and/or growth, you could use this approach to detect excessive cases.

From commentary with Bavarious (below): a high value may also indicate an invalidated allocation (dealloc'd instance). This is completely an implementation detail, and again, not usable in production code. Messaging this allocation would result in a error when zombies are enabled.

What should be done instead?

If you're not responsible for returning the memory at self (that is, you did not write an allocator), leave it alone - it is useless.

You have to learn proper reference counting.

For a better understanding of release and autorelease usage, set up some breakpoints and understand how they are used, in what cases, etc. You'll still have to learn to use reference counting correctly, but this can aid your understanding of why it's useless.

Even simpler: use Instruments to track allocs and ref counts, then analyze the ref counting and callstacks of several objects in an active program.

Historical/explanatory: Why does Apple provide this method in the NSObject protocol if it's not intended to be used? Does Apple's code rely on retainCount for some purpose? If so, why isn't it hidden away somewhere?

We can assume that it is public for two primary reasons:

1. Reference counting proper in managed environments. It's fine for the allocators to use retainCount -- really. It's a very simple concept. When -[NSObject release] is called, the ref counter (unless overridden) may be called, and the object can be deallocated if retainCount is 0 (after calling dealloc). This is all fine at the allocator level. Allocators and zones are (largely) abstracted so... this makes the result meaningless for ordinary clients. See commentary with bbum (below) for details on why retainCount cannot be equal to 0 at the client level, object deallocation, deallocation sequences, and more.

2. To make it available to subclassers who want a custom behavior, and because the other reference counting methods are public. It may be handy in a few cases, but it's typically used for the wrong reasons (e.g. immortal singletons). If you need your own reference counting scheme, then this family may be worth overriding.

For deeper understanding: What are the reasons that an object may have a different retain count than would be assumed from user code? Can you give any examples*** of standard procedures that framework code might use which cause such a difference? Are there any known cases where the retain count is always different than what a new user might expect?

Again, a custom reference counting schemes and immortal objects. NSCFString literals fall into the latter category:

NSLog(@"%qu", [@"MyString" retainCount]); 
// Logs: 1152921504606846975

Anything else you think is worth mentioning about retainCount?

It's useless as a debugging aid. Learn to use leak and zombie analyses, and use them often -- even after you have a handle on reference counting.

---

Update: bbum has posted an article entitled retainCount is useless. The article contains a thorough discussion of why -retainCount isn’t useful in the vast majority of cases.

Wigan

One note: -retainCount never gets to zero. – Orlan

@Bavarious if so, that's an implementation detail of the allocator. can you provide a reference? (the documentation of retain implies that it does reach 0, although the allocator/implementation could test for 1 before decrementing the retain count...) – Wigan

Yeah, I agree it’s an implementation detail. But Apple’s runtime does that, and it can be shown empirically (+alloc, -release, -retainCount) or by inspecting the runtime source code — which I haven’t yet. :-) – Orlan

Thanks. This especially is a good insight: "Threading alone is reason enough to never trust it." – Ichneumon

@Bavarious haha - yeah, i see it now. one example: when deallocated (on 64 bit), they return a NSUInteger which has the first 60 bits of the result set (whatever that represents internally...). – Wigan

Not returning zero is only an implementation detail in that to return zero would require making it sensible to message a deallocated object (which would likely also require never allocating another object at the same address). – Limburger

@Limburger preface: i know you're one of the experts in this domain, but something's not clicking for me regarding your previous statement. i've written reference counters and allocators (granted, using and for c++ programs), but i'm not seeing why retainCount can never be 0, why it is a requisite, or why objects would never be freed using this model. the rationale for this statement always brings me back to the conclusion that these requirements (specifically, that the rc must never be 0) are implementation details of a specific ref counting scheme, (cont) – Wigan

(cont) but not as a constraint which applies to every ref counting scheme. naturally, i would agree that messaging an object that effectively deallocated (either with a rc of 0, or has been physically freed) is considered a programmer's error, and that UB should be the expectation in this case. the program can also be structured that the object is messaged prior to deallocation, and that the rc could be 0 prior to deallocation. (cont) – Wigan

an implementation could take the (general) form: - (oneway void)release { ocallocator_dec_ref_count(self); } and: void ocallocator_dec_ref_count(id Self) { ocallocator_do_dec_ref_count(Self); if (0 == [Self retainCount]) { ocallocator_dispose_object_or_prepare_for_reuse(Self); } } void ocallocator_dispose_object_or_prepare_for_reuse(id Self) { [Self dealloc]; /* now it's an error to message Self */ ocallocator_reset_object(Self); if (ocallocator_needs_more_allocations_in_pool()) { ocallocator_add_to_reuse_pool(Self); } else { ocallocator_free(Self); } } – Wigan

By definition, when the retain count transitions to zero, the allocation backing the object is deallocated. Any subsequent messages to that object (including -retainCount) are undefined in behavior. To fix that, you have to make post-deallocation messaging valid. Ergo, you can never re-use any address for a new object because that breaks said contract (unless you also add some requirement that each allocation have an associated UUID). – Limburger

You could certainly go down that path, but doing so completely removes you from the reality that is Cocoa's/iOS's runtime. Yet, it is also fully supported. You can define your own root class if you want and you could invent whatever allocation patterns you desire. Just don't pass your objects into framework API! (It is actually a very interesting mental exercise -- I've created a handful of root classes over the years to explore different models.) – Limburger

@Limburger we have headed down separate paths here, and we're taking two separate perspectives. i wasn't attempting to illustrate (or support) post-deallocation messaging. instead, i was illustrating how if (0 == [Self retainCount]) can be valid and meaningful by the allocation and refcounting implementation (alone) -- and how this may be used before [Self dealloc] is called. to a client, it is provoking consequences if a problem exists. (cont) – Wigan

i figure your comment originated in response to what i'd written in 'unusual debugging uses': "if it's zero, the object is dead/deallocated, or simply an invalid address". that is to say: assert([arg retainCount]) can be used to prove multiple conditions in a simple statement: 1) arg is not nil 2) arg is a valid, living object, and not a zombie 3) arg points to a valid address. if there is a ref counting issue (negative imbalance), the approach can be used to help identify the problem. – Wigan

If by "separate paths" you mean you are no longer talking about Cocoa/iOS, sure.... if you are talking about Cocoa/iOS, then -- no -- by the very definition of the behavior of the runtime, there is no way that retainCount can ever return zero. At the moment in time that the retain count would transition to 0, the object is no longer valid [under the iOS/Cocoa runtime]. There is no assertion that can be made with assert([arg retainCount]) that cannot also be made by assert(arg) [again, unless you are talking about something other than Cocoa/iOS]. – Limburger

If your claim is that release could be implemented as rc--; if ([self retainCount] == 0) [self dealloc]; then, yes, it could be done that way... but that'd also still mean that retainCount can never return zero outside of the implementation of that implementation of release. – Limburger

@Limburger "If your claim is that release could be implemented as rc--;..." << exactly – Wigan

@Limburger thanks for the explanations. i've demonstrated the unusual debugging uses for the statement: assert([arg retainCount]); above. it's also a cw now. – Wigan

None of your assertions glean any information from the execution of retainCount itself. (1) No message dispatched; objc_msgSend() short-circuits. (2) & (3) No message dispatched; the NSZombie isa short-circuits everything & (3) crashes in objc_msgSend() (hopefully -- behavior is actually undefined). In all three cases, using any other instance method would give you exactly the same resolution of information. – Limburger

I realize it's a silly/unusual means to test multiple conditions. Q: Is there ever any situation at all when it might be useful? A: ...there are a few unusual debugging uses... – Wigan

... but you haven't shown an assertion where retainCount provides more signal than using a more mainstream method or where retainCount's return value is even meaningful. – Limburger

i have the feeling that our exchange has exceeded my intention, and that this would probably be a good time to put this topic to rest. what resolution would you like? would you like me to delete 'unusual debugging use #1' so there is no confusion? – Wigan

Certainly, deleting or clarifying anything that is factually inaccurate will be helpful to whoever stumbles upon this in the future. – Limburger

Answer 2

The general rule of thumb is if you're using this method, you better be damn sure you know what you're doing. If you are using it for debugging a memory leak you're doing it wrong, if you're doing it to see what is going on with an object, you're doing it wrong.

There is one case where I have used it, and found it useful. That is in doing a shared object cache where I wanted to flush the object when nothing had a reference to it anymore. In this situation I waited until the retainCount is equal to 1, and then I can release it knowing that nothing else is holding onto it, this will obviously not work properly in garbage collected environments and there are better ways to do it. But this is still the only 'valid' use case I've seen for it, and isn't something a lot of people will be doing.

Kugler

The use-case you described is a very bad example - even when creating a cache, you should not use retainCount in any way. Cache should not care if there are other users of any object it contains - it should simply release stale objects when it feels like doing so - if there is someone else holding on them, so be it. – Upside

Michal, the point was to evict stale object as soon as possible. For anything going forward I'd use NSCache for this purpose, but at the time this didn't exist. As I said though, I would never recommend that course of action upon anyone. – Kugler

Using retain count for cache is wrong. Cache should never care if someone else is still using cached object, the cache should evict it from itself when an object was not requested recently. Using retainCount there to cache an object for longer and "the point was to evict stale object as soon as possible" are two different things. – Upside

I'm not disagreeing that is is bad to do in practice, but its still answering the question for cases where retainCount can be used, not where it should be used. Which I think we can all agree is nowhere. – Kugler

@Michal: I appreciate your concern about use of this technique in the wild, but this answer (along with your comments) seem to me to be a useful discussion around retainCount. Therefore I have upvoted the answer (which takes it back to 0). – Ichneumon

@Michal: as long as the object is in active use (signified by the retain count of >1), evicting it from the cache doesn't free any memory. Since there is a chance that the object will be used again, it is sensible to hold on to the reference. If the object is removed from the cache, then re-cached before the previous instance is released by whomever was holding on to it, you'll have a superfluous instance floating around. – Kokoschka

@jfortmann, I'm glad someone sees my though process behind that caching. Still wouldn't recommend this in particular, as it is a bit odd. – Kugler

There is no need to call retainCount in your shared cache example; if your cache holds a retain and wants to evict the object, it releases it. That something else holds into it is hopefully irrelevant. If you need finer grained object population management, look elsewhere (above low level retain counting). – Limburger