Overrelease issue with block-captured objects; retain count jumps straight from +2 to 0!

Question

I'm confused by an occasional crash that I'm seeing, which, according to the Zombies instrument, is caused by the over-release of some dictionary values. When I look at the object history for one of these overreleased objects in Instruments, I see that its retain count drops straight from +2 to 0 at one stage. (Take a look at the screenshots at the end of the post). It's not clear to me how this is even possible.

I should say that I only see this crash when profiling with Instruments, so I suppose it could be an Apple bug, but it's probably safer to assume that it's pilot error, which Instruments is merely exposing.

Anyways, I'm constructing a CFDictionary that contains some Core Foundation objects (CFStrings and CFNumbers), and I then cast this to a NSDictionary* and pass it to an Objective-C method. A simplified version of my code is below:

// creates a CFDictionary containing some CFStrings and CFNumbers
void doStuff() 
{
    CFDictionaryRef myDict = CreateMyDictionaryContainingCFTypes();

    dispatch_async(myQueue, ^{
        [someObject receiveDictionary:(NSDictionary*)myDict];
        CFRelease(myDict);  // this line causes a crash. The Zombies instrument
                            // claims a CFString value contained in this
                            // dictionary has already been freed.
    });
}

// ...

- (void)receiveDictionary:(NSDictionary*)dict
{
    NSAutoreleasePool *pool = [NSAutoreleasePool new];

    NSString* str1 = [dict objectForKey:@"key1"];
    NSString* str2 = [dict objectForKey:@"key2"];
    NSNumber* num1 = [dict objectForKey:@"key3"];

    dispatch_async(myOtherQueue, ^{
        [database executeUpdate:@"INSERT INTO blah (x,y,z) VALUES (?, ?, ?)", str1, str2, num1];
    });

    [pool drain];
}

I had thought that str1, str2 and num1 would be seen as Objective-C objects, and would therefore be captured and automatically retained when the block in -receiveDictionary: is copied by the dispatch_async call, and released when that block is released. Indeed, these variables do seem to be captured and retained by the block. However, examining the object history for an overreleased CFString in Instruments, I can see that its reference count is being incremented when the block is copied. Perplexingly, its retain count drops from +2 straight to 0 when a block is released (see the screenshot at the end of the post); I don't know how to tell from the stack trace which block this is. By the time CFRelease is called on the dictionary in the block in doStuff(), some of its values have been deallocated already, and the program crashes.

So where did the extra release call come from? How can an object's retain count drop straight from +2 to 0, as Instruments indicates?

On a whim, I forced the second block to retain the entire dictionary, like so:

dispatch_async(myOtherQueue, ^{
    [database executeUpdate:@"INSERT INTO blah (x,y,z) VALUES (?, ?, ?)", str1, str2, num1];
    [dict self];
});

This seems to make the crash disappear; instruments stops reporting zombies, at least. I cannot for the life of me see why this works, though; surely I just to ensure that the block retains the dictionary values that I'm interested in, not the entire dictionary. So what's going on?

---

Instruments lists the following object history for the zombie CFString, with the object's retain count. I've included screenshots for the interesting events.

#0 +1 CFString is created
#1 +2 CFString added to dictionary
#2 +1 CFString released
#3 +2 CFString is retained when the block in -receiveDictionary: is copied
#4 +0 What the...? The object's retain count dropped straight from +2 to 0!
#5 -1 CFDictionary is released, causing crash

Liebfraumilch

You need to show the contents of CreateMyDictionaryContainingCFTypes(). – Basilius

You are right about the fact that CoreFoundation objects aren't retained implicitly. – Bullfrog

What’s [NSAutoreleasePool new]? – Venous

@Venous +[NSObject new] is a method that returns the result of [[self alloc] init]. In other words, it's just a convenience method. It's most frequently used with NSAutoreleasePool, mostly just out of habit though. – Basilius

Why aren't you creating the dictionary inside the initial block? – Venial

Is CreateMyDictionaryContainingCFTypes creating an autorelease object? – Hyonhyoscine

Answer 1

Finally caught the bug — turned out it wasn't a zombie issue at all, but an unrelated memory corruption issue in a routine to decode base64 data. Nothing to do with retain/release, blocks or GCD. Sigh.

With hindsight, this should have been more obvious. The fact my program was crashing shortly after Instruments reported the overreleased object should have been a clue — if it was actually a zombie issue, you wouldn't have expected a crash. (I think?) The retain count jumping from +2 to 0 probably suggested something other than a simple overrelease as well.

So what have I learned?

• Don't copy-paste code without thoroughly checking it. All base64 conversion routines are not created equal. (Specifically, calling realloc without using its return value is wrong, wrong wrong! It's a pity the Static Analyser doesn't flag this.)
• Don't exclusively rely on Instruments—other tools such as Valgrind can be useful. In this case, Valgrind gave me more accurate information.

Liebfraumilch

Answer 2

What are you using as your CFDictionaryKeyCallBacks and CFDictionaryValueCallBacks when you create the dictionary in CreateMyDictionaryContainingCFTypes()? I can easily replicate this issue if I pass in NULL for both, but I'm not able to replicate it if I pass in &kCFTypeDictionaryKeyCallBacks and &kCFTypeDictionaryValueCallBacks.

Portent

Answer 3

A block when copied will implicitly retain any Objective-C object in it's scope, and then also implicitly release these objects when the block is released.

CFDictionaryRef is a toll-free bridge type for NSDictionary, and as far as blocks are concerned also Objective-C objects. This mean that you do not need to do any addition memory management.

Let me comment your code, and mark the evaluation order.

void doStuff() {
    // 1. myDict must have a retainCount of 1, you named your function Create
    //    and promised so according to Core Foundation men.rules.
    CFDictionaryRef myDict = CreateMyDictionaryContainingCFTypes();

    // 2. dispatch_async will copy your block and retain myDict, since it is in 
    //    scope of the block, myDict has a retainCount of 2
    dispatch_async(myQueue, ^{
        // 4. Block is execute some time later, myDict has a retainCount of 1.
        [someObject receiveDictionary:(NSDictionary*)myDict];

        // 5. Block is done and will be released, along with scoped objects
        //    on exit, retainCount reaches 0, and myDict is released.
    });

    // 3. Release your own copy before function ends, retainCount of 1
    CFRelease(myDict);  // this line causes a crash. The Zombies instrument
}

Groin

Actually, unless CoreFoundation objects are decorated with __attribute__((NSObject)), they are not implicitly retained by the block when it is copied. (See clang.llvm.org/docs/Block-ABI-Apple.txt) As written, that code crashes very quickly, because the dictionary is deallocated at the end of doStuff(). – Liebfraumilch