D Dynamic Arrays - RAII

Question

I admit I have no deep understanding of D at this point, my knowledge relies purely on what documentation I have read and the few examples I have tried.

In C++ you could rely on the RAII idiom to call the destructor of objects on exiting their local scope.

Can you in D?

I understand D is a garbage collected language, and that it also supports RAII. Why does the following code not cleanup the memory as it leaves a scope then?

import std.stdio;

void main() {
      {
            const int len = 1000 * 1000 * 256; // ~1GiB

            int[] arr;
            arr.length = len;
            arr[] = 99;
      }

      while (true) {}
}

The infinite loop is there so as to keep the program open to make residual memory allocations easy visible.

A comparison of a equivalent same program in C++ is shown below.

It can be seen that C++ immediately cleaned up the memory after allocation (the refresh rate makes it appear as if less memory was allocated), whereas D kept it even though it had left scope.

Therefore, when does the GC cleanup?

Alegar

Answer 1

No, you cannot assume that the garbage collector will collect your object at any point in time.

There is, however, a delete keyword (as well as a scope keyword) that can delete an object deterministically.

scope is used like:

{
    scope auto obj = new int[5];
    //....
} //obj cleaned up here

and delete is used like in C++ (there's no [] notation for delete).

There are some gotcha's, though:

• It doesn't always work properly (I hear it doesn't work well with arrays)

• The developers of D (e.g. Andrei) are intending to remove them in later versions, because it can obviously mess up things if used incorrectly. (I personally hate this, given that it's so easy to screw things up anyway, but they're sticking with removing it, and I don't think people can convince them otherwise although I'd love it if that was the case.)

• In its place, there is already a clear method that you can use, like arr.clear(); however, I'm not quite sure what it exactly does yet myself, but you could look at the source code in object.d in the D runtime if you're interested.

---

As to your amazement: I'm glad you're amazed, but it shouldn't be really surprising considering that they're both native code. :-)

Woodcutter

If I cannot assume the GC will cleanup unused memory, how can I assume my program won't run out of memory straight away? For example, if I packed in two of the 2.25GB blocks (different scopes) of my vector3b dynamic array, I ran out of memory instantly, even though the GC should have recognized the first as unused (per RAII)? – Alegar

Also, why does the same problem occur for non-dynamic arrays, should they not follow RAII? Even though, due to the size, they may be Heap allocated. – Alegar

@Daniel: No I mean, you can't assume it will happen at any particular time; it'll happen sometime, but you don't know when. Also, it depends on what you mean by non-dynamic arrays. What you're seeing right now are not arrays, but they're slices (which are just a length-pointer pair). The arrays themselves are in heap memory, and your slices behave just like any other variables -- they don't have a destructor, but if they did, it would run when they were deallocated. So if you use the Array type, it'll destroy itself because it has a destructor. – Woodcutter

@Daniel: See this page for an explanation on how slices work. – Woodcutter

That helped a little bit, but I'm still unsure as to how I am to be able to allocate to large blocks (2.5GB) of memory, automatically deallocate one, then allocate another, without running out of memory. And whether or not that is just simply not plausible with slices, which begs the question, are slices ever safe to use if you must rely on the GC to "at some point" clean up the memory? – Alegar

@Daniel: If you need a lot of memory like that, I suggest you try this: Try doing what you were already doing, and using either scope or delete; if it didn't work, use the GC's malloc and free instead, and just slice those up and work with them like regular slices. Only the management will differ that way. (I'm not sure I get your last question though.) – Woodcutter

OK, I got the right behavior when I put the slices in different scopes in terms of functions, it appears D doesn't readily differentiate the 'blocks' I was using... It was keeping both, as the scope wasn't "different" or some such. With a completely seperate scope, it would not re-allocate, just re-use the memory, which meant it was incredibly fast compared to its C++ comparison, which allocated, deallocated, the reallocated (2.5GB each time! Not cheap :)). I know this is all bad practice, and you would avoid that in C++, but I'm just trying to feel the back end of D, like you can with C++. – Alegar

In short, don't use ?anonymous? scopes in D :P – Alegar

@Daniel: Oh I see what you mean, yeah, I hadn't noticed it before, interesting. :) – Woodcutter

Both scope and delete are going to be deprecated in D2. – January

I'm loving D... I'm almost scared I'm going to run into something that is going to stop me using it. Every turn I make I'm finding a fix for something I disliked in C++ or C. Are lack of interfacing to C/C++ libraries major (especially if I need to link to some eventually)? – Alegar

Answer 2

scope declarations are going in D2, so I'm not terribly certain on the semantics, but what I'd imagine is happening is that scope T[] a; only allocates the array struct on the stack (which needless to say, already happens, regardless of scope). As they are going, don't use scope (using scope(exit) and friends is different -- keep using them).

Dynamic arrays always use the GC to allocate their memory -- there's no getting around that. If you want something more deterministic, using std.container.Array would be the simplest manner, as I think you could pretty much drop it in where your scope vector3b array is:

Array!vector3b array

Just don't bother setting the length to zero -- the memory will be free'd once it goes out of scope (Array uses malloc/free from libc under the hood).

Dulcine