Soft (not: weak) references in C++ - Is it possible? Is there an implementation?

Question

In C++ I'm using boost::shared_ptr and boost::weak_ptr to automatically delete objects that are no longer needed. I know these work with reference counting.

In Java, memory is managed by a garbage collector, which consideres the built-in object references as strong, WeakReference as weak and SoftReference as something in between (may be collected by the GC, but may as well survive the GC), which is really handy for caching objects for some time, but throwing them away as soon as free memory is getting low.

So now I'm back in C++ and I miss the comfort of having soft references. I wonder if soft referencing is practicable with reference counting at all. When the last strong reference to an object is cleared, and there remains a soft reference, when would it be deleted after all? I could think of some schemes, but none of them seem clever to me.

Just in case there are proper semantics for soft references along with reference counting, I wonder if this has already been implemented, maybe in a way that's even compatible with boost::shared_ptr (or the C++ TR1 equivalent std::shared_ptr for that matter).

If the answer to both questions is no, what are the alternatives in an object caching scenario?

EDIT: Of course I'm talking of a situation when caching is actually useful, because the objects are costly to construct (think of several access to a database and queries of a network), yet there are too many to keep them all forever.

Bladder

What on earth is a soft reference when it's at home? – Zugzwang

@DeadMG: en.wikipedia.org/wiki/Soft_reference – Taper

@DeadMG: What do you mean? I'm not a native english speaker, so if this comment involves some kind of humor, I guess I just don't get it. If it's a real question, well, I don't get it neither. – Bladder

I don't get it either -- it must be some kind of reference. – Diegodiehard

A soft reference is just a grade of weak reference. Like the link I pointed out shows. In the JVM the memory collection algorithms will first try and collect garbage. Failing that they collect weak references. Failing that they collect soft references. Failing that they puke. – Taper

"When it's at home" is a phrase meaning...er...not much really. "What on earth is a soft reference when it's at home?" means "What (on earth) is a soft reference?" – Lowgrade

@Brian Hooper: Thanks, that was exactly the information I was missing :) – Bladder

@JUST: If you were replying to my comment, it was a joke, albeit a weak one. – Diegodiehard

@Amnon: It was a soft one, not a weak one. – Taper

Sorry about the downvote, I negated it with an upvote. I was clicking randomly on the screen when I hit the down arrow :-( – Burny

@Burny No problem. You can normally take back a downvote by clicking the down arrow again, so your vote becomes neutral again. I can see that there is currently no downvote on this question (you need 1000 reputation to Show total up and down vote counts). – Bladder

Answer 1

As others pointed out, you can find referenced counted pointers (and their attendant weak counterparts) in the Boost library, but what's missing from the soft reference idea is some awareness of the runtime environment's memory constraints. In Java, for example, a SoftReference isn't materially different from a WeakReference in its capabilities; rather, it's the contract for how the runtime will preserve or evict the two kinds of references in the face of memory pressure that differs.

In order to mimic this behavior in C++, you'd have to build a memory-aware reference cache that held strong references on objects that the rest of your application would hold weakly. When the cache determined that the application was scratching its memory usage ceiling — or any other constraining criteria — it would release the strong references, surrendering the objects for "collection" (reaching the zero reference count) and allowing the weak references in use to later detect invalidation.

Vidovic

Let me ask... Isn't that somewhat what a garbage collector does? Reclaim memory when the programs need it? – Mendelism

Yes, a garbage collector can make memory previously allocated to no-longer-reachable objects available for new objects, either eagerly or only upon demand. The question wasn't about how to integrate a garbage collector, but rather if one could emulate the pressure awareness of a collector. Using strong and weak pointers in C++ affords one kind of garbage collection, but there's no way to express a desire to "pin an object only until we need more memory." Strong pointers pin unconditionally, and weak pointers don't pin at all. Soft pointers exist in between those extremes. – Vidovic

Answer 2

If you really want to replicate this behavior you may use a garbage collector (like this: http://www.hpl.hp.com/personal/Hans_Boehm/gc/) and use it to take care of your object or a subset of them, where using SoftReferences would be useful.

But I'd prefer to go for a solution more native to C++ than replicating Java bahavior - but nothing stops you of doing that.

Telestich

Answer 3

You can implement your own LRU cache, and a new smart_pointer associated with such a cache. I don't think such a structure exists in Boost or standard C++ (off the top of my head anyway). If you are doing a web application or something... you can use libmemcached, which is the C Interface to memcached.

I find it hard to think of a situation where such an object would be so costly to construct / destroy... while it would be cheap to reinitialize... that a LRU cache would become useful. But if you really need one, you have the tools to actually build it.

Burny

Answer 4

You can move your soft-referenced data outside your application into the OS using something like buffcacher.

I know of no library offering this, I've only ever rolled my own.

Its so fast and fair that it gets useful to cache the validation of 'secure cookies' in webservers and other tasks that seem almost small for a conventional cache.

Inconformity

Answer 5

No, there is no such thing in C++. Nor should there be. Every object serves an important purpose. If it didn't, why do you still have it? Keeping objects around in this way is a memory leak. If you need an object, you need it. If you don't, destroy it. There is no in-between between useful and useless, either it serves a purpose or it doesn't.

If you're desperate, it is not impossible to write your own garbage collector and implement such a thing yourself. But I'd recommend against needing them or using them at all.

Edit: In object caching, people normally use LRU caches. When you have a cache miss, the reference to the least recently used object is destroyed (if the cache is full), the new object is created and put in as the most recently used, and all the others are moved down. However, you typically need to be retrieving items from disk before you actually need a caching strategy in C++. The cost of creating most objects is simply not that great.

Zugzwang

I'm not sure I see how a soft reference, as Brian describes it, is a memory leak while a LRU cache is not. Either you need the object, or you don't. – Vestigial

In resonse to your edit I edited my answer, too. I forgot to mention that may objects are rather coslty to create. The least-recently-used strategy is actually a good idea. Anyway, I'm asking myself if I need to put that manually into the implementation of a cache-mananging object, or if there is similar functionality which can be put into something like a weak reference. – Bladder

@Dennis: An LRU has a fixed size. It can be cleaned. It's controlled by the owning object. Soft references are uncontrolled. If you implement an LRU, then you have control over that memory. If you use soft references, anyone could use a soft reference. LRU caches are clearly defined and owned. Soft references are a gaggle of everybody. – Zugzwang

@Brian: the language itself has no similar functionality. The only reason it exists in Java is that you already have a garbage collector it can piggy-back on. without a GC, someone else has to make the decision of when to destroy the referenced object. And then it might as well be implemented in a third-party library. There's no single "right answer" which could be incorporated in the language itself. – Tiptoe

"Gaggle of everybody" = this object is worth having if there's space for it, not if there's not. "LRU cache" = I will use exactly 1GB of memory, and I don't care if that means your other apps won't run. On a desktop OS, running out of memory is a rare issue and doesn't require much technique on the part of apps. "If there's space" is basically always true. On constrained systems it can be really important that apps co-operate over memory use, and soft references are one (relatively blunt) tool for doing so. – Conant

@Steve: I already expressed my opinion that "this object is worth having if there's space for it and not if there's not" is quite flawed. – Zugzwang

This is crazy. Using fixed-size cacehs is not as efficient as using dynamic caches that scale on outside resource factors such as available memory. – Inconformity