pimpl for a templated class

Question

I want to use the pimpl idiom to avoid having users of my library need our external dependencies (like boost, etc) however when my class is templated that seems to be impossible because the methods must be in the header. Is there something I can do instead?

Flattop

Answer 1

If the class is templated, your users essentially need to compile it (and this is literally true in the most widely-used C++ implementations) and so they need your external dependencies.

The simplest solution is to put the bulk of your class's implementation in a non-template base class (or encapsulated member object of some class). Solve the module-hiding problem there.

And then write the template derived (or enclosing) class to add type safety to it.

For example, suppose you have a template that provides the amazing ability to allocate on first access (omitting the necessary copy constructor, assignment, destructor):

template <class T>
class MyContainer
{
    T *instance_;

public:
    MyContainer() : instance_(0) {}

    T &access()
    {
        if (instance_ == 0)
            instance_ = new T();

        return *instance_;
    }
};

If you wanted the "logic" to be separated into a non-template base class, you'd have to parameterise the behaviour in the non-template way, which is to say, use virtual functions:

class MyBase
{
    void *instance_;

    virtual void *allocate() = 0;

public:
    MyBase() : instance_(0) {}

    void *access()
    {
        if (instance_ == 0)
            instance_ = allocate();

        return instance_;
    }
};

Then you can add the type-awareness in the outer layer:

template <class T>
class MyContainer : MyBase
{
    virtual void *allocate()
        { return new T(); }

public:
    T &access()
        { return *(reinterpret_cast<T *>(MyBase::access())); }
};

i.e. You use virtual functions to allow the template to "fill in" the type-dependent operations. Obviously this pattern would only really make sense if you have some business logic that is worth the effort of hiding.

Turntable

I think this approach could be also useful if you don't want your preprocessor definitions (#define), constants etc. to be visible. As a developer, I don't want to see implementation details of a class/library that I'm using, especially in auto-complete list. You may want to hide those even if your business logic is unworthy of hiding. – Lacewing

You can avoid virtual functions if you use CRTP. – Angelynanger

Not if you're trying to avoid using templates (the T in CRTP) because you want separately compiled modules. – Turntable

Answer 2

You can explicitly instantiate templates in the source file, but that is possible only if you know what the template type is going to be. Otherwise, do not use pimpl idiom for templates.

Something like this :

header.hpp :

#ifndef HEADER_HPP
#define HEADER_HPP

template< typename T >
class A
{
  // constructor+methods + pimpl
};

#endif

source.cpp :

#include "header.hpp"

// implementation

// explicitly instantiate for types that will be used
template class A< int >;
template class A< float >;
// etc...

Celia

-1 Don't use auto_ptr for PIMPL (it's undefined behavior to instantiate auto_ptr with an incomplete type). It does work if you define both constructor and destructor for the outer class. But in that case you don't need a smart pointer. – Grajeda

Yes, both OK (although I'm not sure about the details of how to use unique_ptr in this case; I would just use shared_ptr and accept the overhead as the cost of not requiring people to read fine print in the standard). Cheers, – Grajeda

@Alf Fixed the example, but Which paragraph exactly tells it is invalid to instantiate auto_ptr with an incomplete type? – Leviticus

unique_ptr should be used in this case instead of shared_ptr. Why? Because pimpl is not shared. I use unique_ptr in my code as much as shared_ptr; people tend to overuse shared_ptr inappropriately (like in this situation). – Flattop

unique_ptr also requires a user-defined destructor, just like auto_ptr did. I don't see this as an issue, honestly... it doesn't have to do anything, just not be compiler generated, and show up after Impl is completed. – Hessler

@Vjo: in C++98, §17.4.3.6/2 4th dash; in C++11, §17.6.4.8/2 5th dash. "the effects are undefined ... if an incomplete type (3.9) is used as a template argument when instantiating a template component, unless specifically allowed for that component." Cheers & hth., – Grajeda

@Dave - how do you know pimpl is not shared? If A is copied, then there would be two instances of A sharing the same object pointed to by their pimpl members. By switching to unique_ptr you lose the ability to make copies of A altogether, because unique_ptr cannot be truly copied (only "stolen" from an Rvalue reference). It depends on how A will be used. If you go around changing shared_ptr into unique_ptr because you have deduced that this will be okay, one day you're going to break a build! – Turntable

@Alf: While I was mistaken that is is okay to use auto_ptr this way [I've never noticed that particular clause before], unique_ptr most certainly can be used with an incomplete type. – Hessler

@Dennis: yes, that's what I said. i'm just not sure how far you have to go to support it in that case. whereas with shared_ptr that's well known (due to it's being used for this for so many years). cheers, – Grajeda

@DanielEarwicker in A's copy ctor member initialization list you would write pimpl(new *right.pimpl.get()). The usage you described isn't copying pimpl, it's sharing it. That is a very big difference and usually one that would be undesired and broken - however it could be a design choice to do it that way. And btw, to address your last line ego trip thing - don't do that unless you're RIGHT. – Flattop

@Dave - yes, the usage I describe is sharing it, hence my comment to you, "how do you know pimpl is not shared?" Sharing is the reason why you'd use shared_ptr, hence the name. Hardly a sign that something is "broken" if you use it for its intended purpose: to allow references to a shared object to be freely copied. And if you switch such a usage to unique_ptr, the program simply won't compile anymore. You cannot determine whether this is a valid change just by looking at the containing class. You have to see how it is used. – Turntable

@DanielEarwicker You wrote By switching to unique_ptr you lose the ability to make copies of A altogether, because unique_ptr cannot be truly copied (only "stolen" from an Rvalue reference) - which I already responded to. I'm not sure what you're defending now - seemingly just grasping at straws. – Flattop

The above example, in its original form, used shared_ptr to hold a member called pimpl. You said "unique_ptr should be used in this case instead of shared_ptr. Why? Because pimpl is not shared." That is simply not a reliable assumption. Thanks to the use of shared_ptr, the program may contain a statement a1 = a2, where those are variables of type A. If you change it to use unique_ptr, that line of code will no longer compile. In other words, pimpl may indeed be shared. Which is in contradiction to what you said. Is that any clearer? – Turntable

@DanielEarwicker Lol, this is getting cyclical. I already clearly explained that a1 = a2 will compile with unique_ptrs too and I even told you exactly what to write in A's copy ctor to make it happen correctly. This is dumb, you just aren't reading the other half of the argument or something. I'm out. – Flattop

You suggested copy constructing the object pointed to by pimpl. What if half the uses of the template instantiate it with types that cannot be copied? It just defers the exact same problem: by removing shared_ptr you remove it's sole defining feature: the ability to copy a reference without copying the object it refers to. – Turntable

@Dave Have read it many times, it's a great GOTW. It suggests using unique_ptr when designing a new PIMPL class. Meanwhile, my point was that if an existing class uses shared_ptr, it is quite possible that you won't be able eliminate all uses of it and replace them with copy operations, because the payload is (or contains) something for which "copy" is a meaningless operation. – Turntable

@DanielEarwicker You're unbelievable. We were (reread the comments dude) arguing whether unique_ptr or shared_ptr should be used for the Pimpl idiom. Also, read #5577422 – Flattop

Answer 3

There are two general solutions:

• while the interface depends on some type T, it defers to a more weakly typed implementation (e.g. one using void* pointers directly or trough type erasure), or

• you support only a specific and quite limited number of types.

The second solution is relevant for e.g. char/wchar_t-dependent stuff.

The first solution was quite common in the early days of C++ templates, because at that time compilers were not good at recognizing commonalities in the generated machine code, and would introduce so called “code bloat”. Today, much to the surprise of any novice who tries it out, a templated solution can often have smaller machine code footprint than a solution relying on runtime polymorphism. Of course, YMMV.

Cheers & hth.,

Grajeda

Someone's attempted edit suggested removing the "directly or through type erasure" elaboration and the "specific and..." qualification. The elaboration is necessary for meaning, and the qualification is necessary for correctness. The current accepted-as-solution answer is an example of the "directly". There is as yet no example of "type erasure", and the only mention is in this answer; it would be a shame if someone succeeded in deleting that. – Grajeda