Preventing header explosion in C++ (or C++0x)

Question

Lets say with have generic code like the following:

y.hpp:

#ifndef Y_HPP
#define Y_HPP

// LOTS OF FILES INCLUDED

template <class T>
class Y 
{
public:
  T z;
  // LOTS OF STUFF HERE
};

#endif

Now, we want to be able to use a Y in a class (say X) we create. However, we don't want users of X to have to include the Y headers.

So we define a class X, something like this:

x.hpp:

#ifndef X_HPP
#define X_HPP

template <class T>
class Y;

class X
{
public:
  ~X();
  void some_method(int blah);
private:
  Y<int>* y_;
};

#endif

Note that, because y_ is a pointer, we don't need to include its implementation.

The implementation is in x.cpp, which is separately compiled:

x.cpp:

#include "x.hpp"
#include "y.hpp"

X::~X() { delete y_; }
void X::someMethod(int blah) { y_->z = blah; }

So now our clients can just include "x.hpp" to use X, without including and having to process all of "y.hpp" headers:

main.cpp:

#include "x.hpp"

int main() 
{
  X x;
  x.blah(42);
  return 0; 
}

And now we can compile main.cpp and x.cpp separately, and when compiling main.cpp I don't need to include y.hpp.

However with this code I've had to use a raw pointer, and furthermore, I've had to use a delete.

So here are my questions:

(1) Is there a way I could make Y a direct member (not a pointer to Y) of X, without needing to include the Y headers? (I strongly suspect the answer to this question is no)

(2) Is there a way I could use a smart pointer class to handle the heap allocated Y? unique_ptr seems like the obvious choice, but when I change the line in x.hpp

from:

Y<int>* y_; 

to:

std::unique_ptr< Y<int> > y_;

and include , and compile with c++0x mode, I get the error:

/usr/include/c++/4.4/bits/unique_ptr.h:64: error: invalid application of ‘sizeof’ to incomplete type ‘Y<int>’ 
/usr/include/c++/4.4/bits/unique_ptr.h:62: error: static assertion failed: "can't delete pointer to incomplete type"

so is there anyway to do this by using a standard smart pointer instead of a raw pointer and also a raw delete in a custom destructor?

Solution:

Howard Hinnant has got it right, all we need to do is change x.hpp and x.cpp in the following fashion:

x.hpp:

#ifndef X_HPP
#define X_HPP

#include <memory>

template <class T>
class Y;

class X
{
public:
  X(); // ADD CONSTRUCTOR FOR X();
  ~X();
  void some_method(int blah);
private:
  std::unique_ptr< Y<int> > y_;
};

#endif

x.cpp:

#include "x.hpp"
#include "y.hpp"

X::X() : y_(new Y<int>()) {} // ADD CONSTRUCTOR FOR X();
X::~X() {}
void X::someMethod(int blah) { y_->z = blah; }

And we're good to use unique_ptr. Thanks Howard!

Rationale behind solution:

People can correct me if I'm wrong, but the issue with this code was that the implicit default constructor was trying to default initialize Y, and because it doesn't know anything about Y, it can't do that. By explicitly saying we will define a constructor elsewhere, the compiler thinks "well, I don't have to worry about constructing Y, because it's compiled elsewhere".

Really, I should have added a constructor in the first place, my program is buggy without it.

Sward

It should be noted that what you're describing has several names such as pimpl idiom and Cheshire Cat. See en.wikipedia.org/wiki/Opaque_pointer for more info. – Groin

as a side note, it seems to me that avoiding many includes is not a good enough reason for using the pimpl idiom. Are you trying to reduce the compilation time? If yes, you are doing this for the price of complicating things (as your question shows), using heap instead of stack etc. – Carreno

Answer 1

You can use either unique_ptr or shared_ptr to handle the incomplete type. If you use shared_ptr, you must outline ~X() as you have done. If you use unique_ptr you must outline both ~X() and X() (or whatever constructor you're using to construct X). It is the implicitly generated default ctor of X that is demanding a complete type Y<int>.

Both shared_ptr and unique_ptr are protecting you from accidentally calling delete on an incomplete type. That makes them superior to a raw pointer which offers no such protection. The reason unique_ptr requires the outlining of X() boils down to the fact that it has a static deleter instead of dynamic deleter.

Edit: Deeper clarification

Because of the static deleter vs dynamic deleter difference of unique_ptr and shared_ptr, the two smart pointers require the element_type to be complete in different places.

unique_ptr<A> requires A to be complete for:

• ~unique_ptr<A>();

But not for:

• unique_ptr<A>();
• unique_ptr<A>(A*);

shared_ptr<A> requires A to be complete for:

• shared_ptr<A>(A*);

But not for:

• shared_ptr<A>();
• ~shared_ptr<A>();

And finally, the implicitly generated X() ctor will call both the smart pointer default ctor and the smart pointer dtor (in case X() throws an exception - even if we know it will not).

Bottom line: Any member of X that calls a smart pointer member where the element_type is required to be complete must be outlined to a source where the element_type is complete.

And the cool thing about unique_ptr and shared_ptr is that if you guess wrong on what needs to be outlined, or if you don't realize a special member is being implicitly generated that requires a complete element_type, these smart pointers will tell you with a (sometimes poorly worded) compile time error.

Fomentation

Answer 2

1) You are right, the answer is "no": compiler should know the size of member-object, and it cannot know it without having definition of Y type.

2) boost::shared_ptr (or tr1::shared_ptr) doesn't require complete type of object. So if you can afford overhead implied by it, it would help:

The class template is parameterized on T, the type of the object pointed to. shared_ptr and most of its member functions place no requirements on T; it is allowed to be an incomplete type, or void.

Edit: have checked unique_ptr docs. Seems you can use it instead: just be sure that ~X() is defined where unique_ptr<> is constructed.

Gawain

Alexander: Why does unique_ptr require it when a raw pointer does not? – Sward

Put it simple, because unique_ptr needs to "see" destructor of Y when destructor of X is called. – Gawain

Alexander: How does shared_ptr get away with not needing this? What trick does shared_ptr do that unique_ptr can not? – Sward

shared_ptr memorizes its "deleter" function (which is destructor by default) during construction. Extremely clever trick. More of that at the end of this article: artima.com/cppsource/top_cpp_aha_moments.html – Gawain

Answer 3

If you don't like the extra pointer necessary to use the pimpl idiom, try this variant. First, define X as an abstract base class:

// x.hpp, guard #defines elided

class X
{
protected:
    X();

public:
    virtual ~X();

public:
    static X * create();
    virtual void some_method( int blah ) = 0;
};

Note that Y doesn't feature here. Then, create an impl class which derives from X:

 #include "Y.hpp"
    #include "X.hpp"

class XImpl 
: public X
{
    friend class X;

private:
    XImpl();

public:
    virtual ~XImpl();

public:
    virtual void some_method( int blah ) = 0;

private:
    boost::scoped_ptr< Y< int > > m_y;
};

X declared a factory function, create(). Implement this to return an XImpl:

// X.cpp

#include "XImpl.h"

X * X::create()
{
    return new XImpl();
}

Users of X can include X.hpp, which has no inclusion of y.hpp. You get something which looks a little like pimpl, but which doesn't have the explicit extra pointer to an impl object.

Anthracite