Is std::unique_ptr<T> required to know the full definition of T?

Question

I have some code in a header that looks like this:

#include <memory>

class Thing;

class MyClass
{
    std::unique_ptr< Thing > my_thing;
};

If I include this header in a cpp that does not include the Thing type definition, then this does not compile under VS2010-SP1:

1>C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\memory(2067): error C2027: use of undefined type 'Thing'

Replace std::unique_ptr by std::shared_ptr and it compiles.

So, I'm guessing that it's the current VS2010 std::unique_ptr's implementation that requires the full definition and it's totally implementation-dependant.

Or is it? Is there something in it's standard requirements that makes impossible for std::unique_ptr's implementation to work with a forward declaration only? It feels strange as it should only hold a pointer to Thing, shouldn't it?

Swim

The best explanation of when you do and do not need a complete type with the C++0x smart pointers is Howard Hinnant's "Incomplete types and shared_ptr/unique_ptr" The table at the end should answer your question. – Ovular

Thanks for the pointer James. I had forgotten where I put that table! :-) – Doggoned

https://mcmap.net/q/65964/-how-do-i-use-unique_ptr-for-pimpl – Corliss

@JamesMcNellis The link to Howard Hinnant's website is down. Here is the web.archive.org version of it. In any case, he answered it perfectly below with the same content :-) – Juliannjulianna

Another good explanation is given in Item 22 of Scott Meyers' Effective modern C++. – Jointure

howardhinnant.github.io/incomplete.html – Indescribable

Answer 1

Adopted from here.

Most templates in the C++ standard library require that they be instantiated with complete types. However shared_ptr and unique_ptr are partial exceptions. Some, but not all of their members can be instantiated with incomplete types. The motivation for this is to support idioms such as pimpl using smart pointers, and without risking undefined behavior.

Undefined behavior can occur when you have an incomplete type and you call delete on it:

class A;
A* a = ...;
delete a;

The above is legal code. It will compile. Your compiler may or may not emit a warning for above code like the above. When it executes, bad things will probably happen. If you're very lucky your program will crash. However a more probable outcome is that your program will silently leak memory as ~A() won't be called.

Using auto_ptr<A> in the above example doesn't help. You still get the same undefined behavior as if you had used a raw pointer.

Nevertheless, using incomplete classes in certain places is very useful! This is where shared_ptr and unique_ptr help. Use of one of these smart pointers will let you get away with an incomplete type, except where it is necessary to have a complete type. And most importantly, when it is necessary to have a complete type, you get a compile-time error if you try to use the smart pointer with an incomplete type at that point.

No more undefined behavior

If your code compiles, then you've used a complete type everywhere you need to.

class A
{
    class impl;
    std::unique_ptr<impl> ptr_;  // ok!

public:
    A();
    ~A();
    // ...
};

Type completeness requirements for unique_ptr and shared_ptr

shared_ptr and unique_ptr require a complete type in different places. The reasons are obscure, having to do with a dynamic deleter vs a static deleter. The precise reasons aren't important. In fact, in most code it isn't really important for you to know exactly where a complete type is required. Just code, and if you get it wrong, the compiler will tell you.

However, in case it is helpful to you, here is a table which documents several operations of shared_ptr and unique_ptr with respect to completeness requirements.

Operation	unique_ptr	shared_ptr
P() (default constructor)	incomplete	incomplete
P(const P&) (copy constructor)	—	incomplete
P(P&&) (move constructor)	incomplete	incomplete
~P() (destructor)	complete	incomplete
P(A*) (constructor from ptr)	incomplete	complete
operator=(const P&) (copy assignment)	—	incomplete
operator=(P&&) (move assignment)	complete	incomplete
reset()	complete	incomplete
reset(A*)	complete	complete

Any operations requiring pointer conversions require complete types for both unique_ptr and shared_ptr.

The unique_ptr<A>{A*} constructor can get away with an incomplete A only if the compiler is not required to set up a call to ~unique_ptr<A>(). For example if you put the unique_ptr on the heap, you can get away with an incomplete A. More details on this point can be found in BarryTheHatchet's answer here.

Doggoned

I added constructors to my classes and they now work with forward declaration of Mytype when I use unique_ptr<MyType>. Thanks! – Swim

Excellent answer. I'd +5 it if I could. I'm sure I'll be referring back to this in my next project, in which I'm attempting to make full use of smart pointers. – Novikoff

if one can explain what the table means I guess it will help more people – Tombolo

One more note: A class constructor will reference the destructors of its members (for the case where an exception is thrown, those destructors need to be called). So while unique_ptr's destructor needs a complete type, it is not enough to have a user defined destructor in a class - it also needs a constructor. – Affront

@JohannesSchaub-litb: Agreed. I was hopeful that a noexcept constructor would not need to reference destructors as you correctly state, but it appears that this is not the case. – Doggoned

@HowardHinnant i don't know what the top-of-trunk state of affairs is on that. There recently were talking about this in the context of accessibility of subobject destructors from within constructors. It seems that it was decided that the subobject destructors need always be accessible (worst-case scenario). I'm going to assume it's the same for requiring their instantiations. – Affront

So basically, if you use the pimpl idiom, unique_ptr only works on a POD class/struct? If you do define a constructor, you need the shared_ptr? – Antecedency

@Zoomulator: No. You can use unique_ptr for pimpl. You just need to outline your special members. And it is safe in that if you accidentally forget, the compiler will remind you. You won't accidentally get into undefined run time behavior as you might with auto_ptr in C++98/03. – Doggoned

For those wondering, this capability is stated in the standard in the following places: [unique.ptr]/5 for unique_ptr and [util.smartptr.shared]/2 shared_ptr. unique_ptr must have a complete type wherever it calls default_delete::operator() because of [unique.ptr.dltr.dflt]/4. – Allgood

-1 The linked article (and therefore the answer) doesn't explain why unique_ptr and shared_ptr have different requirements. – Caryophyllaceous

@HowardHinnant: What's the reasoning behind C++ containers requiring complete types? It's completely possible to avoid this (Boost.Container does so -- containers don't need to know the sizes of their targets a priori) so why the arbitrary requirement? – Trimming

@Mehrdad: This decision was made for C++98, which is before my time. However I believe the decision came from a concern about implementability, and the difficulty of specification (i.e. exactly which parts of a container do or do not require a complete type). Even today, with 15 years of experience since C++98, it would be a non-trivial task to both relax the container specification in this area, and ensure that you don't outlaw important implementation techniques or optimizations. I think it could be done. I know it would be a lot of work. I'm aware of one person making the attempt. – Doggoned

@HowardHinnant: I see. In theory I think it should be possible because container objects only contain pointers -- which means incomplete types should work just fine. In practice though I can imagine you'd run into problems. Thanks for the explanation! – Trimming

I think unique_ptr's constructor also ends up being a C in this table. See here. – Benempt

@Barry: Your statements about exception safety are correct. But if you construct a unique_ptr in such a way that the compiler does not have set up a call to the destructor, the T* constructor can operate with an incomplete T. One example would be putting the unique_ptr on the heap. – Doggoned

@HowardHinnant True. Maybe worth adding a footnote to your answer since it's linked from everywhere? – Benempt

Because it's not obvious from the above comments, for anyone having this problem because they define a unique_ptr as a member variable of a class, just explicitly declare a destructor (and constructor) in the class declaration (in the header file) and proceed to define them in the source file (and put the header with the complete declaration of the pointed-to class in the source file) to prevent the compiler auto-inlining the constructor or destructor in the header file (which triggers the error). https://mcmap.net/q/65966/-forward-declaration-with-unique_ptr-duplicate also helps remind me of this. – Rochdale

So I am still somewhat confused. I have a class which has a method returning a null unique_ptr<T>. But it seems that it needs to know the complete type T. So I am not sure what's going on. – Yearling

Note my own painful experience related to this topic: Default-initializing the unique pointer (i.e. std::unique_ptr<Thing> my_thing{};) also prevents forward class declaration. The solution is straightforward: Don't default-initialize the unique pointer. – Duval

@RolandSarrazin Just had the same experience. Was pretty bad since I could not reproduce it: wandbox.org/permlink/rmmbeU49lDBwTsNV . After switching the compiler version, this seems to be possible since GCC9. Latest Clang 10 does not work however. – Ito

@typ1232: This compiles with clang: wandbox.org/permlink/MnEXt9sw4NIZLiWW – Doggoned

I don't know if anyone is still looking here, but I have a very simple pImpl pattern here: wandbox.org/permlink/9joq3PnkEZ6j8TI8 and it suffers from this same issue. Can anyone help? – Anticholinergic

wandbox.org/permlink/B7Jvdg5t1veGrCGu – Doggoned

For some reason, in MSVC, one can use = nullptr or { nullptr } initialization on the std::unique_ptr to incomplete type member, but on gcc and clang, it does not work. (Note: I tested gcc and clang for C++14, 17, and 20.) – Boondoggle

Answer 2

The compiler needs the definition of Thing to generate the default destructor for MyClass. If you explicitly declare the destructor and move its (empty) implementation to the CPP file, the code should compile.

Spitfire

I think this is the perfect opportunity to use a defaulted function. MyClass::~MyClass() = default; in the implementation file seems less likely to be inadvertently removed later down the road by somebody who assumes the destuctor body was erased rather than deliberately left blank. – Neman

@Dennis Zickefoose : Unfortunately the OP is using VC++, and VC++ does not yet support defaulted and deleted class members. – Tokoloshe

+1 for how to move door into .cpp file. Also it seems MyClass::~MyClass() = default doesn't move it into implementation file on Clang. (yet?) – Sequacious

You also need to move the implementation of the constructor to the CPP file, at least on VS 2017. See for example this answer: https://mcmap.net/q/65965/-unique_ptr-and-forward-declaration – Jannjanna

Answer 3

This isn't implementation-dependent. The reason that it works is because shared_ptr determines the correct destructor to call at run-time - it isn't part of the type signature. However, unique_ptr's destructor is part of its type, and it must be known at compile-time.

Liquidity

Answer 4

It looks like current answers are not exactly nailing down why default constructor (or destructor) is problem but empty ones declared in cpp isn't.

Here's whats happening:

If outer class (i.e. MyClass) doesn't have constructor or destructor then compiler generates the default ones. The problem with this is that compiler essentially inserts the default empty constructor/destructor in the .hpp file. This means that the code for default contructor/destructor gets compiled along with host executable's binary, not along with your library's binaries. However this definitions can't really construct the partial classes. So when linker goes in your library's binary and tries to get constructor/destructor, it doesn't find any and you get error. If the constructor/destructor code was in your .cpp then your library binary has that available for linking.

This is nothing to do with using unique_ptr or shared_ptr and other answers seems to be possible confusing bug in old VC++ for unique_ptr implementation (VC++ 2015 works fine on my machine).

So moral of the story is that your header needs to remain free of any constructor/destructor definition. It can only contain their declaration. For example, ~MyClass()=default; in hpp won't work. If you allow compiler to insert default constructor or destructor, you will get a linker error.

One other side note: If you are still getting this error even after you have constructor and destructor in cpp file then most likely the reason is that your library is not getting compiled properly. For example, one time I simply changed project type from Console to Library in VC++ and I got this error because VC++ did not added _LIB preprocessor symbol and that produced exact same error message.

Heiney

Thank you! That was a very succinct explanation of an incredibly obscure C++ quirk. Saved me a lot of trouble. – Mauri

This really doesn't explain the other part "Replace std::unique_ptr by std::shared_ptr and it compiles." The moral of the story you told in the answer, why is it not applicable to shared_ptr. – Additional

Answer 5

Just for completeness:

Header: A.h

class B; // forward declaration

class A
{
    std::unique_ptr<B> ptr_;  // ok!  
public:
    A();
    ~A();
    // ...
};

Source A.cpp:

class B {  ...  }; // class definition

A::A() { ... }
A::~A() { ... }

The definition of class B must be seen by constructor, destructor and anything that might implicitely delete B. (Although the constructor doesn't appear in the list above, in VS2017 even the constructor needs the definition of B. And this makes sense when considering that in case of an exception in the constructor the unique_ptr is destroyed again.)

Corm

Answer 6

I was looking for a way to use the PIMPL idiom with std::unique_ptr. This guide is a great resource.

In short, here's what you can do to make it work:

my_class.h

#include <memory>

class Thing;

class MyClass
{
    ~MyClass(); // <--- Added
    std::unique_ptr< Thing > my_thing;
};

my_class.cpp

MyClass::~MyClass() = default; // Or a custom implementation

Monopode

Answer 7

The full definition of the Thing is required at the point of template instantiation. This is the exact reason why the pimpl idiom compiles.

If it wasn't possible, people would not ask questions like this.

Farcical

Answer 8

The simple answer is just use shared_ptr instead.

Selaginella

Answer 9

As for me,

QList<QSharedPointer<ControllerBase>> controllers;

Just include the header ...

#include <QSharedPointer>

Transceiver

Answer not related and not relevant to the question. – Fencible