Why can templates only be implemented in the header file?
Asked Answered
E

19

2281

Quote from The C++ standard library: a tutorial and handbook:

The only portable way of using templates at the moment is to implement them in header files by using inline functions.

Why is this?

(Clarification: header files are not the only portable solution. But they are the most convenient portable solution.)

Enforce answered 30/1, 2009 at 10:6 Comment(8)
While it is true that placing all template function definitions into the header file is probably the most convenient way to use them, it is still not clear what's "inline" doing in that quote. There's no need to use inline functions for that. "Inline" has absolutely nothing to do with this.Ameline
A template is not like a function which can be compiled into byte code. It is just a pattern to generate such a function. If you put a template on its own into a *.cpp file, there is nothing to compile. Moreover, the explicite instanciation is actually not a template, but the starting point to make a function out of the template which ends up in the *.obj file.Bricebriceno
@AnT perhaps they meant "inline" not as the keyword but rather as "methods implemented at the place of declaration, inside the class".Ventriloquy
@AnTstandswithRussia Template functions are implicitly inlineFourhanded
@palapapa: Not true: function templates can have their definition appear in multiple translation units like inline functions can, but they are not actually inline. That matters in rare cases with explicit instantiations and also with C++20 modules.Stull
@DavisHerring Thank you. #11527915 also confirms it. Could you elaborate on why that matters with explicit instantiations and modules?Fourhanded
@palapapa: That sounds like another question (or maybe two, if it’s not “When does this matter?” but “What does this mean?”).Stull
Another canonical Q&A: Storing C++ template function definitions in a .CPP fileJoyce
R
1992

Caveat: It is not necessary to put the implementation in the header file, see the alternative solution at the end of this answer.

Anyway, the reason your code is failing is that, when instantiating a template, the compiler creates a new class with the given template argument. For example:

template<typename T>
struct Foo
{
    T bar;
    void doSomething(T param) {/* do stuff using T */}
};

// somewhere in a .cpp
Foo<int> f; 

When reading this line, the compiler will create a new class (let's call it FooInt), which is equivalent to the following:

struct FooInt
{
    int bar;
    void doSomething(int param) {/* do stuff using int */}
};

Consequently, the compiler needs to have access to the implementation of the methods, to instantiate them with the template argument (in this case int). If these implementations were not in the header, they wouldn't be accessible, and therefore the compiler wouldn't be able to instantiate the template.

A common solution to this is to write the template declaration in a header file, then implement the class in an implementation file (for example .tpp), and include this implementation file at the end of the header.

Foo.h

template <typename T>
struct Foo
{
    void doSomething(T param);
};

#include "Foo.tpp"

Foo.tpp

template <typename T>
void Foo<T>::doSomething(T param)
{
    //implementation
}

This way, implementation is still separated from declaration, but is accessible to the compiler.

Alternative solution

Another solution is to keep the implementation separated, and explicitly instantiate all the template instances you'll need:

Foo.h

// no implementation
template <typename T> struct Foo { ... };

Foo.cpp

// implementation of Foo's methods

// explicit instantiations
template class Foo<int>;
template class Foo<float>;
// You will only be able to use Foo with int or float

If my explanation isn't clear enough, you can have a look at the C++ Super-FAQ on this subject.

Raimes answered 30/1, 2009 at 10:26 Comment(43)
Actually the explicit instantiation needs to be in a .cpp file which has access to the definitions for all of Foo's member functions, rather than in the header.Lied
"the compiler needs to have access to the implementation of the methods, to instantiate them with the template argument (in this case int). If these implementations were not in the header, they wouldn't be accessible" But why is an implementation in the .cpp file not accessible to the compiler? A compiler can also access .cpp information, how else would it turn them into .obj files? EDIT: answer to this question is in the link provided in this answer...Imprison
I don't think this explains the question that clearly, the key thing is obviously related with the compilation UNIT which is not mentioned in this postEyespot
is 'struct' means the same than 'class' here ?Latvina
@Gabson: structs and classes are equivalent with the exception that the default access modifier for classes is "private", while it is public for structs. There are some other tiny differences that you can learn by looking at this question.Raimes
@xcrypt, when compiling some other .cpp file, the .cpp with the implementation is unknown (and might even not be available, think library).Crenelation
@Deduplicator: No, the members of a class are private by default, not protected.Raimes
I've added a sentence at the very start of this answer to clarify that the question is based on a false premise. If somebody asks "Why is X true?" when in fact X is not true, we should quickly reject that assumption.Jiggered
Can we use explicit instantiations with template function without a class?Onfre
@mrgloom: Yes, a function template can be explicitely instantiated juste like a class template (see How do I explicitely instantiate a template function).Raimes
Using the template instant generates error C2512 "no appropriate default constructor available", even though i defined the default constructor as foo() { }Obryant
One can never understand this without knowing "separate compilation model", which is described in the link..Necessary
Would like to add that Visual Studio (2015 Community Up3 in my case) might misbehave wildly when trying to implement the first solution (i.e. #including at the bottom) - see this for example. Had to exclude and re-include the implementation file from/to project to make it compile - rather ridiculous..Haydeehayden
I'd like to know, is it possible to do the explicit instantiations from somewhere other than the class' header or source file? For instance, do them in main.cpp?Argentine
@Birger To instantiate a template, you need its definition. If the definition is not in the header file, the only place you can do instantiations is in the implementation file that contains the definition. If the definition is in the header file, anyone with access to the header file can instantiate the template, either by using it directly (by defining a variable of the type template, or calling the function template for instance) or by making an explicit instantiation (instantiating the template without using it).Raimes
Explicit instantiation still needs the definition of the template. In this sense, it's not the second way, there is only one way .Necessary
I created a template library .cpp and .h , and only included the header . Was trying to know why I am getting linker error for hours and finally with this answer I understand whySpathose
@Andreyua Do you mean the example with explicit instantiations? I don't really see what additional information I could give, you just need to write in a .cpp the lines that instantiate your template with the arguments you want.Raimes
"Consequently, the compiler needs to have access to the implementation of the methods". Why would the compiler need to instantiate the methods? It does not when I'm declaring a non-template function, which is implemented in another cpp. Then, I think, it's the linker's job to find the implementation according to the function signature or how it does it. I don't see why the linker wouldn't just grab the instantiated signature and in addition to finding a normal function implementation, in case of a template signature it would find a template definition to generate the code.Jollenta
Aha, it's not what linkers do, generating code, right? Then I think the thought process above should be in the answer. Because normally (when not using templates), compiler does not need to have access to the implementation. And that's maybe the why that people want to know.Jollenta
@Adam: I agree - that wording in this answer is not quite right. But I think Luc was trying to simplify (and used "compiler" in place of "your compiler and linker").Carr
Is this why builds slow down dramatically after adding header libraries?Apodal
I think, answer needs an update.it's no longer "only in header", it's implementation-dependent, according to ISO standard's description of translation process. Compilers allowed to store data about template instantiation and use it on link stage. But that means that templates might be defined in .cpp files but it's up to compiler to support that.Fluxmeter
Emphasis on using a file extensions like .tpp in the second case (it must not be .cpp, otherwise there will be problems, or the use of macros is required to prevent cyclical inclusions).Filtration
@LightnessRacesinOrbit Hi, Big Boss. I would like to ask: since the answer indicates a way which is to do some explicit instantiations in .cpp file. So I think in all the other files that use any instantiations of that template, I should/must write extern template *delaration* for saving costs, am I right? Otherwise there's no meaning of using explicit instantiations.Decurved
@Rick: I guess you can do. I don't believe it's necessary.Carr
Thanks for the answer. I get it but that's an stupid limitation! like it's not rocket science, they could just make it possible. I think I will go with explicit instantiation solution for now.Rositaroskes
I wonder why I don't get any undefined reference error when I define my template function in the same .cpp file as my main(). How does the compiler know the type then? Oh! Maybe because I am using the same function in the same .cpp file which is like explicitly using it, haha? A bit late to think that but anyway I will leave my comment here.Xyster
It is also good to know that if you define a templatized class hierarchy, each class in the hierarchy needs to be explicitly intantiated, not only the "leaf" classes.Duplessis
Shouldn't be the tpp file having the definitions declared as inline? If the #include directive behaves like a physical code copy-paste, then the definitions are outside the class declaration, therefore they should be marked as inline, right?Substandard
@Eyespot Hi, would you mind explains more, I am confused why the compiler can not just create a declaration instead of a full definition like other concrete classes can do.Mentalism
@Substandard Function templates definition that are not fully specialized do not need to be marked inline (see https://mcmap.net/q/16549/-does-it-make-any-sense-to-use-inline-keyword-with-templates/20984)Raimes
@anddero Why can't a .cpp extension be used? I don't understand how a cyclical include will result when the .cpp template definition is not including anything, or how .tpp resolves this.Multiple
@JoshuaJurgensmeier @anddero C++ doesn't care about extensions, you can call your file foo.cpp, foo.tpp, foo.bar or foo it won't change a thing. The important thing is that you don't want to compile this file directly, you just want to include it in the header that contains the declaration.Raimes
@JoshuaJurgensmeier @LucTouraille I literally tested this in a CMake environment. When I named the template file .tpp, errors from cyclical inclusions did not occur. The moment I named it .cpp, they did. I was stuck on this problem for a long time, because I also thought extensions would not matter, but in my case, for whatever reason, it did.Filtration
@anddero Maybe because of some globbing, some CMake projects do thatRaimes
I've used the first solution for decades (except I use an extension of .ft instead of.tpp). You include the .ft file in any .cpp file that introduces a new specialization.Billiton
@LucTouraille C++ doesn't care about extensions, but many IDEs do, and they're also helpful hints for humans.Billiton
@Billiton Yes, I wouldn't recommend calling such a file .cpp as it is not meant to be compiled directly, and I wouldn't call it .h or .hpp either because it is not a regular include file. What I meant is that .tpp is just a random convention, not something the compiler (or even the IDE) will understand.Raimes
@LucTouraille Agreed in principle; however, many IDEs let you configure the properties of file extensions, for the purpose of syntax highlighting and the like.Billiton
Is it possible to put implementation in separate file with functions? Question here: #72332273Undis
Will it work if .cc were used instead of .tpp for the solution where we include it in the end of header file?Omophagia
@FarhatShahir Yes, there is nothing special about the .tpp extension, it is just something I saw used at the time I wrote this answer. C++ doesn't care about extensions, you could call your file .foo and include it just as well. When working with Qt you often include .moc files for instance.Raimes
L
353

It's because of the requirement for separate compilation and because templates are instantiation-style polymorphism.

Lets get a little closer to concrete for an explanation. Say I've got the following files:

  • foo.h
    • declares the interface of class MyClass<T>
  • foo.cpp
    • defines the implementation of class MyClass<T>
  • bar.cpp
    • uses MyClass<int>

Separate compilation means I should be able to compile foo.cpp independently from bar.cpp. The compiler does all the hard work of analysis, optimization, and code generation on each compilation unit completely independently; we don't need to do whole-program analysis. It's only the linker that needs to handle the entire program at once, and the linker's job is substantially easier.

bar.cpp doesn't even need to exist when I compile foo.cpp, but I should still be able to link the foo.o I already had together with the bar.o I've only just produced, without needing to recompile foo.cpp. foo.cpp could even be compiled into a dynamic library, distributed somewhere else without foo.cpp, and linked with code they write years after I wrote foo.cpp.

"Instantiation-style polymorphism" means that the template MyClass<T> isn't really a generic class that can be compiled to code that can work for any value of T. That would add overhead such as boxing, needing to pass function pointers to allocators and constructors, etc. The intention of C++ templates is to avoid having to write nearly identical class MyClass_int, class MyClass_float, etc, but to still be able to end up with compiled code that is mostly as if we had written each version separately. So a template is literally a template; a class template is not a class, it's a recipe for creating a new class for each T we encounter. A template cannot be compiled into code, only the result of instantiating the template can be compiled.

So when foo.cpp is compiled, the compiler can't see bar.cpp to know that MyClass<int> is needed. It can see the template MyClass<T>, but it can't emit code for that (it's a template, not a class). And when bar.cpp is compiled, the compiler can see that it needs to create a MyClass<int>, but it can't see the template MyClass<T> (only its interface in foo.h) so it can't create it.

If foo.cpp itself uses MyClass<int>, then code for that will be generated while compiling foo.cpp, so when bar.o is linked to foo.o they can be hooked up and will work. We can use that fact to allow a finite set of template instantiations to be implemented in a .cpp file by writing a single template. But there's no way for bar.cpp to use the template as a template and instantiate it on whatever types it likes; it can only use pre-existing versions of the templated class that the author of foo.cpp thought to provide.

You might think that when compiling a template the compiler should "generate all versions", with the ones that are never used being filtered out during linking. Aside from the huge overhead and the extreme difficulties such an approach would face because "type modifier" features like pointers and arrays allow even just the built-in types to give rise to an infinite number of types, what happens when I now extend my program by adding:

  • baz.cpp
    • declares and implements class BazPrivate, and uses MyClass<BazPrivate>

There is no possible way that this could work unless we either

  1. Have to recompile foo.cpp every time we change any other file in the program, in case it added a new novel instantiation of MyClass<T>
  2. Require that baz.cpp contains (possibly via header includes) the full template of MyClass<T>, so that the compiler can generate MyClass<BazPrivate> during compilation of baz.cpp.

Nobody likes (1), because whole-program-analysis compilation systems take forever to compile , and because it makes it impossible to distribute compiled libraries without the source code. So we have (2) instead.

Lifelike answered 11/5, 2013 at 3:54 Comment(12)
emphasized quote a template is literally a template; a class template is not a class, it's a recipe for creating a new class for each T we encounterConstantinople
I'd like to know, is it possible to do the explicit instantiations from somewhere other than the class' header or source file? For instance, do them in main.cpp?Argentine
@Birger You should be able to do it from any file that has access to the full template implementation (either because it's in the same file or via header includes).Lifelike
How's this an answer? It provides no solution but rhetoric only.Corwun
@Corwun It's not rhetoric. The question is "why do you have to implement templates in a header?", so I explained the technical choices the C++ language makes that lead to this requirement. Before I wrote my answer others already provided workarounds that are not full solutions, because there can't be a full solution. I felt those answers would be complemented by a fuller discussion of the "why" angle of the question.Lifelike
imagine it this way folks... if you weren't using templates (to efficiently code what you needed), you'd only be offering a few versions of that class anyway. so you have 3 options. 1). don't use templates. (like all other classes/functions, nobody cares that others can't alter the types) 2). use templates, and document which types they can use. 3). give them the whole implementation (source) bonus 4). give them the whole source in case they want to make a template from another one of your classes ;)Amphora
Loved this answer, as it clearly explains the why?.Tentage
Thank you, sir. This is one of the most fascinating posts I've ever read on SO.Harmaning
@Lifelike Your example with foo.cpp and bar.cpp is perfectly clear, but I have a quick question: the way you're expecting to compile and generate the executable is the following g++ -o program foo.cpp bar.cpp right?Avionics
@VoB Not really. The core point of my post is that separate compilation of templates from their implementations isn't really possible. In g++ -o program foo.cpp bar.cpp it's possible to imagine that the compiler could see a template instantiation in bar.cpp and use an implementation in food.cpp, because it's processing both of those files. But it cannot do that when you compile them separately (say if food.cpp is compiled into a library, which is only later used to build a program containing bar.cpp). So it doesn't do that in your example either, to keep the rules consistent.Lifelike
Ok, so it works only when I have small programs and I do some tricks like the one with .tpp files (see the accepted answer), which actually relies on the inclusion of the .tpp files, so in practice I'm just pasting the content of the .tpp file inside the header. So I think I got the take-home message: write both declarations and definitions in the header, right? @LifelikeAvionics
@VoB Yes, a .tpp file in that sense is just a naming convention for a kind of header file. A "header file" isn't something specific to the C++ compiler, it's just what we call a file that we intend to include into other compilation units by using #include. If it helps you work with your code to have template implementations in a separate file from the one that describes the interface of a .cpp file, and to give those template implementation files a specific extension like .tpp, then go for it! The compiler doesn't know or care about the difference, but it can help humans.Lifelike
I
277

Plenty correct answers here, but I wanted to add this (for completeness):

If you, at the bottom of the implementation cpp file, do explicit instantiation of all the types the template will be used with, the linker will be able to find them as usual.

Edit: Adding example of explicit template instantiation. Used after the template has been defined, and all member functions has been defined.

template class vector<int>;

This will instantiate (and thus make available to the linker) the class and all its member functions (only). Similar syntax works for function templates, so if you have non-member operator overloads you may need to do the same for those.

The above example is fairly useless since vector is fully defined in headers, except when a common include file (precompiled header?) uses extern template class vector<int> so as to keep it from instantiating it in all the other (1000?) files that use vector.

Intertwine answered 13/8, 2009 at 13:49 Comment(8)
Ugh. Good answer, but no real clean solution. Listing out all possible types for a template does not seem to go with what a template is supposed to be.Uproot
This can be good in many cases but generally breaks the purpose of template which is meant to allow you to use the class with any type without manually listing them.Regelate
vector is not a good example because a container is inherently targeting "all" types. But it does happen very frequently that you create templates that are only meant for a specific set of types, for instance numeric types: int8_t, int16_t, int32_t, uint8_t, uint16_t, etc. In this case, it still makes sense to use a template, but explicitly instantiating them for the whole set of types is also possible and, in my opinion, recommended.Adularia
Used after the template has been defined, "and all member functions has been defined". Thanks !Magalimagallanes
I'd like to know, is it possible to do the explicit instantiations from somewhere other than the class' header or source file? For instance, do them in main.cpp?Argentine
Explicit instantiations must be in a place where the implementation of the template methods is visible. If your implementations are in a cpp file, then (without silliness) you can only explicitly instantiate in that same cpp file, not in any other cpp file.Sail
I feel like I'm missing something … I put the explicit instantiation for two types into the class's .cpp file and the two instantiations are referred to from other .cpp files, and I still get the linking error that the members are not found.Birthplace
I feel the need to add that explicitly instantiating all numeric types feels inferior to using type traits if one has access to the feature. Any distaste over having to have the full template implementation in the same file as the declarations can be solved via a .tpp file or some similar solution.Barbarese
H
95

Templates need to be instantiated by the compiler before actually compiling them into object code. This instantiation can only be achieved if the template arguments are known. Now imagine a scenario where a template function is declared in a.h, defined in a.cpp and used in b.cpp. When a.cpp is compiled, it is not necessarily known that the upcoming compilation b.cpp will require an instance of the template, let alone which specific instance would that be. For more header and source files, the situation can quickly get more complicated.

One can argue that compilers can be made smarter to "look ahead" for all uses of the template, but I'm sure that it wouldn't be difficult to create recursive or otherwise complicated scenarios. AFAIK, compilers don't do such look aheads. As Anton pointed out, some compilers support explicit export declarations of template instantiations, but not all compilers support it (yet?).

Herbarium answered 30/1, 2009 at 10:23 Comment(7)
"export" is standard, but it's just hard to implement so most of the compiler teams just haven't done yet.Plumule
export doesn't eliminate the need for source disclosure, nor does it reduce compile dependencies, while it requires a massive effort from compiler builders. So Herb Sutter himself asked compiler builders to 'forget about' export. As the time investment needed would be better spend elsewhere...Via
So I don't think export isn't implemented 'yet'. It'll probably never get done by anyone else than EDG after the others saw how long it took, and how little was gainedVia
If that interests you, the paper is called "Why we can't afford export", it's listed on his blog (gotw.ca/publications) but no pdf there (a quick google should turn it up though)Via
Ok, thanks for good example and explanation. Here is my question though: why compiler cannot figure out where template is called, and compile those files first before compiling definition file? I can imagine it can be done in a simple case... Is the answer that interdependencies will mess up the order pretty fast?Gallon
"When a.cpp is compiled, it is not necessarily known that the upcoming compilation b.cpp will require an instance of the template, let alone which specific instance would that be." This cluster F is by design, in other words.Corwun
@Plumule The export keyword is actually reused in C++20, although for a completely different meaning ...Irwin
A
76

Actually, prior to C++11 the standard defined the export keyword that would make it possible to declare templates in a header file and implement them elsewhere. In a manner of speaking. Not really, as the only ones who ever implemented that feature pointed out:

Phantom advantage #1: Hiding source code. Many users, have said that they expect that by using export they will no longer have to ship definitions for member/nonmember function templates and member functions of class templates. This is not true. With export, library writers still have to ship full template source code or its direct equivalent (e.g., a system-specific parse tree) because the full information is required for instantiation. [...]

Phantom advantage #2: Fast builds, reduced dependencies. Many users expect that export will allow true separate compilation of templates to object code which they expect would allow faster builds. It doesn’t because the compilation of exported templates is indeed separate but not to object code. Instead, export almost always makes builds slower, because at least the same amount of compilation work must still be done at prelink time. Export does not even reduce dependencies between template definitions because the dependencies are intrinsic, independent of file organization.

None of the popular compilers implemented this keyword. The only implementation of the feature was in the frontend written by the Edison Design Group, which is used by the Comeau C++ compiler. All others required you to write templates in header files, because the compiler needs the template definition for proper instantiation (as others pointed out already).

As a result, the ISO C++ standard committee decided to remove the export feature of templates with C++11.

Anissa answered 30/1, 2009 at 13:38 Comment(9)
...and a couple of years later, I finally understood what export would actually have given us, and what not... and now I wholeheartedly agree with the EDG people: It would not have brought us what most people (myself in '11 included) think it would, and the C++ standard is better off without it.Anissa
@Anissa : this paper is political, repetitive and badly written. that's not usual standard level prose there. Uneedingly long and boring, saying basically 3 times the same things accross tens of pages. But I am now informed that export is not export. That's a good intel !Constantinople
@v.oddou: Good developer and good technical writer are two seperate skillsets. Some can do both, many can't. ;-)Anissa
@Constantinople The paper isn't just badly written, it's disinformation. Also it's a spin on reality: what are actually extremely strong arguments for exports are mixed in a way to make it sound like they are against export: “discovering numerous ODRrelated holes in the standard in the presence of export. Before export, ODR violations didn’t have to be diagnosed by the compiler. Now it’s necessary because you need to combine internal data structures from different translation units, and you can’t combine them if they’re actually representing different things, so you need to do the checking.”Aoristic
"must now add which translation unit it was in when it happened" Duh. When you're forced to use arguments that lame, you don't have any argument. Of course you are going to mention file names in your errors, what's the deal? That anyone falls for that BS is mind boggling. "Even experts like James Kanze find it hard to accept that export really is like this." WHAT?!!!!Aoristic
@curiousguy: I don't follow most of your points, actually. And being the only ones to ever implement export gives the opinions of the EDG people some credibility. Anyway, export is history, so it's somewhat pointless to necro an answer this old.Anissa
@Anissa Please explain that "credibility" thing: who is credible about what? And no, it isn't "history". It's a massive fraud on gullible people to make up false history. It's the present as these claims have been considered seriously on SO.Aoristic
@Anissa I still don't see a case made against export in the paper. (I see a case for export.)Aoristic
I thought export would be great to have as well, until that paper made me understand why EDG had been so much against it from the beginning, and why it would not have brought us what I thought it would. I could re-read that paper to brush up my understanding, and then explain it to you. But 1) export is dead, 2) this is not a discussion forum, and 3) not in that tone. You've been the one picking up the hostility on a years-old thread. (And I haven't "insulted the committee".)Anissa
J
42

Although standard C++ has no such requirement, some compilers require that all function and class templates need to be made available in every translation unit they are used. In effect, for those compilers, the bodies of template functions must be made available in a header file. To repeat: that means those compilers won't allow them to be defined in non-header files such as .cpp files

There is an export keyword which is supposed to mitigate this problem, but it's nowhere close to being portable.

Jetblack answered 30/1, 2009 at 10:15 Comment(3)
Why can't I implement them in .cpp file with the keyword "inline"?Enforce
You can, and you don't have to put "inline" even. But you'd be able to use them just in that cpp file and nowhere else.Plumule
This is almost the most accurate answer, except "that means those compilers won't allow them to be defined in non-header files such as .cpp files" is patently false.Carr
T
37

Templates are often used in headers because the compiler needs to instantiate different versions of the code, depending on the parameters given/deduced for template parameters, and it's easier (as a programmer) to let the compiler recompile the same code multiple times and deduplicate later. Remember that a template doesn't represent code directly, but a template for several versions of that code. When you compile a non-template function in a .cpp file, you are compiling a concrete function/class. This is not the case for templates, which can be instantiated with different types, namely, concrete code must be emitted when replacing template parameters with concrete types.

There was a feature with the export keyword that was meant to be used for separate compilation. The export feature is deprecated in C++11 and, AFAIK, only one compiler implemented it. You shouldn't make use of export. Separate compilation is not possible in C++ or C++11 but maybe in C++17, if concepts make it in, we could have some way of separate compilation.

For separate compilation to be achieved, separate template body checking must be possible. It seems that a solution is possible with concepts. Take a look at this paper recently presented at the standards committee meeting. I think this is not the only requirement, since you still need to instantiate code for the template code in user code.

The separate compilation problem for templates I guess it's also a problem that is arising with the migration to modules, which is currently being worked.

EDIT: As of August 2020 Modules are already a reality for C++: https://en.cppreference.com/w/cpp/language/modules

Tensiometer answered 12/5, 2013 at 16:42 Comment(1)
As of now I know of know implementation in the common compilers for modules.Athanor
T
29

Even though there are plenty of good explanations above, I'm missing a practical way to separate templates into header and body.

My main concern is avoiding recompilation of all template users, when I change its definition.

Having all template instantiations in the template body is not a viable solution for me, since the template author may not know all if its usage and the template user may not have the right to modify it.

I took the following approach, which works also for older compilers (gcc 4.3.4, aCC A.03.13).

For each template usage there's a typedef in its own header file (generated from the UML model). Its body contains the instantiation (which ends up in a library which is linked in at the end).

Each user of the template includes that header file and uses the typedef.

A schematic example:

MyTemplate.h:

#ifndef MyTemplate_h
#define MyTemplate_h 1

template <class T>
class MyTemplate
{
public:
  MyTemplate(const T& rt);
  void dump();
  T t;
};

#endif

MyTemplate.cpp:

#include "MyTemplate.h"
#include <iostream>

template <class T>
MyTemplate<T>::MyTemplate(const T& rt)
: t(rt)
{
}

template <class T>
void MyTemplate<T>::dump()
{
  cerr << t << endl;
}

MyInstantiatedTemplate.h:

#ifndef MyInstantiatedTemplate_h
#define MyInstantiatedTemplate_h 1
#include "MyTemplate.h"

typedef MyTemplate< int > MyInstantiatedTemplate;

#endif

MyInstantiatedTemplate.cpp:

#include "MyTemplate.cpp"

template class MyTemplate< int >;

main.cpp:

#include "MyInstantiatedTemplate.h"

int main()
{
  MyInstantiatedTemplate m(100);
  m.dump();
  return 0;
}

This way only the template instantiations will need to be recompiled, not all template users (and dependencies).

Twickenham answered 12/5, 2016 at 14:2 Comment(2)
I like this approach with the exception of the MyInstantiatedTemplate.h file and added MyInstantiatedTemplate type. It's a little cleaner if you don't use that, imho. Checkout my answer on a different question showing this: https://mcmap.net/q/23174/-storing-c-template-function-definitions-in-a-cpp-fileEutherian
This takes best of two worlds. I wish this answer was rated higher! Also see the link above for a slightly cleaner implementation of the same idea.Coimbra
E
17

It means that the most portable way to define method implementations of template classes is to define them inside the template class definition.

template < typename ... >
class MyClass
{

    int myMethod()
    {
       // Not just declaration. Add method implementation here
    }
};
Electromechanical answered 30/1, 2009 at 10:53 Comment(0)
T
12

The compiler will generate code for each template instantiation when you use a template during the compilation step. In the compilation and linking process .cpp files are converted to pure object or machine code which in them contains references or undefined symbols because the .h files that are included in your main.cpp have no implementation YET. These are ready to be linked with another object file that defines an implementation for your template and thus you have a full a.out executable.

However since templates need to be processed in the compilation step in order to generate code for each template instantiation that you define, so simply compiling a template separate from it's header file won't work because they always go hand and hand, for the very reason that each template instantiation is a whole new class literally. In a regular class you can separate .h and .cpp because .h is a blueprint of that class and the .cpp is the raw implementation so any implementation files can be compiled and linked regularly, however using templates .h is a blueprint of how the class should look not how the object should look meaning a template .cpp file isn't a raw regular implementation of a class, it's simply a blueprint for a class, so any implementation of a .h template file can't be compiled because you need something concrete to compile, templates are abstract in that sense.

Therefore templates are never separately compiled and are only compiled wherever you have a concrete instantiation in some other source file. However, the concrete instantiation needs to know the implementation of the template file, because simply modifying the typename T using a concrete type in the .h file is not going to do the job because what .cpp is there to link, I can't find it later on because remember templates are abstract and can't be compiled, so I'm forced to give the implementation right now so I know what to compile and link, and now that I have the implementation it gets linked into the enclosing source file. Basically, the moment I instantiate a template I need to create a whole new class, and I can't do that if I don't know how that class should look like when using the type I provide unless I make notice to the compiler of the template implementation, so now the compiler can replace T with my type and create a concrete class that's ready to be compiled and linked.

To sum up, templates are blueprints for how classes should look, classes are blueprints for how an object should look. I can't compile templates separate from their concrete instantiation because the compiler only compiles concrete types, in other words, templates at least in C++, is pure language abstraction. We have to de-abstract templates so to speak, and we do so by giving them a concrete type to deal with so that our template abstraction can transform into a regular class file and in turn, it can be compiled normally. Separating the template .h file and the template .cpp file is meaningless. It is nonsensical because the separation of .cpp and .h only is only where the .cpp can be compiled individually and linked individually, with templates since we can't compile them separately, because templates are an abstraction, therefore we are always forced to put the abstraction always together with the concrete instantiation where the concrete instantiation always has to know about the type being used.

Meaning typename T get's replaced during the compilation step not the linking step so if I try to compile a template without T being replaced as a concrete value type that is completely meaningless to the compiler and as a result object code can't be created because it doesn't know what T is.

It is technically possible to create some sort of functionality that will save the template.cpp file and switch out the types when it finds them in other sources, I think that the standard does have a keyword export that will allow you to put templates in a separate cpp file but not that many compilers actually implement this.

Just a side note, when making specializations for a template class, you can separate the header from the implementation because a specialization by definition means that I am specializing for a concrete type that can be compiled and linked individually.

Taryntaryne answered 19/7, 2016 at 1:10 Comment(0)
B
10

If the concern is the extra compilation time and binary size bloat produced by compiling the .h as part of all the .cpp modules using it, in many cases what you can do is make the template class descend from a non-templatized base class for non type-dependent parts of the interface, and that base class can have its implementation in the .cpp file.

Boehmenist answered 27/7, 2016 at 5:1 Comment(1)
This response should be modded up quite more. I "independently" discovered your same approach and was specifically looking for somebody else to have used it already, since I'm curious if it's an official pattern and whether it's got a name. My approach is to implement a class XBase wherever I need to implement a template class X, putting the type-dependent parts in X and all the rest in XBase.Inventor
L
10

Just to add something noteworthy here. One can define methods of a templated class just fine in the implementation file when they are not function templates.


myQueue.hpp:

template <class T> 
class QueueA {
    int size;
    ...
public:
    template <class T> T dequeue() {
       // implementation here
    }

    bool isEmpty();

    ...
}    

myQueue.cpp:

// implementation of regular methods goes like this:
template <class T> bool QueueA<T>::isEmpty() {
    return this->size == 0;
}


main()
{
    QueueA<char> Q;

    ...
}
Labionasal answered 19/7, 2018 at 0:49 Comment(6)
For real man???If that's true then your answer should be checked as correct one.Why does anyone need all those hacky voodo stuff if you can just define non template member methods in .cpp?Windhoek
Well that doesn't work.At least on MSVC 2019,getting unresolved external symbol for a member function of template class.Windhoek
I don't have MSVC 2019 to test. This is allowed by the C++ standard. Now, MSVC is notorious for not always adhering to the rules. If you haven't already, try Project Settings -> C/C++ -> Language -> Conformance Mode -> Yes (permissive-).Labionasal
This exact example works but then you can't call isEmpty from any other translation unit besides myQueue.cpp ...Potts
so this can be a good strategy for moving bulky functions to .cpp files and declaring them private, while the public functions stay in header file and call them.Moon
What use case can this example actually work for? In which case do you put a declaration of a template class in its own header, then include it and have to define your implementation? Why not just move everything in the same file as your main()?Almoner
O
9

A way to have separate implementation is as follows.

inner_foo.h

template <typename T>
struct Foo
{
    void doSomething(T param);
};

foo.tpp

#include "inner_foo.h"

template <typename T>
void Foo<T>::doSomething(T param)
{
    //implementation
}

foo.h

#include <foo.tpp>

main.cpp

#include <foo.h>

inner_foo.h has the forward declarations. foo.tpp has the implementation and includes inner_foo.h; and foo.h will have just one line, to include foo.tpp.

On compile time, contents of foo.h are copied to foo.tpp and then the whole file is copied to foo.h after which it compiles. This way, there is no limitations, and the naming is consistent, in exchange for one extra file.

I do this because static analyzers for the code break when it does not see the forward declarations of class in *.tpp. This is annoying when writing code in any IDE or using YouCompleteMe or others.

Ornithomancy answered 13/5, 2017 at 1:42 Comment(2)
s/inner_foo/foo/g and include foo.tpp at the end of foo.h. One less file.Endogen
user246672 had it slightly wrong -- just include the .tpp files (I use .ft) in the .cpp files that need them.Billiton
N
7

That is exactly correct because the compiler has to know what type it is for allocation. So template classes, functions, enums,etc.. must be implemented as well in the header file if it is to be made public or part of a library (static or dynamic) because header files are NOT compiled unlike the c/cpp files which are. If the compiler doesn't know the type is can't compile it. In .Net it can because all objects derive from the Object class. This is not .Net.

Nightrider answered 17/9, 2011 at 3:40 Comment(2)
"header files are NOT compiled" - that's a really odd way of describing it. Header files can be part of a translation unit, just like a "c/cpp" file.Slovak
In fact, it's almost the opposite of the truth, which is that header files are very frequently compiled many times, whereas a source file is usually compiled once.Dahliadahlstrom
K
5

I suggest looking at this gcc page which discusses the tradeoffs between the "cfront" and "borland" model for template instantiations.

https://gcc.gnu.org/onlinedocs/gcc-4.6.4/gcc/Template-Instantiation.html

The "borland" model corresponds to what the author suggests, providing the full template definition, and having things compiled multiple times.

It contains explicit recommendations concerning using manual and automatic template instantiation. For example, the "-repo" option can be used to collect templates which need to be instantiated. Or another option is to disable automatic template instantiations using "-fno-implicit-templates" to force manual template instantiation.

In my experience, I rely on the C++ Standard Library and Boost templates being instantiated for each compilation unit (using a template library). For my large template classes, I do manual template instantiation, once, for the types I need.

This is my approach because I am providing a working program, not a template library for use in other programs. The author of the book, Josuttis, works a lot on template libraries.

If I was really worried about speed, I suppose I would explore using Precompiled Headers https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html

which is gaining support in many compilers. However, I think precompiled headers would be difficult with template header files.

Kerbela answered 17/2, 2020 at 20:30 Comment(0)
A
3

(copying here from a closed duplicate)

I prefer to have all of my functions in the .cpp file, regardless of whether they are template functions or regular functions. And there is a way to do that with some basic #ifndef magic. Here's what you can do:

main.cpp

#include "myclass.hpp"

int main()
{
  // ...
}

myclass.hpp

#ifndef MYCLASS
#define MYCLASS

template<class T>
class MyClass
{
  T val;
public:
  MyClass(T val_);
}

#define MYCLASS_FUNCTIONS
#include "myclass.cpp"

#endif

myclass.cpp

#ifndef MYCLASS_FUNCTIONS
#include "myclass.hpp"

// regular functions:
// ...

#else
 
// template functions:
template<class T>
MyClass<T>::MyClass(T val_)
    :val(val_)
{}

// ...
#endif

Here's how the precompiler sees it. We have two .cpp files.

  1. When we compile main.cpp we:
  2. include myclass.hpp
  3. check that MYCLASS is undefined, and it is
  4. define it
  5. give compiler the definitions of the generated class (from template class)
  6. include myclass.cpp
  7. define MYCLASS_FUNCTIONS
  8. check if MYCLASS_FUNCTIONS is defined, it is
  9. give compiler the definitions of the generated functions (from template functions)
  10. When we compile myclass.cpp
  11. check if MYCLASS_FUNCTIONS is defined, it isn't
  12. include myclass.hpp
  13. check that MYCLASS is undefined, and it is
  14. define it
  15. give compiler the definitions of the class
  16. include myclass.cpp
  17. include myclass.hpp again
  18. this time MYCLASS is defined so do nothing inside, return to myclass.cpp
  19. check if MYCLASS_FUNCTIONS is defined, it is
  20. give compiler the definition of the generated functions (from template functions)
  21. exit include twice
  22. pass to the compiler all the regular functions
Almoner answered 21/7, 2023 at 16:13 Comment(0)
A
1

With motivation from Moshe's answer from: https://mcmap.net/q/23170/-why-can-templates-only-be-implemented-in-the-header-file

A small contribution from my side with an extended example. Let's say there is an overall OperationSuccess and it contains a ResponseSuccess which has a generic type in it.

ResponseSuccess.h

template <class T>
class ResponseSuccess {
public:
    ResponseSuccess(const ResponseStatus responseStatus, const T& data) :
        m_responseStatus(responseStatus),
        m_data(data) {}

    ~ResponseSuccess() = default;

    // Basis requirement, have Copy/Move constructor/delete assignment operator

    ResponseStatus getResponseStatus() const {
        return m_responseStatus;
    }

    T getData() const {
        return m_data;
    };

private:
    ResponseStatus m_responseStatus;

    T m_data;
};

OperationSuccess.h

template <class T>
class OperationResponse {
public:
    explicit OperationResponse(ResponseSuccess<T> responseSuccess) :
        m_responseSuccess(std::move(responseSuccess)) {}

    ~OperationResponse() = default;
    // Basis requirement, have Copy/Move constructor/delete assignment operator

    ResponseSuccess<T> getResponseSuccess() const {
        return m_responseSuccess;
    }

private:
    ResponseSuccess<T> m_responseSuccess;
    // have a failure, in case required
};

Usage:

MyObject myObj(<ctor_args>);
    ResponseSuccess<MyObject> responseSuccess(ResponseStatus::SUCCESS, myObj);
    OperationResponse<MyObject> successOperationResponse(responseSuccess);
..
// Fetches the response -> successOperationResponse.getResponseSuccess();
Arakawa answered 25/4, 2023 at 4:56 Comment(0)
C
-5

Another reason that it's a good idea to write both declarations and definitions in header files is for readability. Suppose there's such a template function in Utility.h:

template <class T>
T min(T const& one, T const& theOther);

And in the Utility.cpp:

#include "Utility.h"
template <class T>
T min(T const& one, T const& other)
{
    return one < other ? one : other;
}

This requires every T class here to implement the less than operator (<). It will throw a compiler error when you compare two class instances that haven't implemented the "<".

Therefore if you separate the template declaration and definition, you won't be able to only read the header file to see the ins and outs of this template in order to use this API on your own classes, though the compiler will tell you in this case about which operator needs to be overridden.

Commonweal answered 19/3, 2019 at 19:33 Comment(0)
B
-12

I had to write a template class an d this example worked for me

Here is an example of this for a dynamic array class.

#ifndef dynarray_h
#define dynarray_h

#include <iostream>

template <class T>
class DynArray{
    int capacity_;
    int size_;
    T* data;
public:
    explicit DynArray(int size = 0, int capacity=2);
    DynArray(const DynArray& d1);
    ~DynArray();
    T& operator[]( const int index);
    void operator=(const DynArray<T>& d1);
    int size();
    
    int capacity();
    void clear();
    
    void push_back(int n);
    
    void pop_back();
    T& at(const int n);
    T& back();
    T& front();
};

#include "dynarray.template" // this is how you get the header file

#endif

Now inside you .template file you define your functions just how you normally would.

template <class T>
DynArray<T>::DynArray(int size, int capacity){
    if (capacity >= size){
        this->size_ = size;
        this->capacity_ = capacity;
        data = new T[capacity];
    }
    //    for (int i = 0; i < size; ++i) {
    //        data[i] = 0;
    //    }
}

template <class T>
DynArray<T>::DynArray(const DynArray& d1){
    //clear();
    //delete [] data;
    std::cout << "copy" << std::endl;
    this->size_ = d1.size_;
    this->capacity_ = d1.capacity_;
    data = new T[capacity()];
    for(int i = 0; i < size(); ++i){
        data[i] = d1.data[i];
    }
}

template <class T>
DynArray<T>::~DynArray(){
    delete [] data;
}

template <class T>
T& DynArray<T>::operator[]( const int index){
    return at(index);
}

template <class T>
void DynArray<T>::operator=(const DynArray<T>& d1){
    if (this->size() > 0) {
        clear();
    }
    std::cout << "assign" << std::endl;
    this->size_ = d1.size_;
    this->capacity_ = d1.capacity_;
    data = new T[capacity()];
    for(int i = 0; i < size(); ++i){
        data[i] = d1.data[i];
    }
    
    //delete [] d1.data;
}

template <class T>
int DynArray<T>::size(){
    return size_;
}

template <class T>
int DynArray<T>::capacity(){
    return capacity_;
}

template <class T>
void DynArray<T>::clear(){
    for( int i = 0; i < size(); ++i){
        data[i] = 0;
    }
    size_ = 0;
    capacity_ = 2;
}

template <class T>
void DynArray<T>::push_back(int n){
    if (size() >= capacity()) {
        std::cout << "grow" << std::endl;
        //redo the array
        T* copy = new T[capacity_ + 40];
        for (int i = 0; i < size(); ++i) {
            copy[i] = data[i];
        }
        
        delete [] data;
        data = new T[ capacity_ * 2];
        for (int i = 0; i < capacity() * 2; ++i) {
            data[i] = copy[i];
        }
        delete [] copy;
        capacity_ *= 2;
    }
    data[size()] = n;
    ++size_;
}

template <class T>
void DynArray<T>::pop_back(){
    data[size()-1] = 0;
    --size_;
}

template <class T>
T& DynArray<T>::at(const int n){
    if (n >= size()) {
        throw std::runtime_error("invalid index");
    }
    return data[n];
}

template <class T>
T& DynArray<T>::back(){
    if (size() == 0) {
        throw std::runtime_error("vector is empty");
    }
    return data[size()-1];
}

template <class T>
T& DynArray<T>::front(){
    if (size() == 0) {
        throw std::runtime_error("vector is empty");
    }
    return data[0];
    }
Barbour answered 11/3, 2019 at 18:38 Comment(1)
Most people would define a header file to be anything that propagates definitions to source files. So you may have decided to use the file extension ".template" but you've written a header file.Obduce

© 2022 - 2024 — McMap. All rights reserved.