Template instantiation details of GCC and MS compilers
Asked Answered
E

2

47

Could anyone provide a comparison or specific details of how is template instantiation handled at compile and/or link time in GCC and MS compilers? Is this process different in the context of static libraries, shared libraries and executables? I found this doc about how GCC handles it but I'm not sure if the information is still referring to the current state of things. Should I use the flags they suggest there when compiling my libraries e.g. -fno-implicit-templates?

What I know (might not necessarily be correct) is that:

  • templates will be instantiated when actually used
  • templates will be instantiated as a result of explicit instantiations
  • duplicate instantiation is usually handled by folding duplicate instantiations, or by deferring instantiation until link time
Enzymology answered 24/8, 2011 at 21:14 Comment(3)
And now is the moment to look like a total dork(the story of my life): I think the downvote is there just to account for the bounty(as I got immediately subtracted the 50 reputation when I "bountified" the question).Enzymology
I do not think that bounties change the "value" of a question.Shilling
Well, Microsoft does not implement two phase lookup properly for starters. (this is a euphemism: there is no two phase lookup at all)Playwriting
S
63

Point of instantiation

templates will be instantiated when actually used

Not exactly, but roughly. The precise point of instantiation is a bit subtle, and I delegate you over to the section named Point of instantiation in Vandevoorde's/Josuttis' fine book.

However, compilers do not necessarily implement the POIs correctly: Bug c++/41995: Incorrect point of instantiation for function template


Partial instantiation

templates will be instantiated when actually used

That is partially correct. It is true for function templates, but for class templates, only the member functions that are used are instantiated. The following is well-formed code:

#include <iostream>

template <typename> struct Foo {
    void let_me_stay() {
        this->is->valid->code. get->off->my->lawn;
    }

    void fun() { std::cout << "fun()" << std::endl; } 
};


int main () {
    Foo<void> foo;
    foo.fun();
}

let_me_stay() is checked syntactically (and the syntax there is correct), but not semantically (i.e. it is not interpreted).


Two phase lookup

However, only dependent code is interpreted later; clearly, within Foo<>, this is dependent upon the exact template-id with which Foo<> is instantiated, so we postponed error-checking of Foo<>::let_me_alone() until instantiation time.

But if we do not use something that depends on the specific instantiation, the code must be good. Therefore, the following is not well-formed:

$ cat non-dependent.cc
template <typename> struct Foo {
    void I_wont_compile() { Mine->is->valid->code. get->off->my->lawn; }
};
int main () {} // note: no single instantiation

Mine is a completely unknown symbol to the compiler, unlike this, for which the compiler could determine it's instance dependency.

The key-point here is that C++ uses a model of two-phase-lookup, where it does checking for non-dependent code in the first phase, and semantic checking for dependent code is done in phase two (and instantiation time) (this is also an often misunderstood or unknown concept, many C++ programmers assume that templates are not parsed at all until instantiation, but that's only myth coming from, ..., Microsoft C++).


Full instantiation of class templates

The definition of Foo<>::let_me_stay() worked because error checking was postponed to later, as for the this pointer, which is dependent. Except when you would have made use of

explicit instantiations

cat > foo.cc
#include <iostream>

template <typename> struct Foo {
    void let_me_stay() { this->is->valid->code. get->off->my->lawn; }
    void fun() { std::cout << "fun()" << std::endl; } 
};

template struct Foo<void>;
int main () {
    Foo<void> foo;
    foo.fun();
}

g++ foo.cc
error: error: ‘struct Foo<void>’ has no member named ‘is’


Template definitions in different units of translation

When you explicitly instantiate, you instantiate explicitly. And make all symbols visible to the linker, which also means that the template definition may reside in different units of translation:

$ cat A.cc
template <typename> struct Foo {
    void fun();  // Note: no definition
};
int main () {
    Foo<void>().fun();
}

$ cat B.cc
#include <iostream>
template <typename> struct Foo {
    void fun();

};
template <typename T>
void Foo<T>::fun() { 
    std::cout << "fun!" << std::endl;
}  // Note: definition with extern linkage

template struct Foo<void>; // explicit instantiation upon void

$ g++ A.cc B.cc
$ ./a.out
fun!

However, you must explicitly instantiate for all template arguments to be used, otherwise

$ cat A.cc
template <typename> struct Foo {
    void fun();  // Note: no definition
};
int main () {
    Foo<float>().fun();
}
$ g++ A.cc B.cc
undefined reference to `Foo<float>::fun()'

Small note about two-phase lookup: Whether a compiler actually implements two-phase lookup is not dictated by the standard. To be conformant, however, it should work as if it did (just like addition or multiplication do not necessarily have to be performed using addition or multiplication CPU instructions.

Shilling answered 30/8, 2011 at 10:4 Comment(3)
Very nice point about dependent vs. non-dependent(I certainly was not aware of that). I just checked the following code: template <typename> struct Foo { void I_wont_compile() { this->is->valid->code. get->off->my->lawn; } void Test() { fdsh "s.w" = 6; wtf? } }; int main () { } It compiles fine in VC++ 2008 but does not compile in GCC. Does anyone know why VC++ behaves that way?(maybe because of the overhead incurred at compile time?)Enzymology
VC++ has had more problems in the past. You may browse the boost-sources and see many workarounds related to (older versions) of VC++. Another bug I am aware of is that you can befriend with template parameters, which would be nice to implement a final class as in C#, but which is actually invalid C++: codeguru.com/cpp/cpp/cpp_mfc/stl/article.php/c4143 .Shilling
Well the bounty has expired whitout having the chance to actually award it, as I haven't been active on the site for a couple of days. I'm not sure how to proceed. Should I open another one(I think this answer deserves the bounty)?Enzymology
W
-2

Edit: It turns out that what I wrote below is contrary to the C++ standard. It is true for Visual C++, but false for compilers that use "two-phase name lookup".

As far as I know, what you say is correct. Templates will be instantiated when actually used (including when declared as a member of another type, but not when mentioned in a function declaration (that does not have a body)) or as a result of explicit instantiations.

A problem with templates is that if you use the same template (e.g. vector) in several different compilation units (.cpp files), the compiler repeats the work of instantiating the template in each .cpp file, thus slowing down compilation. IIRC, GCC has some (non-standard?) mechanism that can be used to avoid this (but I don't use GCC). But Visual C++ always repeats this work, unless you use explicit template instantiation in a precompiled header (but even this will slow down your compile, since a larger PCH file takes longer to load.) Afterward, the linker then eliminates the duplicates. Note: a comment below linked to a page which tells us that not all compilers operate this way. Some compilers defer function instantiation until link time, which should be more efficient.

A template is not fully instantiated when it is first used. In particular, functions in the template are not instantiated until they are actually called. You can easily verify this by adding a nonsense function to a template you are actively using:

void Test() { fdsh "s.w" = 6; wtf? }

You won't get an error unless you instantiate the template explicitly, or try to call the function.

I expect static libraries (and object files) will store the object code of all templates that were instantiated. But if your program has a certain static library as a dependency, you can't actually call the template functions that were already instantiated therein, at least not in VC++, which always requires the source code (with function bodies) of a template class in order to call functions in it.

I don't think it's possible to call a template function in a shared library (when you don't have the source code of the template function you want to call).

Weisbrodt answered 26/8, 2011 at 22:3 Comment(19)
Even my editor knows that code stinks. Have you even TRIED it? The code is going to be fully parsed but identifiers not resolved (many semantic checks ar postponed to instantiation time). Sometimes for that reason you have to disambiguate template-parameter dependent names (the famous typename keyword, e.g.); At instantiation time, semantic errors on substitution aren't fatal (google SFINAE)Lachrymator
Yes, I just tried it. There is no compiler error because I don't call Test() from anywhere. Template functions can be understood as glorified macros. Like a macro, you don't get an error if you don't use it.Weisbrodt
What compiler is that? If there is no compiler error, the compiler is the error. Let me guess. Borland C++?Lachrymator
@Lachrymator "The instantiation of a class template is always done as soon as it is needed in a compilation. However, the instantiations of template functions, member functions of template classes, and static data members of template classes (hereafter referred to as template entities) are not necessarily done immediately" ... taken from comeaucomputing.com/4.0/docs/userman/ati.htmlEnzymology
@celavek: Ok... I'll take that as authoritative (comeau being the only compiler to ever have attempted proper support for template export -- they know their stuff)Lachrymator
That template export feature looks nice; my compile times are getting pretty long and I think it's because I use lots of templates.Weisbrodt
@Lachrymator I don't mean to be patronizing or anything :), but just to add a little more weight to the above I just found an example in "C++ Templates The Complete Guide" that illustrates exactly what Qwertie was saying - see pages 143 - 145(by the way David Vandevoorde, one of the authors of the above mentioned book, works for Edison Design Group).Enzymology
@Qwertie: don't get your hopes up: the export feature has been dropped from the standard; see also #2442386Lachrymator
And it has been dropped in the standard more or less at the request of the Edison Design Group, who I believe provided the only commercial implementation of export (Comeau is based on an EDG front end).Unshroud
@sehe: You're confusing export, which was rarely used and has been dropped, and extern, which declares that the template has already been instantiated in another translation unit. See '[temp.explicit]' paragraph 2.Donothingism
@Dave S: Really? Have you checked that? I think (note constructive mode) that you are confusing export and extern :) Perhaps it is because of the unfortunate fact that MSVC++ has supported a proprietary extension known as extern template declarations, that kind of overlaps the export feature. However, by no means is it the same.Lachrymator
@sehe: From the Feb 2011-2-28 draft: Section 14.7.2: There are two forms of explicit instantiation: an explicit instantiation definition and an explicit instantiation declaration. An explicit instantiation declaration begins with the extern keyword. And paragraph 10, same section: explicit instantiation declarations have the effect of suppressing the implicit instantiation of the entity to which they refer. Donothingism
The draft spec also no longer lists 'export template' as legal syntax, but has added 'extern template' which is new. I believe the behavior is the same as the MSVC++ behavior (and GCC behavior) that they implemented as an extension.Donothingism
@Dave S: exactly. I was referring to old specs only, and I wasn't confusing anything. I simply wasn't referring to the new-fangled extern template specs as it was ... irrelevant. For completeness, your reference is more than welcome, though.Lachrymator
little bump: This post is still contains some totally wrong information.Shilling
@phresnel: "Q: What is two-phase name lookup? A: This is the process specified by the standard for looking up names in templates. It is called "two-phase name lookup" because it divides names used in a template into two categories (dependent and non-dependent) that are resolved at different times. It was introduced into the draft standard some time in 1993 or 1994 but unfortunately has not been implemented by many vendors until quite recently. It makes name resolution more reliable, but is incompatible with a lot of older template code." - - This has no obvious relation to my post.Weisbrodt
@Qwertie: I am refering to 'void Test() { fdsh "s.w" = 6; wtf? } ' and that it won't give an error unless instantiated. This is wrong and a (widespread) misconception; only non-standards-conforming compilers allow this code to survive phase 1.Shilling
Huh, void Test() { fdsh "s.w" = 6; wtf? } is still there and described as valid until instantiated. That is still not true, and the three downvotes are still justified. Why not edit it? You are spreading misinformation.Shilling
Another year has passed and this answer is still wrong. Sorry, I am flagging this.Shilling

© 2022 - 2024 — McMap. All rights reserved.