Recursive explicit template instantiation to export symbols for a library

Question

In my previous question I asked is recursive explicit template instantiation possible. I saw that it is indeed possible; however, this instantiation turns out to be effective locally only, the symbols of the recursively instantiated template are not exported to the object file and thus do not appear in the (shared) library. So I ask question here more precisely as in my previous post:

Given a template like

template<int dim> class Point { ... };

this template can be instantiated explicitly like

template class Point<0>;
template class Point<1>;
template class Point<2>;
template class Point<3>;

which exports the symbols of Point<0>, ..., Point<3> into the object file of the current translation unit. Instead of instantiating every template individually like above, I would like to instantiate them recursively with just one call.

Any solutions that achieves this is fine, be it in the style of template meta-programming, via a helper class like

template class RecursiveInstantiate<Point, 3>;

or via the preprocessor. Here I looked into the boost preprocessor library, which seems to have some loop constructs. However, I never used the boost preprocessor library (any advice is appreciated) but on a first glance I am skeptical if the loops can be used together with an explicit template instantiation.

Any advice, also an explanation why it is impossible what I want to achieve is appreciated.

---

In fact I am interested in generalizing this for classes with multiple template parameters likeNode<int i1,int i2,int i3> for all combination of i1,i2,i3 in {0,1,2,3}. But I hope to be able to work out this second part by myself. As usual I want to use the explicit instantiations to speed up the compilation times by only defining the templates in one translation unit, thus I need the methods to template to be exported in the object file.

I hope for a compiler independent solution, but if that is not possible I need it for Linux with g++/clang.

---

See below for a survey of the solutions that I got and the final solutions that I made out of this.

Anthony

I never had problems externalizing templates by explicit instantiation with g++/mingw. Possibly you've put those into an unnamed namespace? (See also #7182859 , the section called "Template definitions in different units of translation") – Rust

@phresnel: do you mean me with @Trilinos below? I'm not sure if I understand your question. The explicit instantiation template class Point<0>; works perfectly fine, just the recursive explicit instantiation, via template meta-programming (see the above linked post) this did not export symbols in the object file. – Anthony

Doing a survey of the different solutions and the one you retained is a good idea, but you should probably put that in an answer of its own instead of "polluting" the original question (plus, you might get some additional reputation since you provide interesting information). – Gainey

@Luc Touraill: you are right, I changed it accordingly. – Anthony

Answer 1

Seems like a job for Boost.Preprocessor:

#include <boost/preprocessor/repetition/repeat.hpp>

#define INSTANTIATE(_, n, type) template class type<n>;

BOOST_PP_REPEAT(3, INSTANTIATE, Point)

Of course, you can embed this into another macro to make it look better:

#define INSTANTIATE(_, n, type) template class type<n>;
#define INSTANTIATE_N(n, type) BOOST_PP_REPEAT(n, INSTANTIATE, type)

INSTANTIATE_N(3, Point)

Gainey

This looks like what I need. BOOST_PP_REPEAT(4, INSTANTIATION, Point) is exactly equivalent to the above four lines. I will try to generalize this now for multiple integer argument, Node<int i1,int i2,int i3> and report a solution once I found it. – Anthony

Answer 2

Survey of the Solutions

Via Boost Preprocessor (BOOST_PP_REPEAT)

See the solution proposed by Luc Touraille

#define INSTANTIATE(_, n, type) template class type<n>;
#define INSTANTIATE_N(n, type) BOOST_PP_REPEAT(n, INSTANTIATE, type)
INSTANTIATE_N(4, Point)

Via Template Metaprogramming

See the solution and the very good explanation by phresnel below. This sounds like the preferable approach. Unfortunately explicit instantiation can only be used at the global level (a language limitation) and thus it cannot be used recursively. If the instantiation is done implicitely (see my previos question) then only those symbols are defined (and thus exported to the object file) that are actually used, i.e. you need to define each symbol of the class once (with a dummy).

This is not a very nice approach, but it avoids possibly nasty problem (and portability issues) of the preprocessor solutions.

Via Boost Preprocessor for multiple parameters.

Finally I also dived deeper into the Boost preprocessing library and tried to extend the result to create instantiations of the form

  template class Node< int , 0 , 0 >;
  template class Node< int , 1 , 0 >;
  template class Node< int , 1 , 1 >;
  template class Node< int , 2 , 0 >;
  template class Node< int , 2 , 1 >;
  template class Node< int , 2 , 2 >;
  template class Node< float , 0 , 0 >;
  template class Node< float , 1 , 0 >;
  template class Node< float , 1 , 1 >;
  template class Node< float , 2 , 0 >;
  template class Node< float , 2 , 1 >;
  template class Node< float , 2 , 2 >;

So a template Node<Scalar, pdim, ldim> with Scalar and arithmetic type pdim and integer and ldim <= pdim another integer.

I only was able to extend the approach using BOOST_PP_REPEAT for templates with two arguments as BOOST_PP_REPEAT can currently be only nested 3 levels deep. Two level for the template parameters and one level for BOOST_PP_ENUM was the maximum I could achieve with this technique. It is more flexible to use the file iteration technique which supports up to 5 levels.

This code I was able to generate with the code

#define INTTOTYPE0 (int, (float, (double, _)))
#define NUM_TEMPLATE_ARGS 3
#define MAX_TEMPLATE_PARAM0 2
#define MAX_TEMPLATE_PARAM1(i0) 2
#define MAX_TEMPLATE_PARAM2(i0, i1) i1
#define CLASSNAME Node 

#include "util/templateRecInstant.h"

the four instantiations for the Point class in the question can be generated via

#define NUM_TEMPLATE_ARGS 1
#define MAX_TEMPLATE_PARAM0 3
#define CLASSNAME Point 

#include "util/templateRecInstant.h"

Both methods are achieved with the file "util/templateRecInstant.h" with the contents

#if !BOOST_PP_IS_ITERATING

#define MY_FILE "util/templateRecInstant.h"

#include <boost/preprocessor/iteration/iterate.hpp>
#include <boost/preprocessor/list/at.hpp>

#ifndef NUM_TEMPLATE_ARGS
#error need to define NUM_TEMPLATE_ARGS
#endif

#ifndef MAX_TEMPLATE_PARAM0
#error need to specify MAX_TEMPLATE_PARAM0,  MAX_TEMPLATE_PARAM1, ..., up tp NUM_TEMPLATE_ARGS
#endif

#ifndef DEFAULT_INTTOTYPE
#define DEFAULT_INTTOTYPE (0, (1, (2, (3, (4, (5, (6, (7, (8, (9, _))))))))))
#endif

#ifndef INTTOTYPE0
#define INTTOTYPE0 DEFAULT_INTTOTYPE
#endif
#ifndef INTTOTYPE1
#define INTTOTYPE1 DEFAULT_INTTOTYPE
#endif
#ifndef INTTOTYPE2
#define INTTOTYPE2 DEFAULT_INTTOTYPE
#endif
#ifndef INTTOTYPE3
#define INTTOTYPE3 DEFAULT_INTTOTYPE
#endif

#if NUM_TEMPLATE_ARGS > 0
   #define BOOST_PP_ITERATION_PARAMS_1 (3, (0, \
         MAX_TEMPLATE_PARAM0, MY_FILE ))
   #include BOOST_PP_ITERATE()
#endif

#if NUM_TEMPLATE_ARGS == 0
  template class CLASSNAME< \
  >;
#endif

#undef MY_FILE
#undef NUM_TEMPLATE_ARGS 
#undef CLASSNAME 
#undef MAX_TEMPLATE_PARAM0
#undef MAX_TEMPLATE_PARAM1
#undef MAX_TEMPLATE_PARAM2
#undef MAX_TEMPLATE_PARAM3
#undef INTTOTYPE0
#undef INTTOTYPE1
#undef INTTOTYPE2
#undef INTTOTYPE3


#elif BOOST_PP_ITERATION_DEPTH() == 1

#if NUM_TEMPLATE_ARGS > 1
   #define BOOST_PP_ITERATION_PARAMS_2 (3, (0, \
         MAX_TEMPLATE_PARAM1(BOOST_PP_FRAME_ITERATION(1)), \
         MY_FILE))
   #include BOOST_PP_ITERATE()
#endif

#if NUM_TEMPLATE_ARGS == 1
  template class CLASSNAME< \
  BOOST_PP_LIST_AT( INTTOTYPE0, BOOST_PP_FRAME_ITERATION(1)) \
  >;
#endif

#elif BOOST_PP_ITERATION_DEPTH() == 2

#if NUM_TEMPLATE_ARGS > 2
   #define BOOST_PP_ITERATION_PARAMS_3 (3, (0, \
         MAX_TEMPLATE_PARAM2(BOOST_PP_FRAME_ITERATION(1) \
                           , BOOST_PP_FRAME_ITERATION(2) \
           ), \
         MY_FILE))
   #include BOOST_PP_ITERATE()
#endif

#if NUM_TEMPLATE_ARGS == 2
  template class CLASSNAME< \
    BOOST_PP_LIST_AT( INTTOTYPE0, BOOST_PP_FRAME_ITERATION(1)) \
  , BOOST_PP_LIST_AT( INTTOTYPE1, BOOST_PP_FRAME_ITERATION(2)) \
  >;
#endif

#elif BOOST_PP_ITERATION_DEPTH() == 3

#if NUM_TEMPLATE_ARGS > 3
   #define BOOST_PP_ITERATION_PARAMS_4 (3, (0, \
         MAX_TEMPLATE_PARAM3(BOOST_PP_FRAME_ITERATION(1) \
                           , BOOST_PP_FRAME_ITERATION(2) \
                           , BOOST_PP_FRAME_ITERATION(3) \
           ), \
         MY_FILE))
   #include BOOST_PP_ITERATE()
#endif

#if NUM_TEMPLATE_ARGS == 3
  template class CLASSNAME< \
    BOOST_PP_LIST_AT( INTTOTYPE0, BOOST_PP_FRAME_ITERATION(1)) \
  , BOOST_PP_LIST_AT( INTTOTYPE1, BOOST_PP_FRAME_ITERATION(2)) \
  , BOOST_PP_LIST_AT( INTTOTYPE2, BOOST_PP_FRAME_ITERATION(3)) \
  >;
#endif

#elif BOOST_PP_ITERATION_DEPTH() == 4

#if NUM_TEMPLATE_ARGS == 4
  template class CLASSNAME< \
    BOOST_PP_LIST_AT( INTTOTYPE0, BOOST_PP_FRAME_ITERATION(1)) \
  , BOOST_PP_LIST_AT( INTTOTYPE1, BOOST_PP_FRAME_ITERATION(2)) \
  , BOOST_PP_LIST_AT( INTTOTYPE2, BOOST_PP_FRAME_ITERATION(3)) \
  , BOOST_PP_LIST_AT( INTTOTYPE3, BOOST_PP_FRAME_ITERATION(4)) \
  >;
#endif

#if NUM_TEMPLATE_ARGS > 4
#error "NUM_TEMPLATE_ARGS > 4 is not supported (limitation by boost)"
#endif

#endif

If you stumble upon this problem, feel free to use the above code. To generate this code I used 1 as a reference.

Anthony

Answer 3

The problem you are facing is: Indeed RecursiveInstantiate is instantiated completely. But the "enclosed" types are only instantiated to the degree you are using them inside RecursiveInstantiate. I.e., if you are not calling some function Point::xxx, then xxx will not be instantiated, which is the usual behaviour. You would need a syntax for explicit instantiation inside classes or functions.

I think you can't do full instantiation on all contained references, you can explicitly instantiate, but not implicitly.

E.g.,

template <int D> struct Foo {
    static void print();
};

#include <iostream>

int main () {
    Foo<0>::print();
    Foo<1>::print();
    Foo<2>::print();
}

then

#include <iostream>

// Our Foo we'd like to instantiate explicitly and recursively.
template <int D> struct Foo {
    static void print() { std::cout << D << std::endl; }
};


template <int D>
static void instantiate () {
    // refer to everything that should be exported
    Foo<D>::print();
}

template <int D>
struct loop { 
    static void print_inst () { 
        instantiate<D>(); 
        loop<D-1>::print_inst();
    }
};

template <>
struct loop<0> { 
    static void print_inst () { 
        instantiate<0>();
    }
};

template struct loop<2>;

in function instantiate, you would refer to everything that should be exported.

You have to weigh out the best compromise, possibly my approach is the shortest, possibly manual typedown (template class Foo<0>; template class Foo<1> ...) is shorter because you have many member functions, possibly you consider the BOOST way portable enough, even if the standard does not sanction it.

It would be interesting if there would be a syntax for explicit instantiation in non-global contexts.

Rust

This is a nice explanation of what is going on. I will try to make the Boost preprocessor solution working, as I have more than one class and don't want to list all members. Then I will accept your and Luc's answer. – Anthony