Class with types dependant on variadic templating

Question

I recently watched a video which inspired me to write my own neural network system, and I wanted to have the amount of nodes in the network be adjustable.

At first I achieved this at runtime by parsing an array of the numbers of nodes but I was wondering if I could do this at compile time instead. Here's an example of the kind of thing I was hoping to accomplish.

template<int FirstNodes, int SecondNodes, int... OtherNodes>
class Net
{
    tuple<Eigen::Matrix<float, FirstNodes, SecondNodes>, ...> m_weights;
    // More matricies with the values from the OtherNodes
};

As a more detailed example, Net<784, 16, 16, 10> n; n.m_weight should have type

tuple<Eigen::Matrix<float, 784, 16>,
    Eigen::Matrix<float, 16, 16>,
    Eigen::Matrix<float, 16, 10>>

From what I know about C++ and constexpr, this should be possible.

I should add that I was able to do

template<int FirstNodes, int SecondNodes, int... OtherNodes>
class Net
{
public:
    Net()
    {
        auto nodes = {FirstNodes, SecondNodes, OtherNodes...};

        auto i = nodes.begin();
        do 
        {
            // Eigen::Matrix<float, Dynamic, Dynamic>
            Eigen::MatrixXf m(*(i++), *i);
        } while (i+1 != nodes.end());
    }
};

But then I'm just using dynamic matricies again and that isn't what I was hoping for.

Any advice or working examples would be greatly appreciated.

Semiotic

by the way, the line m(*(i++), *i) invokes undefined behavior up to c++14, unspecified in c++17 – Wench

shameless self promoting you might find this repo interesting – Frolicsome

Answer 1

You want some sort of type transformation that given a list of N integers returns a tuple of N - 1 matrices. Here's a C++17 solution:

template <int A, int B, int... Is>
auto make_matrix_tuple()
{   
    if constexpr(sizeof...(Is) == 0)
    {
        return std::tuple<Eigen::Matrix<float, A, B>>{};
    }
    else
    {
        return std::tuple_cat(make_matrix_tuple<A, B>(), 
                            make_matrix_tuple<B, Is...>());
    }
}

live example on wandbox

---

In C++11, you can implement this type transformation recursively:

template <int... Is>
struct matrix_tuple_helper;

template <int A, int B, int... Rest>
struct matrix_tuple_helper<A, B, Rest...>
{
    using curr_matrix = Eigen::Matrix<float, A, B>;
    using type = 
        decltype(
            std::tuple_cat(
                std::tuple<curr_matrix>{},
                typename matrix_tuple_helper<B, Rest...>::type{}
            )
        );
};

template <int A, int B>
struct matrix_tuple_helper<A, B>
{
    using curr_matrix = Eigen::Matrix<float, A, B>;
    using type = std::tuple<curr_matrix>;
};

template <int... Is>
using matrix_tuple = typename matrix_tuple_helper<Is...>::type;

---

C++14 approach:

struct matrix_tuple_maker
{
    template <int A, int B, int C, int... Is>
    static auto get()
    {
        return std::tuple_cat(get<A, B>(), get<B, C, Is...>());
    }

    template <int A, int B>
    static auto get()
    {
        return std::tuple<Eigen::Matrix<float, A, B>>{};
    }
};

static_assert(std::is_same_v<
    decltype(matrix_tuple_maker::get<784, 16, 16, 10>()),
    std::tuple<Eigen::Matrix<float, 784, 16>,
               Eigen::Matrix<float, 16, 16>,
               Eigen::Matrix<float, 16, 10>>
    >);

Cropland

Answer 2

It seems to me that you need two list of integers, out of phase of 1.

If you define a trivial template integer container (in C++14 you can use a std::integer_sequence)

template <int...>
struct iList
 { };

you can define a base class as follows (sorry: used foo instead of Eigen::Matrix)

template <typename, typename, typename = std::tuple<>>
struct NetBase;

// avoid the first couple
template <int ... Is, int J0, int ... Js>
struct NetBase<iList<0, Is...>, iList<J0, Js...>, std::tuple<>>
   : NetBase<iList<Is...>, iList<Js...>, std::tuple<>>
 { };

// intermediate case
template <int I0, int ... Is, int J0, int ... Js, typename ... Ts>
struct NetBase<iList<I0, Is...>, iList<J0, Js...>, std::tuple<Ts...>>
   : NetBase<iList<Is...>, iList<Js...>,
             std::tuple<Ts..., foo<float, I0, J0>>>
 { };

// avoid the last couple and terminate
template <int I0, typename ... Ts>
struct NetBase<iList<I0>, iList<0>, std::tuple<Ts...>>
 { using type = std::tuple<Ts...>; };

and Net simply become (observe out of phase couple of integer lists)

template <int F, int S, int... Os>
struct Net : NetBase<iList<0, F, S, Os...>, iList<F, S, Os..., 0>>
 { };

The following is a full compiling example

#include <tuple>

template <int...>
struct iList
 { };

template <typename, int, int>
struct foo
 { };

template <typename, typename, typename = std::tuple<>>
struct NetBase;

// avoid the first couple
template <int ... Is, int J0, int ... Js>
struct NetBase<iList<0, Is...>, iList<J0, Js...>, std::tuple<>>
   : NetBase<iList<Is...>, iList<Js...>, std::tuple<>>
 { };

// intermediate case
template <int I0, int ... Is, int J0, int ... Js, typename ... Ts>
struct NetBase<iList<I0, Is...>, iList<J0, Js...>, std::tuple<Ts...>>
   : NetBase<iList<Is...>, iList<Js...>,
             std::tuple<Ts..., foo<float, I0, J0>>>
 { };

// avoid the last couple and terminate
template <int I0, typename ... Ts>
struct NetBase<iList<I0>, iList<0>, std::tuple<Ts...>>
 { using type = std::tuple<Ts...>; };

template <int F, int S, int... Os>
struct Net : NetBase<iList<0, F, S, Os...>, iList<F, S, Os..., 0>>
 { };

int main()
 {
   static_assert(std::is_same<
      typename Net<784, 16, 16, 10>::type, 
      std::tuple<foo<float, 784, 16>, foo<float, 16, 16>,
                 foo<float, 16, 10>>>{}, "!");
 }

Ceresin

Answer 3

Here is another C++14 solution. I consider it worth posting because it is non-recursive and readable.

#include <tuple>
#include <utility>

template<class, int, int> struct Matrix {};

template<int... matsizes, std::size_t... matinds>
constexpr auto make_net(
  std::integer_sequence<int, matsizes...>,
  std::index_sequence<matinds...>
) {
  constexpr int sizes[] = {matsizes...};
  return std::tuple< Matrix<float, sizes[matinds], sizes[1+matinds]>... >{};
}

template<int... matsizes>
constexpr auto make_net(
  std::integer_sequence<int, matsizes...> sizes
) {
  static_assert(sizes.size() >= 2, "");
  constexpr auto number_of_mats = sizes.size() - 1;
  return make_net(sizes, std::make_index_sequence<number_of_mats>{});
}

int main () {
  auto net = make_net(std::integer_sequence<int, 784, 16, 16, 10>{});
  using Net = decltype(net);

  static_assert(
    std::is_same<
      std::tuple<
        Matrix<float, 784, 16>,
        Matrix<float, 16, 16>,
        Matrix<float, 16, 10>
      >,
      Net
    >{}, ""
  );

  return 0;
}

Oscillograph