Creating an array initializer from a tuple or variadic template parameters
Asked Answered
A

1

-3

I want to represent the description of a persistent memory layout (e.g. Flash or EEPROM device) statically embedded in the program code (preferably in the ROM section), from a set of variadic template parameters, where the necessary offsets are automatically calculated at compile time.

The goal is to create an appropriate array initializer, that can be iterated at runtime, without the restrictions you'll get with std::get(std::tuple), which requires compile time indexing.


1st approach

I have created a simple data item descriptor class that binds a particular ID (should be provided as an enum type by the client), to the data layout (offset and size):

template
    < typename ItemIdType
    >
struct DataItemDescBase
{
    const ItemIdType id;
    const std::size_t size;
    const std::size_t offset;

    DataItemDescBase(ItemIdType id_, std::size_t size_, std::size_t offset_)
    : id(id_)
    , size(size_)
    , offset(offset_)
    {
    }

    DataItemDescBase(const DataItemDescBase<ItemIdType>& rhs)
    : id(rhs.id)
    , size(rhs.size)
    , offset(rhs.offset)
    {
    }
};

Clients should use this class that binds to a particular data type and offset:

template
    < typename DataType
    , typename ItemIdType
    >
struct DataItemDesc
: public DataItemDescBase<ItemIdType>
{
    typedef DataType DataTypeSpec;

    DataItemDesc(ItemIdType id_, std::size_t offset_ = 0)
    : DataItemDescBase(id_,sizeof(DataTypeSpec),offset_)
    {
    }

    DataItemDesc(const DataItemDesc<DataType,ItemIdType>& rhs)
    : DataItemDescBase(rhs)
    {
    }
};

Finally I want to use a std::array to store the concrete data layouts:

const std::array<DataItemDescBase<ItemIdType>,NumDataItems> dataItemDescriptors;

For the client I'd like to provide an array initializer from either a std::tuple or a variadic template parameter list, thus the offsets of subsequent array elements are automatically calculated from offset + size of the previous element at compile time.

What currently works is that a client can use the following code to initialize the array:

namespace
{
    static const std::array<DataItemDescBase<DataItemId::Values>,4> theDataLayout =
        { { DataItemDesc<int,DataItemId::Values>
             ( DataItemId::DataItem1 )
        , DataItemDesc<short,DataItemId::Values>
             ( DataItemId::DataItem2
             , sizeof(int))
        , DataItemDesc<double,DataItemId::Values>
             ( DataItemId::DataItem3
             , sizeof(int) + sizeof(short))
        , DataItemDesc<char[10],DataItemId::Values>
             ( DataItemId::DataItem4
             , sizeof(int) + sizeof(short) + sizeof(double))
        } };
}

But letting the clients calculate the offsets manually looks error prone and tedious.

TL;DR; Is it posssible to calculate the offsets at compile time, and if yes can you give me a sketch how please?


2nd approach

I've tried the proposal from @Yakk's answer and just introduced a data aware base class for ProcessedEntry like this:

template<typename Key>
struct ProcessedEntryBase {
    const Key id;
    const std::size_t offset;
    const std::size_t size;

    ProcessedEntryBase(Key id_ = Key(), std::size_t offset_ = 0, std::size_t size_ = 0)
    : id(id_)
    , offset(offset_)
    , size(size_) {
    }

    ProcessedEntryBase(const ProcessedEntryBase<Key>& rhs)
    : id(rhs.id)
    , offset(rhs.offset)
    , size(rhs.size) {
    }
};

template<typename Key, Key identifier, typename T, std::size_t Offset>
struct ProcessedEntry
: public ProcessedEntryBase<Key> {
    ProcessedEntry()
    : ProcessedEntryBase<Key>(identifier,Offset,sizeof(T)) {
    }
};

I had intended to use a LayoutManager base class that could be inherited and provided with the concrete layout from a constructor parameter:

template<typename Key, std::size_t NumEntries>
class LayoutManager {
public:
    typedef std::array<ProcessedEntryBase<Key>,NumEntries> LayoutEntriesArray;

    const LayoutEntriesArray& layoutEntries;

    // ...
    // methods to lookup particular entries by id
    // ...

protected:
    LayoutManager(LayoutEntriesArray layoutEntries_)
    : layoutEntries(layoutEntries_) {
    }
};

Client code

ConcreteLayout.hpp;

struct DataItemId {
    enum Values {
        DataItem1 ,
        DataItem2 ,
        DataItem3 ,
        DataItem4 ,
    };
};

class ConcretePersistentLayout
: public LayoutManager<DataItemId::Values,4> {
public:
    ConcretePersistentLayout();
};

ConcreteLayout.cpp:

Layout< DataItemId::Values
    , Entry< DataItemId::Values, DataItemId::DataItem1, int>
    , Entry< DataItemId::Values, DataItemId::DataItem2, short >
    , Entry< DataItemId::Values, DataItemId::DataItem3, double >
    , Entry< DataItemId::Values, DataItemId::DataItem4, char[10] >
    >::type theDataLayout; // using like this gives me a compile error, 
                           // because I have no proper type 'prepend' 
                           // I'd guess
}

ConcretePersistentLayout::ConcretePersistentLayout()
: LayoutManager<DataItemId::Values,4>(theDataLayout) 
  // ^^^^^^ Would this work to 'unpack' the tuple?
{
}

I want to loosely couple an accessor class with the LayoutManager that that takes the id, calculates the persistent memory device address, fetches the data and casts to the data type bound to the key/id. I planned to let the client specify the key/data type bindings explicitly, thus static checks can be done on accessor functions.


Finally

I have something in production now, based on @Yakk's extended answer after that first round of asking for more clarification.


Also in regard to comments:

  1. In this case I'm aware about the slicing problem, and it's guaranteed that the derived (template) classes stored in the std::array<ProcessedEntryBase> won't add any more data members or such. Functional binding (casting) is done separately.

  2. The indices trick proposed, also was a good hint about how to unpack variadic template parameters at runtime for indexed access, iterating.

Atalaya answered 15/8, 2013 at 11:32 Comment(5)
You can't use DataItemDescBase<ItemIdType> in a collection (like std::array) if you want to store inherited classes in it (like DataItemDesc). See e.g. What is the slicing problem or the Wikipedia article on object slicing.Popularity
I have a suspicion that your question may find its answer in this indices trickAlgin
@JoachimPileborg But the inherited class doesn't provide any own data or virtual methods or such. It's just meant as a wrapper to initialize the base class. Could be done with a completely decoupled wrapper class also.Monson
In that case it should work okay, as long as you are aware of the (potential) problem.Popularity
Replace your "only used at construction" class with a function or functor: there is little advantage to your design if you do not intend on storing such instances, or manipulating stuff based on the type (which gives me an idea).Togs
T
3

In order for compile time accumulation to occur, you have to have a compile time sequence.

An easy way to do this would be to use variadic templates. Each entry would be an identifier and a size of a particular element, or the identifier and type of a particular element.

The top level bundle of entries would be a Layout:

template<std::size_t offset, typename Key, typename... Entries>
struct LayoutHelper {
  typedef std::tuple<> type;
};
template<typename Key, typename... Entries>
struct Layout:LayoutHelper<0, Key, Entries...> {};

Each entry would be:

template<typename Key, Key identifier, typename Data>
struct Entry {};

Then, we do something like this:

template<typename Key, Key identifier, typename Data, std::size_t Offset>
struct ProcessedEntry {};

template<std::size_t offset, typename Key, Key id0, typename D0, typename... Entries>
struct LayoutHelper<offset, Key, Entry<Key, id0, D0>, Entries...>
{
    typedef typename prepend
        < ProcessedEntry< Key, id0, D0, offset >
        , typename LayoutHelper<offset+sizeof(D0), Key, Entries...>::type
        >::type type;
};

Use would look like:

Layout< FooEnum, Entry< FooEnum, eFoo, char[10] >, Entry< FooEnum, eFoo2, double > > layout;

which, after writing or finding a prepend that takes an element and a tuple, and prepends the element at the front, would mean that Layout<blah>::type would contain a tuple that describes the layout of your data.

template<typename T, typename Pack>
struct prepend;
template<typename T, template<typename...>class Pack, typename... Ts>
struct prepend<T, Pack<Ts...>> {
  typedef Pack<T, Ts...> type;
};
// use: prepend<int, std::tuple<double>::type is std::tuple<int, double>
// this removes some ::type and typename boilerplate, if it works in your compiler:
template<typename T, typename Pack>
using Prepend = typename prepend<T, Pack>::type;

You'd then unpack that tuple into a std::array if you wanted. You'd use the indexes trick to do this (there are many examples on stack overflow that use this same trick in different ways).

Or, you could take your ProcessedEntry and add in methods to access data, then write a Key searching compile-time program that walks the tuple, looking for the matching Key, and then returns the offset and size (or even type) as compile time code. Maybe take an array<N, unsigned char> as an argument and do the reintepret_cast, returning a reference-to-data.

Removing the repeated FooEnum would be good via using aliases.

Togs answered 15/8, 2013 at 15:58 Comment(12)
That looks pretty promising for what I want to do, I'll try this out later.Monson
'You'd then unpack that tuple into a std::array' could you elaborate a bit on this. I don't get what array element type to use for this. Providing a base type for ProcessedEntry that just keeps the data for id, offset and size. I need to access the particular layout entries at runtime by id to let an appropriate accessor class access a specific layout entry and calculate the address and data size to read from the concrete persistent memory device (or do I need it runtime calculated, not sure anymore?).Monson
Also don't fully get how to 'writing or finding a prepend that takes an element and a tuple, and prepends the element at the front'. Finding an appropriate one would be preferable of course.Monson
@g-makulik will take a bit, hard to type code on phone. Bug me if no more within 24 hr.Togs
'will take a bit, hard to type code on phone' Ha! You f***ing Nerds ;o), s.o. should provide an app for that! No, problem need to have an appropriate solution next week or so. Have a look at my updates then. Actually I'm on a few days off holiday ;o) ...Monson
I'm sure now! I need to access the address information at runtime and from input demands, thus 'unpacking to a static const array' is the interesting part!Monson
Bugging now! I've been trying answer ant other suggestions from this SO thread: Convert std::tuple to std::array C++11, but without success. I've got errors saying that I'm trying to use incomplete types in the conversion (relating Key template parameter). I'd guess I need to have some specialization for the case, or inject Key to the converter somehow. I don't really get where to approach.Monson
Sorry, didn't notice your update. Will proceed with that first.Monson
OK! I have used your prepend implementation and this works fine. But I still have trouble to unpack the resulting tuple of Layout to be used as array initializer.Monson
@g-makulik the easiest way would be to mix the standard indexes trick with some functions that extract the compile-time data and produce a run-time struct to store in the std::array if you want to go that way. Start with a template function that maps an ProcessedEntry<IdType, id, Data, offset> to a struct containing id and offset: call it get_array_entry. Next, use indexes trick to unpack the tuple with get_array_entry(get<I>(processed_tuple))..., and use that to construct the std::array< array_entry<IdType>>.Togs
Note, however, the std::array version is less efficient at runtime than one that directly uses an IdType and the std::tuple<ProcessedEntry<...>> and the std::array<char, ???> raw storage (or char[] raw storage pointer) to return Data&: but the more efficient one is harder to write. I would definitely write the "make the array of offsets" version first before trying the harder one that does the offset lookup at compile time.Togs
Hi and THX for your explanations and guidance. At least, to 'unpack' the tuple to an array I've been using a 'brutal' reinterpret_cast to std::array<ProcessedEntryBase<Key>,std::tuple_size<Layout<...>::type>::value> and this works pretty well for me. All the other methods I've seen to unpack to arrays involve memory allocation somehow (which I don't want).Monson

© 2022 - 2024 — McMap. All rights reserved.