How to get reflection-like functionality in C, without x-macros

Question

Related to this question on Software Engineering about easily serializing various struct contents on demand, I found an article which uses x-macros to create struct metadata needed for "out of the box" struct serialization. I've also seen similar techniques for "smart enums", but it boils down to the same principle, getting a string representation of an enum, or a struct's field value by its name, or something similar.

However experienced C programmers on Stack Overflow state that the x-macros should be avoided as the "last resort":

• Generic enum to text lookup in C
• Nested macro iteration with C preprocessor
• How to access member of struct dynamically in C?

I could probably find many more related threads, but unfortunately I didn't bookmark them so this is just some Google-fu.

Perhaps the correct answer is something like Protocol Buffers? But why would creating struct definition in a different language (.proto definitions) and then running a build step to generate C files be preferable to using the built-in preprocessor for the same thing? And the issue is that these techniques still don't let me retrieve a single struct by name, I must share the same definition between two projects and keep them in sync.

So the question is then: If x-macros are "last resort", which approach for my problem (easily serializing various internal data when requested from a different device) would be "first resort", or anything before resorting to macro hell?

Pyramidon

Just stop thinking "generic" and work with a specific case. That was the conclusion I came up with for myself after years of having this dilemma. – Begay

@EugeneSh.: this is an actual project, we have a microcontroller operated device which can communicate with other devices, but one of the requirements is that we must be able to query its state (i.e. access the in-memory structs on demand). So far we just created different messages for each query, then we would parse the request and serializers each specific message type using a different hand-crafted function. So this approach seems like less work, meaning less room for errors. – Pyramidon

I have had a similar project requirement. So we have defined a special "telemetry" protocol which had some defined method of accessing specific types of data. For the most "generic" access we have had a special command for accessing specific memory. That's it. – Begay

You could look into preprocessor part of Boost. It may or may not work in C, but you could port some parts of it. With it it's possible to make a macro (something like ReflectEnum( (a , 1) (b , 2) )) which would generate both the enum and the reflection data. – Rebarbative

@Pyramidon You want to fiddle around in the memory of a running program remotely. There is already a fantastic generic tool that does this, including knowing struct fields, variable names, enum names, function names, etc. It's a debugger. gdb has a remote debugging protocol and as far as I remember it can talk over pretty much anything. I haven't used it for almost 20 years, but back then I used it from a PC to debug a kernel running on a Sparc, so it kind of matches what you're talking about. – Raptorial

@Pyramidon Of course, in actual real production I would definitely manually define each parameter to query, for security if nothing else, but as a generic "read any value from a running program" tool nothing beats a proper debugger. – Raptorial

Thanks, however this is a bare-metal ARM project, it doesn't run linux, so I am not sure if it's even possible to attach the debugger remotely? Serial + ethernet ports are used for the actual functionality, so they cannot be reserved for debugging, so I don't have a clue how this would even work in practice? – Pyramidon

Answer 1

With a bit of preprocessor magic taken from Boost we can make a macro able to generate reflectable enums.

I managed to construct a simple proof-of-concept implementation provided below.

---

First, the usage. Following:

ReflEnum(MyEnum,
    (first)
    (second , 42)
    (third)
)

Gets expanded to:

enum MyEnum
{
    first,
    second = 42,
    third,
};

const char *EnumToString_MyEnum(enum MyEnum param)
{
    switch (param)
    {
      case first:
        return "first";
      case second:
        return "second";
      case third:
        return "third";
      default:
        return "<invalid>";
    }
}

Thus a complete program could look like this:

#include <stdio.h>

/*
 * Following is generated by the below ReflEnum():
 *   enum MyEnum {first, second = 42, third};
 *   const char *EnumToString_MyEnum(enum MyEnum value) {}
*/
ReflEnum(MyEnum,
    (first)
    (second , 42)
    (third)
)

int main()
{
    enum MyEnum foo = second;
    puts(EnumToString_MyEnum(foo));  // -> "second"
    puts(EnumToString_MyEnum(43));   // -> "third"
    puts(EnumToString_MyEnum(9001)); // -> "<invalid>"
}

---

And here is the implementation itself.

It consists of two parts. The code itself and a preprocessor magic header shamelessly ripped off from Boost.

The code:

#define ReflEnum_impl_Item(...) PPUTILS_VA_CALL(ReflEnum_impl_Item_, __VA_ARGS__)(__VA_ARGS__)
#define ReflEnum_impl_Item_1(name)        name,
#define ReflEnum_impl_Item_2(name, value) name = value,

#define ReflEnum_impl_Case(...) case PPUTILS_VA_FIRST(__VA_ARGS__): return PPUTILS_STR(PPUTILS_VA_FIRST(__VA_ARGS__));

#define ReflEnum(name, seq) \
    enum name {PPUTILS_SEQ_APPLY(seq, ReflEnum_impl_Item)}; \
    const char *EnumToString_##name(enum name param) \
    { \
        switch (param) \
        { \
            PPUTILS_SEQ_APPLY(seq, ReflEnum_impl_Case) \
            default: return "<invalid>"; \
        } \
    }

It shouldn't be too hard to extend the code to support string->enum conversion; ask in the comments if you're not sure.

The magic:

Note that the preprocessor magic has to be generated by a script, and you have to choose a maximum enum size when generating it. The generation is easy and left as an exercise to the reader.

Boost defaults the size to 64, the code below was generated for size 4.

#define PPUTILS_E(...) __VA_ARGS__

#define PPUTILS_VA_FIRST(...) PPUTILS_VA_FIRST_IMPL_(__VA_ARGS__,)
#define PPUTILS_VA_FIRST_IMPL_(x, ...) x

#define PPUTILS_PARENS(...) (__VA_ARGS__)
#define PPUTILS_DEL_PARENS(...) PPUTILS_E __VA_ARGS__

#define PPUTILS_CC(a, b) PPUTILS_CC_IMPL_(a,b)
#define PPUTILS_CC_IMPL_(a, b) a##b

#define PPUTILS_CALL(macro, ...) macro(__VA_ARGS__)

#define PPUTILS_VA_SIZE(...) PPUTILS_VA_SIZE_IMPL_(__VA_ARGS__,4,3,2,1,0)
#define PPUTILS_VA_SIZE_IMPL_(i1,i2,i3,i4,size,...) size

#define PPUTILS_STR(...) PPUTILS_STR_IMPL_(__VA_ARGS__)
#define PPUTILS_STR_IMPL_(...) #__VA_ARGS__

#define PPUTILS_VA_CALL(name, ...) PPUTILS_CC(name, PPUTILS_VA_SIZE(__VA_ARGS__))

#define PPUTILS_SEQ_CALL(name, seq) PPUTILS_CC(name, PPUTILS_SEQ_SIZE(seq))

#define PPUTILS_SEQ_DEL_FIRST(seq) PPUTILS_SEQ_DEL_FIRST_IMPL_ seq
#define PPUTILS_SEQ_DEL_FIRST_IMPL_(...)

#define PPUTILS_SEQ_FIRST(seq) PPUTILS_DEL_PARENS(PPUTILS_VA_FIRST(PPUTILS_SEQ_FIRST_IMPL_ seq,))
#define PPUTILS_SEQ_FIRST_IMPL_(...) (__VA_ARGS__),

#define PPUTILS_SEQ_SIZE(seq) PPUTILS_CC(PPUTILS_SEQ_SIZE_0 seq, _VAL)
#define PPUTILS_SEQ_SIZE_0(...) PPUTILS_SEQ_SIZE_1
#define PPUTILS_SEQ_SIZE_1(...) PPUTILS_SEQ_SIZE_2
#define PPUTILS_SEQ_SIZE_2(...) PPUTILS_SEQ_SIZE_3
#define PPUTILS_SEQ_SIZE_3(...) PPUTILS_SEQ_SIZE_4
#define PPUTILS_SEQ_SIZE_4(...) PPUTILS_SEQ_SIZE_5
// Generate PPUTILS_SEQ_SIZE_i
#define PPUTILS_SEQ_SIZE_0_VAL 0
#define PPUTILS_SEQ_SIZE_1_VAL 1
#define PPUTILS_SEQ_SIZE_2_VAL 2
#define PPUTILS_SEQ_SIZE_3_VAL 3
#define PPUTILS_SEQ_SIZE_4_VAL 4
// Generate PPUTILS_SEQ_SIZE_i_VAL

#define PPUTILS_SEQ_APPLY(seq, macro) PPUTILS_SEQ_CALL(PPUTILS_SEQ_APPLY_, seq)(macro, seq)
#define PPUTILS_SEQ_APPLY_0(macro, seq)
#define PPUTILS_SEQ_APPLY_1(macro, seq) PPUTILS_CALL(macro, PPUTILS_SEQ_FIRST(seq))
#define PPUTILS_SEQ_APPLY_2(macro, seq) PPUTILS_CALL(macro, PPUTILS_SEQ_FIRST(seq)) PPUTILS_SEQ_CALL(PPUTILS_SEQ_APPLY_, PPUTILS_SEQ_DEL_FIRST(seq))(macro, PPUTILS_SEQ_DEL_FIRST(seq))
#define PPUTILS_SEQ_APPLY_3(macro, seq) PPUTILS_CALL(macro, PPUTILS_SEQ_FIRST(seq)) PPUTILS_SEQ_CALL(PPUTILS_SEQ_APPLY_, PPUTILS_SEQ_DEL_FIRST(seq))(macro, PPUTILS_SEQ_DEL_FIRST(seq))
#define PPUTILS_SEQ_APPLY_4(macro, seq) PPUTILS_CALL(macro, PPUTILS_SEQ_FIRST(seq)) PPUTILS_SEQ_CALL(PPUTILS_SEQ_APPLY_, PPUTILS_SEQ_DEL_FIRST(seq))(macro, PPUTILS_SEQ_DEL_FIRST(seq))
// Generate PPUTILS_SEQ_APPLY_i

Rebarbative

Cool, this is essentially x-macros, but usage is pretty straightforward. – Pyramidon

Answer 2

The "first resort" would typically be one of:

• Group all your data in tables made of arrays/structs, preferably read-only ones, as in the first linked example. The table index is used as the search key to keep the data together ("primary key" to use RDBMS terms). This is fast and readable, but care must be taken during maintenance.

• Group your data according to some OO design. You can use opaque pointers and functions pointers to achieve private encapsulation and polymorphism. When used correctly, this can give state of the art program design. But at the same time it can be somewhat burdensome to write. And if you can't use dynamic memory allocation (embedded systems) then you have to invent a memory pool per class. Works best for more complex "ADT"-like containers and for API design.

That being said, X-macros are somewhat acceptable as long as you don't assume that every reader is familiar with them. I would therefore leave some comments about how the macro lists works, how they expand when used, and how they should be maintained.

From the linked code example, the line #define X(dir) {dir, #dir} should perhaps be commented more properly like this:

/*
  Create a temporary X-macro that expands the DIRECTION_LIST, to form 
  an array initialization list. The format will be:

  {north, "north"},  
  {south, "south"},
  ...
*/
#define X(dir) {dir, #dir}
  DIRECTION_LIST
#undef X

Breadboard