Serializing and deserializing a bit field

Question

I've been handed a document that defines a set of messages that are transmitted and received over a serial communications channel. I'd like to take the incoming messages and deserialize them into objects, and serialize my outbound messages as well. The encoding over the wire is established and not changeable, and consists of various bitfields in the header and varying payloads, e.g.,

class Message{
int msg_num : 7
int dest_addr : 4
bool SRR : 1
bool IDE : 1
int source_addr : 6
//... and so on...
}

I took a look at using protobufs, but it appears that their varint method of encoding is established. I've also looked at boost-serialization, but based on what I've read so far, how the encoding is done there is not entirely clear.

So, a few questions:

• Can I use boost-serialization to convert my bytestream to objects?
• With a goal of not having to roll my own routines for serialization (a maintenance mess), is there a preferred mechanism for accomplishing my task (e.g., a custom boost-serialization Archive, another method I've not discovered)

Susy

Does "int msg_num : 7" mean the field is a 7 bit integer? – Drysalter

Yes. The ": num" indicates the length of the bitfield – Susy

I find this question difficult to answer, as only the packed representation is shown but not the unpacked message objects you would like to serialise and deserialise. – Cooperation

This could get really really messy and ugly if you are cross platform. For example Wintel to Sun Sparc. – Enkindle

Answer 1

I think you won't find an easy-to-use serializer that will match the custom protocol you are looking at. But it look the set of primitives you have (int, bool + size ) are simple enough to be able to write your own decoder/encoder. Simply generating C/C++ code based on the message received. It should be fairly simple task to generate a compilable code taking such description. It should be an automated generation done at the compile time - similar to what protobuf/Corba are doing.

Example: from the specification:

class Message{
    int msg_num : 7
    int dest_addr : 4
    bool SRR : 1
    bool IDE : 1
    int source_addr : 6
    //... and so on...
}

the converter could write a function with body similar to (abstract notation and assuming MSB):

Decoder:

m = new Message()
{
    long long val = 0
    for(int i=0; i<7; i++) {
        val <<= 8
        val += nextByte()    
    }
    m.msg_num = val
}
{
    long long val = 0
    for(int i=0; i<4; i++) {
        val <<= 8
        val += nextByte()    
    }
    m.dest_addr = val
}
{
    int val = nextByte()
    m.SRR = val
}
{
    int val = nextByte()
    m.IDE = val
}
{
    long long val = 0
    for(int i=0; i<6; i++) {
        val <<= 8
        val += nextByte()    
    }
    m.source_addr = val
}
// and so on

Encoder:

{
    long long val = m.msg_num
    for(int i=0;i<7;i++) {
        writeByte(val & 0xFF)
        val >>= 8
    }
}
{
    long long val = m.dest_addr
    for(int i=0;i<4;i++) {
        writeByte(val & 0xFF)
        val >>= 8
    }
}
....

That should be pretty easy to generate and the simplest way to make sure the encoding is custom.

Brandish

AFAIK the standard doesn't force implementation to use any type different from int for bit fields (even if you write in your code a different type, e.g. char or bool) and the standard also leaves it up to the implementation whether bit fields are ordered in memory MSB to LSB or the other way around. Because of that serializing byte by byte may be problematic with bit fields. – Rosabella

Sure. My answer was more an example, than solution - depending how exactly the bytes should be sent over the wire, the exact implementation should reflect the protocol. It was more to show the idea how it could be done: generating the conversion code should do the job. – Brandish

Answer 2

In case you're limited to single platform only (i.e. limited with single byte order), and Message is POD type, you can declare your message as primitive.

Otherwise, in case of boost.serialization at least, you'll have to write code, i.e. 'routines for serialization'. It supports byte order conversions, at least

[Edit] Wrong, it's not primitive, I'm lost in the depths of serialization docs

Wanigan