What type of framing to use in serial communication

Question

In a serial communication link, what is the prefered message framing/sync method?

• framing with SOF and escaping sequences, like in HDLC?
• relying on using a header with length info and CRC?

It's an embedded system using DMA transfers of data from UART to memory. I think the framing method with SOF is most attractive, but maybe the other one is good enough?

Does anyone has pros and cons for these two methods?

Polymer

Your use of "frame" in the context of a UART is incorrect and/or misleading. A UART will only frame each individual character/byte. You are misusing "frame" for what should be called a message, packet, or datagram. Or preface it with an adjective, e.g. message frame. More often, the term I've seen is 'message format'. – Bastille

Answer 1

Following based on UART serial experience, not research.

I have found fewer communication issues when the following are included - or in other words, do both SOF/EOF and (length - maybe)/checkcode. Frame:

1. SOFrame
2. (Length maybe)
3. Data (address, to, from, type, sequence #, opcode, bytes, etc.)
4. CheckCode
5. EOFrame

Invariably, the received "fames" include:

1. good ones - no issues.
2. Corrupt due to sender not sending a complete message (it hung, powered down, or rump power-on transmission) (receiver should timeout stale incomplete messages.)
3. Corrupt due to noise or transmission interference. (byte framing errors, parity, incorrect data)
4. Corrupt due to receiver starting up in the middle of a sent message or missing a few bytes due to input buffer over-run.
5. Shared bus collisions.
6. Break - is this legit in your system?

Whatever framing you use, insure it is robust to address these messages types, promptly validate #1 and rapidly identifying 2-5 and becoming ready for the next frame.

SOF has the huge advantage of it it easy to get started again, if the receiver is lost due to a previous crap frame, etc.

Length is good, but IMHO the least useful. It can limit through-put, if the length needs to be at the beginning of a message. Some low latency operations just do not know the length before they are ready to begin transmitting.

CRC Recommend more than 2-byte. A short check-code does not improve things enough for me. I'd rather have no check code than a 1-byte one. If errors occur from time-to-time only be caught by the check-code, I want something better than a 2-byte's 99.999% of the time, I like a 4-byte's 99.99999997%

EOF so useful!

BTW: If your protocol is ASCII (instead of binary), recommend to not use cr or lf as EOFrame. Maybe only use them out-of-frame where they are not part of a message.

BTW2: If your receiver can auto-detect the baud, it saves on a lot of configuration issues.

BTW3: A sender could consider sending a "nothing" byte (before the SOF) to insure proper SOF syncing.

Flyweight

Missed-match baud, bits per word, parity, unexpected signal levels also causes issues. – Flyweight

"Length is good, but IMHO the least useful" -- Seems like a perspective that is only concerned about text rather than binary data. For a variable-length datagram that has no length specifier, every byte of the binary data would have to be scanned for the possible reserved byte values (i.e. SOF, EOF). And there has to be a method to allow the data to use these reserved byte values (e.g. an escape prefix?). Otherwise you risk false message alignment. – Bastille

@Bastille - a valid concern. I have no text focus. In effect with a binary payload, we could use 7-bit binary or escape for the bytes that match SOF, EOF, and the escape byte itself. IAC, to re-iterate, a length, up front, implies the entire data packet needs to be determined (or fixed) before sending anything - increasing latency. Serial protocol simple lacks any binding of multiple bytes. We must form one and IMO, a SOF/EOF is best and a length is redundant with EOF. – Flyweight

"a length, up front, implies ..." -- In most situations, that is not an issue. IMO initiating transmission before the message is constructed is bizarre. "we could use 7-bit binary" -- For data that is originally 8-bit bytes? You want to add all that pre- and post-processing? "escape for the bytes that match ..." -- Sure, that's a common solution, but affects throughput, maybe severely. You seem to fail to grasp that such a scheme to compensate for omitting length is mandatory for binary data, and will typically end up costing more in processing overhead and reduced throughput. – Bastille

@Bastille "IMO initiating transmission before the message is constructed is bizarre" This is not bizarre in communication cases where the data coming in is itself is a serial stream of indeterminate length. Example RFID - ISO 15693 lacks a length field and is binary. To form the length before conversion of that RF communication to a serial packet would require buffering, perhaps 1000s of bytes. That is costly in an embedded application and most negative: increases latency. – Flyweight

@Bastille The "we could use 7-bit binary" idea to encode 8-bit data simple means to take 7 bites of 8-bit binary data and send as 8 serial bytes. Using a bit to distinguish data from control bytes. Pre/post processing does not severely affect serial communication in a UART/RS-232 case. These serial communication rates range from 9600 baud to a few Mbaud as in bits/second. It is trivial, even for small embedded devices to handle that rate, even if it had to bit-bang out the data. – Flyweight

@Bastille Rather than addressing this personally as in “You seem to fail to grasp”, another approach would be to seek understanding of how to implement a binary communication that lack a length field. – Flyweight

Your alleged example of ISO 15693 doesn't apply to this topic; it's for a physical layer protocol. The start and end sentinels are "code violations", so there's no problem distinguishing the data from the sentinels. You don't want to use a length? Fine, but instead of variable length messages, they should be fixed length. And then you might not even use sentinels. See #16178447 – Bastille