BizTalk Flat File Schema - how to accept a LF or CRLF as the line delimiter

Question

Our client sends us a flat file as input, which we then take and convert to an XML file before sending to the destination system.

The flat file consists of multiple lines, each line is delimited by LF or CRLF.

How do I create a Flat File Schema so that BizTalk can interpret each line of data regardless of whether the line was delimited by LF (0x0A) or CRLF (0x0D 0x0A)?

Taverner

blog.sandro-pereira.com/2015/10/23/… – Spatiotemporal

Answer 1

Problem solved. Here is the solution in case anyone else is wondering:

Since LF and CRLF both share the LF character, I set the line delimiter as LF (0x0A). This works correctly for extracting the full records (with the side effect of having one extra CR character at the end when CRLF is the delimiter).

One can get rid of the extra CR character using a dummy field to absorb the CR character or by using a map.

Notice that since the LF and CRLF delimiters have different lengths (1 and 2 characters respectively), I had to do a couple more changes to the schema to ensure that both are handled correctly.

In my scenario, each line record that was parsed contains 8 positional fields, so having an extra CR character at the end resulted in an error due to Biztalk expecting a certain length for the last field that does not account for the additional CR character. The solution is to increase the length of the 8th field (which is a Filler field in my case) by 1. However, in order to still be able to handle the LF line delimiters, make sure you set the 'Allow Early Termination' flag to TRUE. This way no errors are thrown if the last field is 1 character short of its assigned length (if the CR character was not included).

Taverner

Wouldn't this mean that you would accept malformed records as well? – Mou

Answer 2

Forgive me if I misunderstand the question... it sounds as if each line is a record, but some lines end in LF and others in CRLF, and you need to have both as a delimiter at the same level?

I don't know of a way to specify more than one child delimiter with the flat file schema by itself, but one possible solution could be to write a custom pipeline component for the decode stage of your receive pipeline to replace the CRLFs with LFs, then use LF as the delimiter on the flat file schema.

Satiate

Thank you BiztalkMama. Your answer is definitely an option, but I wanted a way to do this using the Schema Editor rather than a custom Decode Component. Luckily, I discovered that it is possible after all. I appreciate your help though. – Taverner

Answer 3

A simple approach is to set LF as the delimiter and CR as the padding character at the schema level. In case the schema works when testing via Visual Studio but errs out in the pipeline, flip Infix/Postfix values. No need to select Allow Early Termination.

Carrack