C#: Convert COMP-3 Packed Decimal to Human-Readable Value
Asked Answered
M

8

6

I have a series of ASCII flat files coming in from a mainframe to be processed by a C# application. A new feed has been introduced with a Packed Decimal (COMP-3) field, which needs to be converted to a numerical value.

The files are being transferred via FTP, using ASCII transfer mode. I am concerned that the binary field may contain what will be interpreted as very-low ASCII codes or control characters instead of a value - Or worse, may be lost in the FTP process.

What's more, the fields are being read as strings. I may have the flexibility to work around this part (i.e. a stream of some sort), but the business will give me pushback.

The requirement read "Convert from HEX to ASCII", but clearly that didn't yield the correct values. Any help would be appreciated; it need not be language-specific as long as you can explain the logic of the conversion process.

Makepeace answered 27/9, 2008 at 4:18 Comment(0)
M
6

First of all you must eliminate the end of line (EOL) translation problems that will be caused by ASCII transfer mode. You are absolutely right to be concerned about data corruption when the BCD values happen to correspond to EOL characters. The worst aspect of this problem is that it will occur rarely and unexpectedly.

The best solution is to change the transfer mode to BIN. This is appropriate since the data you are transferring is binary. If it is not possible to use the correct FTP transfer mode, you can undo the ASCII mode damage in code. All you have to do is convert \r\n pairs back to \n. If I were you I would make sure this is well tested.

Once you've dealt with the EOL problem, the COMP-3 conversion is pretty straigtforward. I was able to find this article in the MS knowledgebase with sample code in BASIC. See below for a VB.NET port of this code.

Since you're dealing with COMP-3 values, the file format you're reading almost surely has fixed record sizes with fixed field lengths. If I were you, I would get my hands of a file format specification before you go any further with this. You should be using a BinaryReader to work with this data. If someone is pushing back on this point, I would walk away. Let them find someone else to indulge their folly.

Here's a VB.NET port of the BASIC sample code. I haven't tested this because I don't have access to a COMP-3 file. If this doesn't work, I would refer back to the original MS sample code for guidance, or to references in the other answers to this question.

Imports Microsoft.VisualBasic

Module Module1

'Sample COMP-3 conversion code
'Adapted from http://support.microsoft.com/kb/65323
'This code has not been tested

Sub Main()

    Dim Digits%(15)       'Holds the digits for each number (max = 16).
    Dim Basiceqv#(1000)   'Holds the Basic equivalent of each COMP-3 number.

    'Added to make code compile
    Dim MyByte As Char, HighPower%, HighNibble%
    Dim LowNibble%, Digit%, E%, Decimal%, FileName$


    'Clear the screen, get the filename and the amount of decimal places
    'desired for each number, and open the file for sequential input:
    FileName$ = InputBox("Enter the COBOL data file name: ")
    Decimal% = InputBox("Enter the number of decimal places desired: ")

    FileOpen(1, FileName$, OpenMode.Binary)

    Do Until EOF(1)   'Loop until the end of the file is reached.
        Input(1, MyByte)
        If MyByte = Chr(0) Then     'Check if byte is 0 (ASC won't work on 0).
            Digits%(HighPower%) = 0       'Make next two digits 0. Increment
            Digits%(HighPower% + 1) = 0   'the high power to reflect the
            HighPower% = HighPower% + 2   'number of digits in the number
            'plus 1.
        Else
            HighNibble% = Asc(MyByte) \ 16      'Extract the high and low
            LowNibble% = Asc(MyByte) And &HF    'nibbles from the byte. The
            Digits%(HighPower%) = HighNibble%  'high nibble will always be a
            'digit.
            If LowNibble% <= 9 Then                   'If low nibble is a
                'digit, assign it and
                Digits%(HighPower% + 1) = LowNibble%   'increment the high
                HighPower% = HighPower% + 2            'power accordingly.
            Else
                HighPower% = HighPower% + 1 'Low nibble was not a digit but a
                Digit% = 0                  '+ or - signals end of number.

                'Start at the highest power of 10 for the number and multiply
                'each digit by the power of 10 place it occupies.
                For Power% = (HighPower% - 1) To 0 Step -1
                    Basiceqv#(E%) = Basiceqv#(E%) + (Digits%(Digit%) * (10 ^ Power%))
                    Digit% = Digit% + 1
                Next

                'If the sign read was negative, make the number negative.
                If LowNibble% = 13 Then
                    Basiceqv#(E%) = Basiceqv#(E%) - (2 * Basiceqv#(E%))
                End If

                'Give the number the desired amount of decimal places, print
                'the number, increment E% to point to the next number to be
                'converted, and reinitialize the highest power.
                Basiceqv#(E%) = Basiceqv#(E%) / (10 ^ Decimal%)
                Print(Basiceqv#(E%))
                E% = E% + 1
                HighPower% = 0
            End If
        End If
    Loop

    FileClose()   'Close the COBOL data file, and end.
End Sub

End Module
Mantelet answered 27/9, 2008 at 4:56 Comment(5)
Will Binary transfer mode cause any problems with the rest of the file, which contains ASCII data? An upstream process will disqualify any lines that appear to be shorter than the predescribed length, so I may be able to push back and get it changed to a plaintext number. Thanks.Makepeace
Actually, that code was in QBasic... which I know, but haven't used in about 10 years :) I'm so rusty I wasn't able to convert that to C#, let alone how to use that for an 8-character binary field in an ascii file... But thanks for trying.Makepeace
Binary FTP transfer will not alter the file at all.Pelmas
If this is the accepted answer, can you tell us how you ended up getting this to work?! I was searching for a solution to the exact same problem and ended up here. Please help!!! I do not understand QBasic I am too young.Thorr
I've added a VB.NET port. I'm not sure how helpful this is, but it was an interesting exercise porting the old QBASIC code to VB.NET 3.5. It's surprising how few changes were needed to get it compiling and running. However, I would be wary of the input statements, I can't guarantee they will do the binary read correctly. If you have an appropriate test file you should be able to verify the correctness (or incorrectness) of the code.Mantelet
B
11

I have been watching the posts on numerous boards concerning converting Comp-3 BCD data from "legacy" mainframe files to something useable in C#. First, I would like to say that I am less than enamoured by the responses that some of these posts have received - especially those that have said essentially "why are you bothering us with these non-C#/C++ related posts" and also "If you need an answer about some sort of COBOL convention, why don't you go visit a COBOL oriented site". This, to me, is complete BS as there is going to be a need for probably many years to come, (unfortunately), for software developers to understand how to deal with some of these legacy issues that exist in THE REAL WORLD. So, even if I get slammed on this post for the following code, I am going to share with you a REAL WORLD experience that I had to deal with regarding COMP-3/EBCDIC conversion (and yes, I am he who talks of "floppy disks, paper-tape, Disc Packs etc... - I have been a software engineer since 1979").

First - understand that any file that you read from a legacy main-frame system like IBM is going to present the data to you in EBCDIC format and in order to convert any of that data to a C#/C++ string you can deal with you are going to have to use the proper code page translation to get the data into ASCII format. A good example of how to handle this would be:

StreamReader readFile = new StreamReader(path, Encoding.GetEncoding(037); // 037 = EBCDIC to ASCII translation.

This will ensure that anything that you read from this stream will then be converted to ASCII and can be used in a string format. This includes "Zoned Decimal" (Pic 9) and "Text" (Pic X) fields as declared by COBOL. However, this does not necessarily convert COMP-3 fields to the correct "binary" equivelant when read into a char[] or byte[] array. To do this, the only way that you are ever going to get this translated properly (even using UTF-8, UTF-16, Default or whatever) code pages, you are going to want to open the file like this:

FileStream fileStream = new FileStream(path, FIleMode.Open, FIleAccess.Read, FileShare.Read);

Of course, the "FileShare.Read" option is "optional".

When you have isolated the field that you want to convert to a decimal value (and then subsequently to an ASCII string if need be), you can use the following code - and this has been basically stolen from the MicroSoft "UnpackDecimal" posting that you can get at:

http://www.microsoft.com/downloads/details.aspx?familyid=0e4bba52-cc52-4d89-8590-cda297ff7fbd&displaylang=en

I have isolated (I think) what are the most important parts of this logic and consolidated it into two a method that you can do with what you want. For my purposes, I chose to leave this as returning a Decimal value which I could then do with what I wanted. Basically, the method is called "unpack" and you pass it a byte[] array (no longer than 12 bytes) and the scale as an int, which is the number of decimal places you want to have returned in the Decimal value. I hope this works for you as well as it did for me.

    private Decimal Unpack(byte[] inp, int scale)
    {
        long lo = 0;
        long mid = 0;
        long hi = 0;
        bool isNegative;

        // this nybble stores only the sign, not a digit.  
        // "C" hex is positive, "D" hex is negative, and "F" hex is unsigned. 
        switch (nibble(inp, 0))
        {
            case 0x0D:
                isNegative = true;
                break;
            case 0x0F:
            case 0x0C:
                isNegative = false;
                break;
            default:
                throw new Exception("Bad sign nibble");
        }
        long intermediate;
        long carry;
        long digit;
        for (int j = inp.Length * 2 - 1; j > 0; j--)
        {
            // multiply by 10
            intermediate = lo * 10;
            lo = intermediate & 0xffffffff;
            carry = intermediate >> 32;
            intermediate = mid * 10 + carry;
            mid = intermediate & 0xffffffff;
            carry = intermediate >> 32;
            intermediate = hi * 10 + carry;
            hi = intermediate & 0xffffffff;
            carry = intermediate >> 32;
            // By limiting input length to 14, we ensure overflow will never occur

            digit = nibble(inp, j);
            if (digit > 9)
            {
                throw new Exception("Bad digit");
            }
            intermediate = lo + digit;
            lo = intermediate & 0xffffffff;
            carry = intermediate >> 32;
            if (carry > 0)
            {
                intermediate = mid + carry;
                mid = intermediate & 0xffffffff;
                carry = intermediate >> 32;
                if (carry > 0)
                {
                    intermediate = hi + carry;
                    hi = intermediate & 0xffffffff;
                    carry = intermediate >> 32;
                    // carry should never be non-zero. Back up with validation
                }
            }
        }
        return new Decimal((int)lo, (int)mid, (int)hi, isNegative, (byte)scale);
    }

    private int nibble(byte[] inp, int nibbleNo)
    {
        int b = inp[inp.Length - 1 - nibbleNo / 2];
        return (nibbleNo % 2 == 0) ? (b & 0x0000000F) : (b >> 4);
    }

If you have any questions, post them on here - because I suspect that I am going to get "flamed" like everyone else who has chosen to post questions that are pertinent to todays issues...

Thanks, John - The Elder.

Bracey answered 17/4, 2012 at 22:56 Comment(1)
how to read the file and pass it on to Unpack, I am using filestream to open the file and then using StreamReader to read each line and then Converting the line into byte to pass to the Unpack method until end of stream. byte[] b = ASCIIEncoding.ASCII.GetBytes(reader.ReadLine()); Unpack(b,0); But it is giving wrong values during conversion of binary format number which is 13 digits, can you please suggest how to read the file and convert to byte and pass to unpack if I am doing anything wrong. ThanksNerissanerita
M
6

First of all you must eliminate the end of line (EOL) translation problems that will be caused by ASCII transfer mode. You are absolutely right to be concerned about data corruption when the BCD values happen to correspond to EOL characters. The worst aspect of this problem is that it will occur rarely and unexpectedly.

The best solution is to change the transfer mode to BIN. This is appropriate since the data you are transferring is binary. If it is not possible to use the correct FTP transfer mode, you can undo the ASCII mode damage in code. All you have to do is convert \r\n pairs back to \n. If I were you I would make sure this is well tested.

Once you've dealt with the EOL problem, the COMP-3 conversion is pretty straigtforward. I was able to find this article in the MS knowledgebase with sample code in BASIC. See below for a VB.NET port of this code.

Since you're dealing with COMP-3 values, the file format you're reading almost surely has fixed record sizes with fixed field lengths. If I were you, I would get my hands of a file format specification before you go any further with this. You should be using a BinaryReader to work with this data. If someone is pushing back on this point, I would walk away. Let them find someone else to indulge their folly.

Here's a VB.NET port of the BASIC sample code. I haven't tested this because I don't have access to a COMP-3 file. If this doesn't work, I would refer back to the original MS sample code for guidance, or to references in the other answers to this question.

Imports Microsoft.VisualBasic

Module Module1

'Sample COMP-3 conversion code
'Adapted from http://support.microsoft.com/kb/65323
'This code has not been tested

Sub Main()

    Dim Digits%(15)       'Holds the digits for each number (max = 16).
    Dim Basiceqv#(1000)   'Holds the Basic equivalent of each COMP-3 number.

    'Added to make code compile
    Dim MyByte As Char, HighPower%, HighNibble%
    Dim LowNibble%, Digit%, E%, Decimal%, FileName$


    'Clear the screen, get the filename and the amount of decimal places
    'desired for each number, and open the file for sequential input:
    FileName$ = InputBox("Enter the COBOL data file name: ")
    Decimal% = InputBox("Enter the number of decimal places desired: ")

    FileOpen(1, FileName$, OpenMode.Binary)

    Do Until EOF(1)   'Loop until the end of the file is reached.
        Input(1, MyByte)
        If MyByte = Chr(0) Then     'Check if byte is 0 (ASC won't work on 0).
            Digits%(HighPower%) = 0       'Make next two digits 0. Increment
            Digits%(HighPower% + 1) = 0   'the high power to reflect the
            HighPower% = HighPower% + 2   'number of digits in the number
            'plus 1.
        Else
            HighNibble% = Asc(MyByte) \ 16      'Extract the high and low
            LowNibble% = Asc(MyByte) And &HF    'nibbles from the byte. The
            Digits%(HighPower%) = HighNibble%  'high nibble will always be a
            'digit.
            If LowNibble% <= 9 Then                   'If low nibble is a
                'digit, assign it and
                Digits%(HighPower% + 1) = LowNibble%   'increment the high
                HighPower% = HighPower% + 2            'power accordingly.
            Else
                HighPower% = HighPower% + 1 'Low nibble was not a digit but a
                Digit% = 0                  '+ or - signals end of number.

                'Start at the highest power of 10 for the number and multiply
                'each digit by the power of 10 place it occupies.
                For Power% = (HighPower% - 1) To 0 Step -1
                    Basiceqv#(E%) = Basiceqv#(E%) + (Digits%(Digit%) * (10 ^ Power%))
                    Digit% = Digit% + 1
                Next

                'If the sign read was negative, make the number negative.
                If LowNibble% = 13 Then
                    Basiceqv#(E%) = Basiceqv#(E%) - (2 * Basiceqv#(E%))
                End If

                'Give the number the desired amount of decimal places, print
                'the number, increment E% to point to the next number to be
                'converted, and reinitialize the highest power.
                Basiceqv#(E%) = Basiceqv#(E%) / (10 ^ Decimal%)
                Print(Basiceqv#(E%))
                E% = E% + 1
                HighPower% = 0
            End If
        End If
    Loop

    FileClose()   'Close the COBOL data file, and end.
End Sub

End Module
Mantelet answered 27/9, 2008 at 4:56 Comment(5)
Will Binary transfer mode cause any problems with the rest of the file, which contains ASCII data? An upstream process will disqualify any lines that appear to be shorter than the predescribed length, so I may be able to push back and get it changed to a plaintext number. Thanks.Makepeace
Actually, that code was in QBasic... which I know, but haven't used in about 10 years :) I'm so rusty I wasn't able to convert that to C#, let alone how to use that for an 8-character binary field in an ascii file... But thanks for trying.Makepeace
Binary FTP transfer will not alter the file at all.Pelmas
If this is the accepted answer, can you tell us how you ended up getting this to work?! I was searching for a solution to the exact same problem and ended up here. Please help!!! I do not understand QBasic I am too young.Thorr
I've added a VB.NET port. I'm not sure how helpful this is, but it was an interesting exercise porting the old QBASIC code to VB.NET 3.5. It's surprising how few changes were needed to get it compiling and running. However, I would be wary of the input statements, I can't guarantee they will do the binary read correctly. If you have an appropriate test file you should be able to verify the correctness (or incorrectness) of the code.Mantelet
D
3

If the original data was in EBCDIC your COMP-3 field has been garbled. The FTP process has done an EBCDIC to ASCII translation of the byte values in the COMP-3 field which isn't what you want. To correct this you can:

1) Use BINARY mode for the transfer so you get the raw EBCDIC data. Then you convert the COMP-3 field to a number and translate any other EBCDIC text on the record to ASCII. A packed field stores each digit in a half byte with the lower half byte as a sign (F is positive and other values, usually D or E, are negative). Storing 123.4 in a PIC 999.99 USAGE COMP-3 would be X'01234F' (three bytes) and -123 in the same field is X'01230D'.

2) Have the sender convert the field into a USAGE IS DISPLAY SIGN IS LEADING(or TRAILING) numeric field. This stores the number as a string of EBCDIC numeric digits with the sign as a separate negative(-) or blank character. All digits and the sign translate correctly to their ASCII equivalent on the FTP transfer.

Dardar answered 4/11, 2008 at 19:48 Comment(3)
Thanks, we already took care of it. The upstream dependency has altered the flat file to provide the number in human-readable format.Makepeace
What about implied decimal? As in: PIC 999v99 COMP-3Thorr
Implied means the decimal point isn't in the stored value. With a PIC 999.99 USAGE IS DISPLAY SIGN LEADING you get a -12345 string for a -123.45 value or b12345 (leading blank) for a 123.45 positive value. The numeric text does get converted correctly when doing FTP from EBCDIC to ASCII.Dardar
C
2

I apologize if I am way off base here, but perhaps this code sample I'll paste here could help you. This came from VBRocks...

Imports System
Imports System.IO
Imports System.Text
Imports System.Text.Encoding



'4/20/07 submission includes a line spacing addition when a control character is used:
'   The line spacing is calculated off of the 3rd control character.
'
'   Also includes the 4/18 modification of determining end of file.

'4/26/07 submission inclues an addition of 6 to the record length when the 4th control
'   character is an 8.  This is because these records were being truncated.


'Authored by Gary A. Lima, aka. VBRocks



''' <summary>
''' Translates an EBCDIC file to an ASCII file.
''' </summary>
''' <remarks></remarks>
Public Class EBCDIC_to_ASCII_Translator

#Region " Example"

    Private Sub Example()
        'Set your source file and destination file paths
        Dim sSourcePath As String = "c:\Temp\MyEBCDICFile"
        Dim sDestinationPath As String = "c:\Temp\TranslatedFile.txt"

        Dim trans As New EBCDIC_to_ASCII_Translator()

        'If your EBCDIC file uses Control records to determine the length of a record, then this to True
        trans.UseControlRecord = True

        'If the first record of your EBCDIC file is filler (junk), then set this to True
        trans.IgnoreFirstRecord = True

        'EBCDIC files are written in block lengths, set your block length (Example:  134, 900, Etc.)
        trans.BlockLength = 900

        'This method will actually translate your source file and output it to the specified destination file path
        trans.TranslateFile(sSourcePath, sDestinationPath)


        'Here is a alternate example:
        'No Control record is used
        'trans.UseControlRecord = False

        'Translate the whole file, including the first record
        'trans.IgnoreFirstRecord = False

        'Set the block length
        'trans.BlockLength = 134

        'Translate...
        'trans.TranslateFile(sSourcePath, sDestinationPath)



        '*** Some additional methods that you can use are:

        'Trim off leading characters from left side of string (position 0 to...)
        'trans.LTrim = 15

        'Translate 1 EBCDIC character to an ASCII character
        'Dim strASCIIChar as String = trans.TranslateCharacter("S")

        'Translate an EBCDIC character array to an ASCII string
        'trans.TranslateCharacters(chrEBCDICArray)

        'Translates an EBCDIC string to an ASCII string
        'Dim strASCII As String = trans.TranslateString("EBCDIC String")


    End Sub

#End Region    'Example

    'Translate characters from EBCDIC to ASCII

    Private ASCIIEncoding As Encoding = Encoding.ASCII
    Private EBCDICEncoding As Encoding = Encoding.GetEncoding(37)  'EBCDIC

    'Block Length:  Can be fixed (Ex:  134). 
    Private miBlockLength As Integer = 0
    Private mbUseControlRec As Boolean = True        'If set to False, will return exact block length
    Private mbIgnoreFirstRecord As Boolean = True    'Will Ignore first record if set to true  (First record may be filler)
    Private miLTrim As Integer = 0

    ''' <summary>
    ''' Translates SourceFile from EBCDIC to ASCII.  Writes output to file path specified by DestinationFile parameter.
    ''' Set the BlockLength Property to designate block size to read.
    ''' </summary>
    ''' <param name="SourceFile">Enter the path of the Source File.</param>
    ''' <param name="DestinationFile">Enter the path of the Destination File.</param>
    ''' <remarks></remarks>
    Public Sub TranslateFile(ByVal SourceFile As String, ByVal DestinationFile As String)

        Dim iRecordLength As Integer     'Stores length of a record, not including the length of the Control Record (if used)
        Dim sRecord As String = ""         'Stores the actual record
        Dim iLineSpace As Integer = 1    'LineSpace:  1 for Single Space, 2 for Double Space, 3 for Triple Space...

        Dim iControlPosSix As Byte()      'Stores the 6th character of a Control Record (used to calculate record length)
        Dim iControlRec As Byte()          'Stores the EBCDIC Control Record (First 6 characters of record)
        Dim bEOR As Boolean                'End of Record Flag
        Dim bBOF As Boolean = True      'Beginning of file
        Dim iConsumedChars As Integer = 0     'Stores the number of consumed characters in the current block
        Dim bIgnoreRecord As Boolean = mbIgnoreFirstRecord   'Ignores the first record if set.

        Dim ControlArray(5) As Char         'Stores Control Record (first 6 bytes)
        Dim chrArray As Char()              'Stores characters just after read from file

        Dim sr As New StreamReader(SourceFile, EBCDICEncoding)
        Dim sw As New StreamWriter(DestinationFile)

        'Set the RecordLength to the RecordLength Property (below)
        iRecordLength = miBlockLength

        'Loop through entire file
        Do Until sr.EndOfStream = True

            'If using a Control Record, then check record for valid data.
            If mbUseControlRec = True Then
                'Read the Control Record (first 6 characters of the record)
                sr.ReadBlock(ControlArray, 0, 6)

                'Update the value of consumed (read) characters
                iConsumedChars += ControlArray.Length

                'Get the bytes of the Control Record Array
                iControlRec = EBCDICEncoding.GetBytes(ControlArray)

                'Set the line spacing  (position 3 divided by 64)
                '   (64 decimal = Single Spacing; 128 decimal = Double Spacing)
                iLineSpace = iControlRec(2) / 64


                'Check the Control record for End of File
                'If the Control record has a 8 or 10 in position 1, and a 1 in postion 2, then it is the end of the file
                If (iControlRec(0) = 8 OrElse iControlRec(0) = 10) AndAlso _
                    iControlRec(1) = 1 Then

                    If bBOF = False Then
                        Exit Do

                    Else
                        'The Beginning of file flag is set to true by default, so when the first
                        '   record is encountered, it is bypassed and the bBOF flag is set to False
                        bBOF = False

                    End If    'If bBOF = Fals

                End If    'If (iControlRec(0) = 8 OrElse



                'Set the default value for the End of Record flag to True
                '   If the Control Record has all zeros, then it's True, else False
                bEOR = True

                'If the Control record contains all zeros, bEOR will stay True, else it will be set to False
                For i As Integer = 0 To 5
                    If iControlRec(i) > 0 Then
                        bEOR = False

                        Exit For

                    End If    'If iControlRec(i) > 0

                Next    'For i As Integer = 0 To 5

                If bEOR = False Then
                    'Convert EBCDIC character to ASCII
                    'Multiply the 6th byte by 6 to get record length
                    '   Why multiply by 6?  Because it works.
                    iControlPosSix = EBCDICEncoding.GetBytes(ControlArray(5))

                    'If the 4th position of the control record is an 8, then add 6
                    '    to the record length to pick up remaining characters.
                    If iControlRec(3) = 8 Then
                        iRecordLength = CInt(iControlPosSix(0)) * 6 + 6

                    Else
                        iRecordLength = CInt(iControlPosSix(0)) * 6

                    End If

                    'Add the length of the record to the Consumed Characters counter
                    iConsumedChars += iRecordLength

                Else
                    'If the Control Record had all zeros in it, then it is the end of the Block.

                    'Consume the remainder of the block so we can continue at the beginning of the next block.
                    ReDim chrArray(miBlockLength - iConsumedChars - 1)
                    'ReDim chrArray(iRecordLength - iConsumedChars - 1)

                    'Consume (read) the remaining characters in the block.  
                    '   We are not doing anything with them because they are not actual records.
                    'sr.ReadBlock(chrArray, 0, iRecordLength - iConsumedChars)
                    sr.ReadBlock(chrArray, 0, miBlockLength - iConsumedChars)

                    'Reset the Consumed Characters counter
                    iConsumedChars = 0

                    'Set the Record Length to 0 so it will not be processed below.
                    iRecordLength = 0

                End If    ' If bEOR = False

            End If    'If mbUseControlRec = True



            If iRecordLength > 0 Then
                'Resize our array, dumping previous data.  Because Arrays are Zero (0) based, subtract 1 from the Record length.
                ReDim chrArray(iRecordLength - 1)

                'Read the specfied record length, without the Control Record, because we already consumed (read) it.
                sr.ReadBlock(chrArray, 0, iRecordLength)

                'Copy Character Array to String Array, Converting in the process, then Join the Array to a string
                sRecord = Join(Array.ConvertAll(chrArray, New Converter(Of Char, String)(AddressOf ChrToStr)), "")

                'If the record length was 0, then the Join method may return Nothing
                If IsNothing(sRecord) = False Then

                    If bIgnoreRecord = True Then
                        'Do nothing - bypass record

                        'Reset flag
                        bIgnoreRecord = False

                    Else
                        'Write the line out, LTrimming the specified number of characters.
                        If sRecord.Length >= miLTrim Then
                            sw.WriteLine(sRecord.Remove(0, miLTrim))

                        Else
                            sw.WriteLine(sRecord.Remove(0, sRecord.Length))

                        End If    ' If sRecord.Length >= miLTrim

                        'Write out the number of blank lines specified by the 3rd control character.
                        For i As Integer = 1 To iLineSpace - 1
                            sw.WriteLine("")

                        Next    'For i As Integer = 1 To iLineSpace

                    End If    'If bIgnoreRecord = True


                    'Obviously, if we have read more characters from the file than the designated size of the block,
                    '   then subtract the number of characters we have read into the next block from the block size.
                    If iConsumedChars > miBlockLength Then
                        'If iConsumedChars > iRecordLength Then
                        iConsumedChars = iConsumedChars - miBlockLength
                        'iConsumedChars = iConsumedChars - iRecordLength

                    End If

                End If    'If IsNothing(sRecord) = False

            End If    'If iRecordLength > 0

            'Allow computer to process  (works in a class module, not in a dll)
            'Application.DoEvents()

        Loop

        'Destroy StreamReader (sr)
        sr.Close()
        sr.Dispose()

        'Destroy StreamWriter (sw)
        sw.Close()
        sw.Dispose()

    End Sub



    ''' <summary>
    ''' Translates 1 EBCDIC Character (Char) to an ASCII String
    ''' </summary>
    ''' <param name="chr"></param>
    ''' <returns></returns>
    ''' <remarks></remarks>
    Private Function ChrToStr(ByVal chr As Char) As String
        Dim sReturn As String = ""

        'Convert character into byte
        Dim EBCDICbyte As Byte() = EBCDICEncoding.GetBytes(chr)

        'Convert EBCDIC byte to ASCII byte
        Dim ASCIIByte As Byte() = Encoding.Convert(EBCDICEncoding, ASCIIEncoding, EBCDICbyte)

        sReturn = Encoding.ASCII.GetString(ASCIIByte)

        Return sReturn

    End Function



    ''' <summary>
    ''' Translates an EBCDIC String to an ASCII String
    ''' </summary>
    ''' <param name="sStringToTranslate"></param>
    ''' <returns>String</returns>
    ''' <remarks></remarks>
    Public Function TranslateString(ByVal sStringToTranslate As String) As String
        Dim i As Integer = 0
        Dim sReturn As New System.Text.StringBuilder()

        'Loop through the string and translate each character
        For i = 0 To sStringToTranslate.Length - 1
            sReturn.Append(ChrToStr(sStringToTranslate.Substring(i, 1)))

        Next

        Return sReturn.ToString()


    End Function



    ''' <summary>
    ''' Translates 1 EBCDIC Character (Char) to an ASCII String
    ''' </summary>
    ''' <param name="sCharacterToTranslate"></param>
    ''' <returns>String</returns>
    ''' <remarks></remarks>
    Public Function TranslateCharacter(ByVal sCharacterToTranslate As Char) As String

        Return ChrToStr(sCharacterToTranslate)

    End Function



    ''' <summary>
    ''' Translates an EBCDIC Character (Char) Array to an ASCII String
    ''' </summary>
    ''' <param name="sCharacterArrayToTranslate"></param>
    ''' <returns>String</returns>
    ''' <remarks>Remarks</remarks>
    Public Function TranslateCharacters(ByVal sCharacterArrayToTranslate As Char()) As String
        Dim sReturn As String = ""

        'Copy Character Array to String Array, Converting in the process, then Join the Array to a string
        sReturn = Join(Array.ConvertAll(sCharacterArrayToTranslate, _
                            New Converter(Of Char, String)(AddressOf ChrToStr)), "")

        Return sReturn

    End Function


    ''' <summary>
    ''' Block Length must be set.  You can set the BlockLength for specific block sizes (Ex:  134).
    ''' Set UseControlRecord = False for files with specific block sizes (Default is True)
    ''' </summary>
    ''' <value>0</value>
    ''' <returns>Integer</returns>
    ''' <remarks></remarks>
    Public Property BlockLength() As Integer
        Get
            Return miBlockLength

        End Get
        Set(ByVal value As Integer)
            miBlockLength = value

        End Set
    End Property



    ''' <summary>
    ''' Determines whether a ControlKey is used to calculate RecordLength of valid data
    ''' </summary>
    ''' <value>Default value is True</value>
    ''' <returns>Boolean</returns>
    ''' <remarks></remarks>
    Public Property UseControlRecord() As Boolean
        Get
            Return mbUseControlRec

        End Get
        Set(ByVal value As Boolean)
            mbUseControlRec = value

        End Set
    End Property



    ''' <summary>
    ''' Ignores first record if set (Default is True)
    ''' </summary>
    ''' <value>Default is True</value>
    ''' <returns>Boolean</returns>
    ''' <remarks></remarks>
    Public Property IgnoreFirstRecord() As Boolean
        Get
            Return mbIgnoreFirstRecord

        End Get

        Set(ByVal value As Boolean)
            mbIgnoreFirstRecord = value

        End Set
    End Property



    ''' <summary>
    ''' Trims the left side of every string the specfied number of characters.  Default is 0.
    ''' </summary>
    ''' <value>Default is 0.</value>
    ''' <returns>Integer</returns>
    ''' <remarks></remarks>
    Public Property LTrim() As Integer
        Get
            Return miLTrim

        End Get

        Set(ByVal value As Integer)
            miLTrim = value

        End Set
    End Property


End Class
Circumpolar answered 27/9, 2008 at 4:35 Comment(2)
Wow, thanks. I won't ding you for the big paste (This is after all an answers site)... Up vote for a fast answer which looks right - But I can't accept it until I go back to work Monday. Lucky I also know VB.NET... Thanks.Makepeace
Nope, that didn't do it. Given either plaintext numbers or COMP-3 data, it output a bunch of garbage. What I need will translate '<' into '4C', 'b' into '82', '2' into F2, and '0' into 'F0', per my sample data. thanks for trying.Makepeace
S
0

Some useful links for EBCDIC translation:

Translation table - useful to do check some of the values in the packed decimal fields: http://www.simotime.com/asc2ebc1.htm

List of code pages in msdn:
http://msdn.microsoft.com/en-us/library/dd317756(VS.85).aspx

And a piece of code to convert the byte array fields in C#:

// 500 is the code page for IBM EBCDIC International 
System.Text.Encoding enc = new System.Text.Encoding(500);
string value = enc.GetString(byteArrayField);
Suet answered 23/9, 2009 at 13:11 Comment(0)
D
0

The packed fields are the same in EBCDIC or ASCII. Do not run the EBCDIC to ASCII conversion on them. In .Net dump them into a byte[].

You use bitwise masks and shifts to pack/unpack. -- But bitwise ops only apply to integer types in .Net so you need to jump through some hoops!

A good COBOL or C artist can point you in the right direction.

Find one of the old guys and pay your dues (about three beers should do it).

Dot answered 11/1, 2010 at 4:13 Comment(0)
K
0

The “ASCII transfer type” will transfer the files as regular text files. So files becoming corrupt when we transfer packed decimal or binary data files in ASCII transfer type. The “Binary transfer type” will transfer the data in binary mode which handles the files as binary data instead of text data. So we have to use Binary transfer type here. Reference : https://www.codeproject.com/Tips/673240/EBCDIC-to-ASCII-Converter

Once your file is ready, here is the code to convert packed decimal to human readable decimal.

    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;

   namespace ConsoleApp2
   {
    class Program
    {
        static void Main(string[] args)
        {
            var path = @"C:\FileName.BIN.dat";
            var templates = new List<Template>
            {
                new Template{StartPos=1,CharLength=4,Type="AlphaNum"},
                new Template{StartPos=5,CharLength=1,Type="AlphaNum"},
                new Template{StartPos=6,CharLength=8,Type="AlphaNum"},
                new Template{StartPos=14,CharLength=1,Type="AlphaNum"},
                new Template{StartPos=46,CharLength=4,Type="Packed",DecimalPlace=2},
                new Template{StartPos=54,CharLength=5,Type="Packed",DecimalPlace=0},
                new Template{StartPos=60,CharLength=4,Type="Packed",DecimalPlace=2},
                new Template{StartPos=64,CharLength=1,Type="AlphaNum"}
            };

            var allBytes = File.ReadAllBytes(path);
            for (int i = 0; i < allBytes.Length; i += 66)
            {
                var IsLastline = (allBytes.Length - i) < 66;
                var lineLength = IsLastline ? 64 : 66;
                byte[] lineBytes = new byte[lineLength];
                Array.Copy(allBytes, i, lineBytes, 0, lineLength);


                var outArray = new string[templates.Count];
                int index = 0;
                foreach (var temp in templates)
                {
                    byte[] amoutBytes = new byte[temp.CharLength];
                    Array.Copy(lineBytes, temp.StartPos - 1, amoutBytes, 0, 
    temp.CharLength);
                    var final = "";
                    if (temp.Type == "Packed")
                    {
                        final = Unpack(amoutBytes, temp.DecimalPlace).ToString();
                    }
                    else
                    {
                        final = ConvertEbcdicString(amoutBytes);
                    }

                    outArray[index] = final;
                    index++;

                }

                Console.WriteLine(string.Join(" ", outArray));

            }

            Console.ReadLine();
        }


        private static string ConvertEbcdicString(byte[] ebcdicBytes)
        {
            if (ebcdicBytes.All(p => p == 0x00 || p == 0xFF))
            {
                //Every byte is either 0x00 or 0xFF (fillers)
                return string.Empty;
            }

            Encoding ebcdicEnc = Encoding.GetEncoding("IBM037");
            string result = ebcdicEnc.GetString(ebcdicBytes); // convert EBCDIC Bytes -> 
    Unicode string
            return result;
        }

        private static Decimal Unpack(byte[] inp, int scale)
        {
            long lo = 0;
            long mid = 0;
            long hi = 0;
            bool isNegative;

            // this nybble stores only the sign, not a digit.  
            // "C" hex is positive, "D" hex is negative, AlphaNumd "F" hex is unsigned. 
            var ff = nibble(inp, 0);
            switch (ff)
            {
                case 0x0D:
                    isNegative = true;
                    break;
                case 0x0F:
                case 0x0C:
                    isNegative = false;
                    break;
                default:
                    throw new Exception("Bad sign nibble");
            }
            long intermediate;
            long carry;
            long digit;
            for (int j = inp.Length * 2 - 1; j > 0; j--)
            {
                // multiply by 10
                intermediate = lo * 10;
                lo = intermediate & 0xffffffff;
                carry = intermediate >> 32;
                intermediate = mid * 10 + carry;
                mid = intermediate & 0xffffffff;
                carry = intermediate >> 32;
                intermediate = hi * 10 + carry;
                hi = intermediate & 0xffffffff;
                carry = intermediate >> 32;
                // By limiting input length to 14, we ensure overflow will never occur

                digit = nibble(inp, j);
                if (digit > 9)
                {
                    throw new Exception("Bad digit");
                }
                intermediate = lo + digit;
                lo = intermediate & 0xffffffff;
                carry = intermediate >> 32;
                if (carry > 0)
                {
                    intermediate = mid + carry;
                    mid = intermediate & 0xffffffff;
                    carry = intermediate >> 32;
                    if (carry > 0)
                    {
                        intermediate = hi + carry;
                        hi = intermediate & 0xffffffff;
                        carry = intermediate >> 32;
                        // carry should never be non-zero. Back up with validation
                    }
                }
            }
            return new Decimal((int)lo, (int)mid, (int)hi, isNegative, (byte)scale);
        }

        private static int nibble(byte[] inp, int nibbleNo)
        {
            int b = inp[inp.Length - 1 - nibbleNo / 2];
            return (nibbleNo % 2 == 0) ? (b & 0x0000000F) : (b >> 4);
        }

        class Template
        {
            public string Name { get; set; }
            public string Type { get; set; }
            public int StartPos { get; set; }
            public int CharLength { get; set; }
            public int DecimalPlace { get; set; }
        }
    }
   }
Kristelkristen answered 5/6, 2020 at 22:3 Comment(0)
E
0

Files must be transferred as binary. Here's a much shorter way to do it:

using System.Linq;

namespace SomeNamespace
{
    public static class SomeExtensionClass
    {
        /// <summary>
        /// computes the actual decimal value from an IBM "Packed Decimal" 9(x)v9 (COBOL) format
        /// </summary>
        /// <param name="value">byte[]</param>
        /// <param name="precision">byte; decimal places, default 2</param>
        /// <returns>decimal</returns>
        public static decimal FromPackedDecimal(this byte[] value, byte precision = 2)
        {
            if (value.Length < 1)
            {
                throw new System.InvalidOperationException("Cannot unpack empty bytes.");
            }
            double power = System.Math.Pow(10, precision);
            if (power > long.MaxValue)
            {
                throw new System.InvalidOperationException(
                    $"Precision too large for valid calculation: {precision}");
            }
            string hex = System.BitConverter.ToString(value).Replace("-", "");
            var bytes = Enumerable.Range(0, hex.Length)
                     .Select(x => System.Convert.ToByte($"0{hex.Substring(x, 1)}", 16))
                     .ToList();
            long place = 1;
            decimal ret = 0;
            for (int i = bytes.Count - 2; i > -1; i--)
            {
                ret += (bytes[i] * place);
                place *= 10;
            }
            ret /= (long)power;
            return (bytes.Last() & (1 << 7)) != 0 ? ret * -1 : ret;
        }
    }
}
Eluviation answered 11/6, 2020 at 13:17 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.