Is there an smart way to write a fixed length flat file?
Asked Answered
C

9

16

Is there any framework/library to help writing fixed length flat files in java?

I want to write a collection of beans/entities into a flat file without worrying with convertions, padding, alignment, fillers, etcs

For example, I'd like to parse a bean like:

public class Entity{
    String name = "name"; // length = 10; align left; fill with spaces
    Integer id = 123; // length = 5; align left; fill with spaces
    Integer serial = 321 // length = 5; align to right; fill with '0'
    Date register = new Date();// length = 8; convert to yyyyMMdd
}

... into ...

name      123  0032120110505
mikhas    5000 0122120110504
superuser 1    0000120101231

...

Caseose answered 4/5, 2011 at 14:31 Comment(8)
What do you want to do exactly? Right now it sounds like Java's built-in functions should be enough for your needs.Opening
I want to write a collection of beans/entities into a flat file without worrying with convertions, padding, alignment, fillers, etcsCaseose
What you're looking for is not a framework/library to write a flat file. You're looking for a framework/library to convert beans/entities into a flat format which you can write to file.Avert
By the way, why does it have to be a flat file, as opposed to XML, JSON, YAML, Java-Serialization, et.c.?Avert
It must be a text file. It's an interface with a legacy system.Caseose
XML, JSON and YAML are all text. Binary data can be converted to text (e.g. with base64)Opening
I have a pre-defined fixed length(positioned) interface layout which I must follow.Caseose
Can you Integer be null or should these be int?Trample
E
12

If you are still looking for a framework, check out BeanIO at http://www.beanio.org

Exanthema answered 4/8, 2011 at 17:27 Comment(2)
FYI, I've been using BeanIO in various projects after you sugestion. ThanksCaseose
Last release of BeanIO in 2013, Is anyone maintaining it?Rexer
C
14

You're not likely to encounter a framework that can cope with a "Legacy" system's format. In most cases, Legacy systems don't use standard formats, but frameworks expect them. As a maintainer of legacy COBOL systems and Java/Groovy convert, I encounter this mismatch frequently. "Worrying with conversions, padding, alignment, fillers, etcs" is primarily what you do when dealing with a legacy system. Of course, you can encapsulate some of it away into handy helpers. But most likely, you'll need to get real familiar with java.util.Formatter.

For example, you might use the Decorator pattern to create decorators to do the conversion. Below is a bit of groovy (easily convertible into Java):

class Entity{
    String name = "name"; // length = 10; align left; fill with spaces
    Integer id = 123; // length = 5; align left; fill with spaces
    Integer serial = 321 // length = 5; align to right; fill with '0'
    Date register = new Date();// length = 8; convert to yyyyMMdd
}

class EntityLegacyDecorator {
     Entity d
     EntityLegacyDecorator(Entity d) { this.d = d }

     String asRecord() {
         return String.format('%-10s%-5d%05d%tY%<tm%<td',
                               d.name,d.id,d.serial,d.register)
   }
 }

def e = new Entity(name: 'name', id: 123, serial: 321, register: new Date('2011/05/06'))

assert new EntityLegacyDecorator(e).asRecord() == 'name      123  0032120110506'

This is workable if you don't have too many of these and the objects aren't too complex. But pretty quickly the format string gets intolerable. Then you might want decorators for Date, like:

class DateYMD {
     Date d
     DateYMD(d) { this.d = d }
     String toString() { return d.format('yyyyMMdd') }
 }

so you can format with %s:

    String asRecord() {
         return String.format('%-10s%-5d%05d%s',
                               d.name,d.id,d.serial,new DateYMD(d.register))
   }

But for significant number of bean properties, the string is still too gross, so you want something that understands columns and lengths that looks like the COBOL spec you were handed, so you'll write something like this:

 class RecordBuilder {

    final StringBuilder record 

    RecordBuilder(recordSize) {
        record = new StringBuilder(recordSize)
        record.setLength(recordSize)
     }

    def setField(pos,length,String s) { 
       record.replace(pos - 1, pos + length, s.padRight(length))
    }

    def setField(pos,length,Date d) { 
      setField(pos,length, new DateYMD(d).toString())
    }

   def setField(pos,length, Integer i, boolean padded) { 
       if (padded) 
          setField(pos,length, String.format("%0" + length + "d",i))
       else 
          setField(pos,length, String.format("%-" + length + "d",i))
    }

    String toString() { record.toString() }
}

class EntityLegacyDecorator {

     Entity d

     EntityLegacyDecorator(Entity d) { this.d = d }

     String asRecord() {
         RecordBuilder record = new RecordBuilder(28)
         record.setField(1,10,d.name)
         record.setField(11,5,d.id,false)
         record.setField(16,5,d.serial,true)
         record.setField(21,8,d.register)
         return record.toString()
     }

}

After you've written enough setField() methods to handle you legacy system, you'll briefly consider posting it on GitHub as a "framework" so the next poor sap doesn't have to to it again. But then you'll consider all the ridiculous ways you've seen COBOL store a "date" (MMDDYY, YYMMDD, YYDDD, YYYYDDD) and numerics (assumed decimal, explicit decimal, sign as trailing separate or sign as leading floating character). Then you'll realize why nobody has produced a good framework for this and occasionally post bits of your production code into SO as an example... ;)

Cancellate answered 5/5, 2011 at 22:1 Comment(1)
I would love to find such a "framework" in GitHub, because I'm one of those poor saps that did the same. In my custom solution I use a mapping class + dyna beans.Delisadelisle
E
12

If you are still looking for a framework, check out BeanIO at http://www.beanio.org

Exanthema answered 4/8, 2011 at 17:27 Comment(2)
FYI, I've been using BeanIO in various projects after you sugestion. ThanksCaseose
Last release of BeanIO in 2013, Is anyone maintaining it?Rexer
I
3

uniVocity-parsers goes a long way to support tricky fixed-width formats, including lines with different fields, paddings, etc.

Check out this example to write imaginary client & accounts details. This uses a lookahead value to identify which format to use when writing a row:

    FixedWidthFields accountFields = new FixedWidthFields();
    accountFields.addField("ID", 10); //account ID has length of 10
    accountFields.addField("Bank", 8); //bank name has length of 8
    accountFields.addField("AccountNumber", 15); //etc
    accountFields.addField("Swift", 12);

    //Format for clients' records
    FixedWidthFields clientFields = new FixedWidthFields();
    clientFields.addField("Lookahead", 5); //clients have their lookahead in a separate column
    clientFields.addField("ClientID", 15, FieldAlignment.RIGHT, '0'); //let's pad client ID's with leading zeroes.
    clientFields.addField("Name", 20);

    FixedWidthWriterSettings settings = new FixedWidthWriterSettings();
    settings.getFormat().setLineSeparator("\n");
    settings.getFormat().setPadding('_');

    //If a record starts with C#, it's a client record, so we associate "C#" with the client format.
    settings.addFormatForLookahead("C#", clientFields);

    //Rows starting with #A should be written using the account format
    settings.addFormatForLookahead("A#", accountFields);

    StringWriter out = new StringWriter();

    //Let's write
    FixedWidthWriter writer = new FixedWidthWriter(out, settings);

    writer.writeRow(new Object[]{"C#",23234, "Miss Foo"});
    writer.writeRow(new Object[]{"A#23234", "HSBC", "123433-000", "HSBCAUS"});
    writer.writeRow(new Object[]{"A#234", "HSBC", "222343-130", "HSBCCAD"});
    writer.writeRow(new Object[]{"C#",322, "Mr Bar"});
    writer.writeRow(new Object[]{"A#1234", "CITI", "213343-130", "CITICAD"});

    writer.close();

    System.out.println(out.toString());

The output will be:

C#___000000000023234Miss Foo____________
A#23234___HSBC____123433-000_____HSBCAUS_____
A#234_____HSBC____222343-130_____HSBCCAD_____
C#___000000000000322Mr Bar______________
A#1234____CITI____213343-130_____CITICAD_____

This is just a rough example. There are many other options available, including support for annotated java beans, which you can find here.

Disclosure: I'm the author of this library, it's open-source and free (Apache 2.0 License)

Ineslta answered 2/5, 2016 at 3:40 Comment(0)
T
2

The library Fixedformat4j is a pretty neat tool to do exactly this: http://fixedformat4j.ancientprogramming.com/

Teeters answered 15/8, 2013 at 9:24 Comment(0)
P
1

Spring Batch has a FlatFileItemWriter, but that won't help you unless you use the whole Spring Batch API.


But apart from that, I'd say you just need a library that makes writing to files easy (unless you want to write the whole IO code yourself).

Two that come to mind are:

Guava

Files.write(stringData, file, Charsets.UTF_8);

Commons / IO

FileUtils.writeStringToFile(file, stringData, "UTF-8");
Puttier answered 4/5, 2011 at 14:35 Comment(1)
The problem is not writing the file, but formatting each field on a fixed-length file.Caseose
T
0

Don't know of any frame work but you can just use RandomAccessFile. You can position the file pointer to anywhere in the file to do your reads and writes.

Translucent answered 4/5, 2011 at 14:35 Comment(0)
C
0

I've just find a nice library that I'm using:
http://sourceforge.net/apps/trac/ffpojo/wiki

Very simple to configurate with XML or annotations!

Caseose answered 6/5, 2011 at 12:39 Comment(0)
T
-1

A simple way to write beans/entities to a flat file is to use ObjectOutputStream.

public static void writeToFile(File file, Serializable object) throws IOException {
    ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
    oos.writeObject(object);
    oos.close();
}

You can write to a fixed length flat file with

FileUtils.writeByteArrayToFile(new File(filename), new byte[length]);

You need to be more specific about what you want to do with the file. ;)

Trample answered 4/5, 2011 at 14:40 Comment(3)
The problem is not writing the file, but formatting each field on a fixed-length file.Caseose
The solution above solve the problem it can write any object to a file in binary format. If you want text format, just do a base64 conversion or something like that. For the fixed size, just provide more than needed, and you are fine.Malaysia
@mikhas, Why would you have a fixed length file for each field. Can you get an example of what you are talking about e.g. an object you want to write and what you expect to happen.Trample
S
-1

Try FFPOJO API as it has everything which you need to create a flat file with fixed lengths and also it will convert a file to an object and vice versa.

@PositionalRecord
public class CFTimeStamp {

    String timeStamp;

    public CFTimeStamp(String timeStamp) {
        this.timeStamp = timeStamp;
    }

    @PositionalField(initialPosition = 1, finalPosition = 26, paddingAlign = PaddingAlign.RIGHT, paddingCharacter = '0')
    public String getTimeStamp() {
        return timeStamp;
    }

    @Override
    public String toString() {
        try {
            FFPojoHelper ffPojo = FFPojoHelper.getInstance();
            return ffPojo.parseToText(this);
        } catch (FFPojoException ex) {
            trsLogger.error(ex.getMessage(), ex);
        }
        return null;
    }
}
Stultz answered 29/8, 2016 at 15:16 Comment(1)
Do you mind posting an example with what you're referring to?Brolly

© 2022 - 2024 — McMap. All rights reserved.