How do I normalize a CSV file with Encog?

Question

I need to normalize a CSV file. I followed this article written by Jeff Heaton. This is (some) of my code:

File sourceFile = new File("Book1.csv");
File targetFile = new File("Book1_norm.csv");
EncogAnalyst analyst = new EncogAnalyst();
AnalystWizard wizard = new AnalystWizard(analyst);
wizard.wizard(sourceFile, true, AnalystFileFormat.DECPNT_COMMA);
final AnalystNormalizeCSV norm = new AnalystNormalizeCSV();
norm.analyze(sourceFile, false, CSVFormat.ENGLISH, analyst);
norm.setProduceOutputHeaders(false);
norm.normalize(targetFile);

The only difference between my code and the one of the article is this line:

norm.setOutputFormat(CSVFormat.ENGLISH);

I tried to use it but it seems that in Encog 3.1.0, that method doesn't exist. The error I get is this one (it looks like the problem is with the line norm.normalize(targetFile):

Exception in thread "main" org.encog.app.analyst.AnalystError: Can't find column: 11700
    at org.encog.app.analyst.util.CSVHeaders.find(CSVHeaders.java:187)
    at org.encog.app.analyst.csv.normalize.AnalystNormalizeCSV.extractFields(AnalystNormalizeCSV.java:77)
    at org.encog.app.analyst.csv.normalize.AnalystNormalizeCSV.normalize(AnalystNormalizeCSV.java:192)
    at IEinSoftware.main(IEinSoftware.java:55)

Guacharo

Answer 1

I added a FAQ that shows how to normalize a CSV file. http://www.heatonresearch.com/faq/4/2

Transilluminate

Answer 2

Here's a function to do it... of course you need to create an analyst

private EncogAnalyst _analyst;

public void NormalizeFile(FileInfo SourceDataFile, FileInfo NormalizedDataFile)
{
    var wizard = new AnalystWizard(_analyst);
    wizard.Wizard(SourceDataFile, _useHeaders, AnalystFileFormat.DecpntComma);
    var norm = new AnalystNormalizeCSV();
    norm.Analyze(SourceDataFile, _useHeaders, CSVFormat.English, _analyst);
    norm.ProduceOutputHeaders = _useHeaders;
    norm.Normalize(NormalizedDataFile);
}

Sulphurbottom

wow... I was looking for this function as I lost the project I'd originally used it in. I had forgotten that I wrote this back in March! I did a search for normalizing a CSV file using the Encog library and I found my own post! I helped my future self... Perfect! If I could I would upvote my past self's answer! – Sulphurbottom