Append a character for last value in a list in JAVA

Question

Given this input

0000027788|001400000000000000000001224627|G1|||G1
0000027789|001400000000000000000001224627|D1|||G1
0000027790|001400000000000000000001224627|D1|||G1
0000027790|001400000000000000000001224627|D1|||G1
0000027791|001400000000000000000001224627|G2|||G2
0000027792|001400000000000000000001224627|D2|||G2
0000027793|001400000000000000000001224627|D2|||G2
0000027794|001400000000000000000001224627|G6|||G6

I need the 3rd column particularly from the file and find which is the last D1 for the group G1 and similarly last D2 for particular G2. After finding the last value, I need something to be appended in the corresponding line like "LL":

I have tried it, but the line is getting appended parallel to each D1 not only to the last D1.

This is my code:

package com.scb.firstreport;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.StringTokenizer;
public class EDWBatchProcessor {

//static Logger log = Logger.getLogger(EDWBatchProcessor.class.getName());

public static void main(String[] args) throws JRException, NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException {
    //log.debug("Hello this is a debug message");
    File fileDir = new File("D:\\EDWFileProcessing\\simple.txt");
    String line = null;
    String[] split = null;
    try {
        // FileReader reads text files in the default encoding.
        BufferedReader in = new BufferedReader(
                   new InputStreamReader(
                              new FileInputStream(fileDir), "UTF-8"));

          BufferedWriter bufferedWriter = null;
        while((line = in.readLine()) != null) {
            //System.out.println(line);
            split = line.split("\\|");

            List<String> customerList = new ArrayList<String>();

            if(!customerList.contains(split[1])){
                customerList.add(split[1]);
                bufferedWriter =
                        new BufferedWriter(new OutputStreamWriter(
                                new FileOutputStream("D:\\EDWFileProcessing\\output\\"+split[1]+".txt",true), "UTF-8"));

                bufferedWriter.write(line);
                bufferedWriter.newLine();
                bufferedWriter.close();

            }
            else{

                 bufferedWriter.write(line);
                 bufferedWriter.close();
            }

        }   
        final File folder = new File("D:\\EDWFileProcessing\\output");
       listFilesForFolder(folder);
        // Always close files.
        in.close();  

    }
    catch(FileNotFoundException ex) {
        System.out.println(
            "Unable to open file '");                
    }
    catch(IOException ex) {
        System.out.println(
            "Error reading file '" 
          );                  
        // Or we could just do this: 
        // ex.printStackTrace();
    }

   }

private static void listFilesForFolder(File folder) throws NoSuchFieldException, SecurityException, IllegalArgumentException, IllegalAccessException, JRException, IOException {
    for (final File fileEntry : folder.listFiles()) {
        if (fileEntry.isDirectory()) {
            //listFilesForFolder(fileEntry);
        } else {
           // System.out.println(fileEntry.getName().substring(0, 30));
            //System.out.println(fileEntry.getAbsolutePath());

            File fileDir = new File(fileEntry.getAbsolutePath());
            String line = null;
            String lineNew = "000000000000000000000000000000000";
            String[] split = null;
            // FileReader reads text files in the default encoding.
            BufferedReader in = new BufferedReader(
                       new InputStreamReader(
                                  new FileInputStream(fileDir), "UTF-8"));

              BufferedWriter bufferedWriter = null;
              List<String> customerList = new ArrayList<String>();

                List<String> recTypeList = new ArrayList<String>();

            while((line = in.readLine()) != null) {
               // System.out.println(line);
                split = line.split("\\|");
                bufferedWriter =
                            new BufferedWriter(new OutputStreamWriter(
                                    new FileOutputStream("D:\\EDWFileProcessing\\output\\simple\\"+split[1]+".txt",true), "UTF-8"));

                    System.out.println("Split2222222222========>>"+split[2]);
                    System.out.println("Split2222222222========>>"+recTypeList.contains(split[2]));

                    if(!recTypeList.contains(split[2])){
                    recTypeList.add(split[2]);
                    bufferedWriter.newLine();

                    bufferedWriter.write(line);

                    }else{
                        bufferedWriter.newLine();
                        line = line.concat("|LL");
                        bufferedWriter.write(line);
                        System.out.println("line new.....................");
                        //bufferedWriter.newLine();
                        //bufferedWriter.write(lineNew);
                        // bufferedWriter.newLine();
                    }


                    //bufferedWriter.newLine();
                    bufferedWriter.close();
          }   


            in.close();  

        }
   }
}

}

I tried using list, but "LL" is getting appended to the end of G2.

  for (ListIterator<String> it = recTypeList.listIterator(); it.hasNext(); i++) {

         String s1 = it.next();
          if(s1.equals("G2"))
                {

                    int ind=it.previousIndex()-1;
                    String val=recTypeList.get(ind);
                    String lastop=val.concat("LL");
                   bufferedWriter.write(lastop);
                   System.out.println(lastop);
                   System.out.println(val);
                 }

Magdau

I don't see where in the code you're looking for the last G1 or G2? Obviously for every G1 you concat the |LL so you need some mechanism to find the last G1. – Dextran

I need to find the last D1,D2 for every G1 , G2. I have marked the column and I tried finding the last value for particular list but it didn't help@Dextran – Magdau

Ok, I understood that. so your problem is with finding the last D1 and D2 for every G1 and G2 right? – Dextran

Yes..Im trying out few ways using list too .. @Dextran – Magdau

Please provide the file (or part of it) in text format so that we can easily try it ourselves and not type the contents from the image. – Ohm

Please define last D1. How are records expected to be sorted by D1? Order in input file? – Gisser

In order to find the last line containing D1, you need to find the next line that doesn't contain D1, and then add LL to the previous line. – Desperado

Also, it's worth using the try-with-resources statement to make your code simpler and easier to read. – Desperado

Answer 1

Here is how you can find the last D# for each G# and append an LL at the end. If we knew more information about how the format behaves it could have been made simpler. I substituted reading the file by putting it as a string and splitting the lines so that lines is what you have after you finish reading all the lines.

public class Test {

    public static void main(String[] args) {

        String input =
                "0000027788|001400000000000000000001224627|G1|        |         |G1\r\n" + 
                "0000027789|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027790|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027790|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027791|001400000000000000000001224627|G2|        |         |G2\r\n" + 
                "0000027792|001400000000000000000001224627|D2|        |         |G2\r\n" + 
                "0000027793|001400000000000000000001224627|D2|        |         |G2\r\n" + 
                "0000027794|001400000000000000000001224627|G6|        |         |G6";

        String[] lines = input.split("\r\n");
        String[][] parts = new String[lines.length][];
        for (int i = 0; i < lines.length; i++)
            parts[i] = lines[i].split("\\|");

        String currG = "G1";
        String lastD = "";
        for (int i = 1; i < lines.length; i++) {
            if (parts[i][2].startsWith("G")) {
                System.out.println("Last D for " + currG + " is " + lastD + " at line " + (i-1));
                lines[i-1] += "   LL";
                currG = parts[i][2];
            }
            else
                lastD = parts[i][2];
        }

        System.out.println();
        for (int i = 0; i < lines.length; i++)
            System.out.println(lines[i]);
    }
}

Output:

Last D for G1 is D1 at line 3
Last D for G2 is D2 at line 6

0000027788|001400000000000000000001224627|G1|        |         |G1
0000027789|001400000000000000000001224627|D1|        |         |G1
0000027790|001400000000000000000001224627|D1|        |         |G1
0000027790|001400000000000000000001224627|D1|        |         |G1   LL
0000027791|001400000000000000000001224627|G2|        |         |G2
0000027792|001400000000000000000001224627|D2|        |         |G2
0000027793|001400000000000000000001224627|D2|        |         |G2   LL
0000027794|001400000000000000000001224627|G6|        |         |G6

My assumptions are that the second column has only G# or D# and that in line 0 it is G1.

Edit: If I add to my above assumptions that under each G# there are Ds with only the same # then this is shorter:

public class Test {

    public static void main(String[] args) {

        String input =
                "0000027788|001400000000000000000001224627|G1|        |         |G1\r\n" + 
                "0000027789|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027790|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027790|001400000000000000000001224627|D1|        |         |G1\r\n" + 
                "0000027791|001400000000000000000001224627|G2|        |         |G2\r\n" + 
                "0000027792|001400000000000000000001224627|D2|        |         |G2\r\n" + 
                "0000027793|001400000000000000000001224627|D2|        |         |G2\r\n" + 
                "0000027794|001400000000000000000001224627|G6|        |         |G6";

        String[] lines = input.split("\r\n");
        String[][] parts = new String[lines.length][];
        for (int i = 0; i < lines.length; i++)
            parts[i] = lines[i].split("\\|");

        String currG = "G1";
        for (int i = 1; i < lines.length; i++) {
            if (parts[i][2].startsWith("G")) {
                System.out.println("Last D" + parts[i-1][2].substring(1) + " for " + currG + " is at line " + (i-1));
                lines[i-1] += "   LL";
                currG = parts[i][2];
            }
        }

        System.out.println();
        for (int i = 0; i < lines.length; i++)
            System.out.println(lines[i]);
    }
}

Edit2: with file reading and writing

public class Test {

    public static void main(String[] args) {

        String input = "path\\to\\input\\text.txt";
        String output = "path\\to\\output\\text.txt";
        BufferedReader in;
        BufferedWriter out;
        try {
            in = new BufferedReader(new InputStreamReader(new FileInputStream(input), "UTF-8"));
            out = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(output, true), "UTF-8"));

            String line, lastLine;
            lastLine = in.readLine();
            while ((line = in.readLine()) != null) {
                String[] parts = line.split("\\|");
                if (parts[2].startsWith("G")) {
                    lastLine += "   LL";
                }
                out.write(lastLine);
                out.write(System.lineSeparator());
                lastLine = line;
            }
            out.write(lastLine);

            in.close();
            out.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Handle the exceptions properly.

Here is how it works:

lastLine keeps the previous line read while line is searched for a new G. If one is found, then lastLine must contain the last D of the previous G. Here are the first iterations:

lastLine: G1 0
line:     D1 1
---
lastLine: D1 1
line:     D1 2
---
lastLine: D1 2
line:     D1 3
---
lastLine: D1 3
line:     G2 4
// line starts with G, so append LL to lastLine because it's the last D for G1.
---
lastLine: G2 4
line:     D2 5
...

Ohm

Thanks but input must be read using file reader and written using file writer and Its hard to give the input in code because the input has more than 30000 rows. So file reading will be efficient @Ohm – Magdau

@Magdau Your question is not about how to read and write a file, it's about how to parse and change it. The algorithm is the same. You read each line and so do I. Just apply the algorithm to your lines. Same for writing the files, just output to a file instead of the console. – Ohm

@Magdau When reading the file you only need to hold the i and i-1 lines. You will notice that my loops only looks at these 2. You don't need to read the 30000 lines at once. – Ohm

Unless you are his boss/teacher and thus know exactly what he wants, the code makes too many assumptions: 1. there's an implicit lexicographic ordering among G's and D's 2. input is sorted by G and D 3. he's searching the last D for parent G, or the same X for (GX, DX) I wrote a code sample with two distinct algorithms for sorted and unsorted input, and that deals with G and D as opaque strings – Gisser

@Gisser I noted my assumptions and presented algorithms for various cases. The code works for the given input and it is the asker's responsibility to give the details as I have stated in my question. Also, "too many" might be from your point of view, you can't know that. When the OP commented on my answer they were not concerned with my assumptions. – Ohm

I have this feeling that if he could understand your code and pick the right answer by himself, he would have not posted his question in the first place – Gisser

Thanks a lot it works perfectly but I have a doubt here. In the first iteration of reading the file, the value of lastline and line will be G1 but then how is it waiting to append LL till the G2 comes? and is the line, lastline=line doing all the wonders to append LL to last "D#" ?@Ohm – Magdau

I understood it perfectly but for the first G1, it will satisfy the condition but it has got no D# before G1 . The condition clearly says, append LL to the last D# for particular G#. Why isn't it throwing any error for first G1 as it has got no previous D#. @Ohm – Magdau

@Magdau Because I'm manually assigning lastLine = in.readLine() before the loops starts. That is to say, the check inside the loop for startsWith("G") only starts from line 1 which has D1 and not line 0 with G1. – Ohm

Awesome.. Thanks a lot @Ohm – Magdau

@Magdau You can also use a debugger and step line by line to figure these things out. It will show you what each variable holds and how it changes. – Ohm

Answer 2

If your input file is sorted by G and D columns you can do it in one pass:

public void processSorted() throws Exception {
    reset();

    previousG = null;

    while (next()) {
        finishLastLine();
        out.print(line);
        previousDep = values[2];
        previousG = group();
    }

    finishLastLine();
}

private void finishLastLine() {
    if (previousG != null && previousDep != null) {
        if (!group().equals(previousG) || !dep().equals(previousDep)) {
            if (previousG.equals("G1") && previousDep.equals("D1")) {
                out.print("|LL");
            } else if (previousG.equals("G2") && previousDep.equals("D2")) {
                out.print("|LL");
            }
        }
        out.println();
    }
}

If it's not sorted, you first need to find the last records for D1 and D2 (you need to reach the end to tell it), and then read the input once again to write the output file (unless you are told that processing time is important and there will always be enough memory to handle the input):

public void scanUnsorted() throws Exception {
    reset();
    while (next()) {
        switch (group()) {
            case "G1":
                if (dep().equals("D1"))
                    lastD1 = index;
                break;
            case "G2":
                if (dep().equals("D2"))
                    lastD2 = index;
                break;
        }
    }
}

public void write() throws Exception {
    reset();
    while (next()) {
        out.print(line);
        if (lastD1 >= 0 && index == lastD1
         || lastD2 >= 0 && index == lastD2
        ) {
            out.print("|LL");
        }
        out.println();
    }
    out.flush();
}

Here is the full runnable gist with test cases.

Gisser

Answer 3

First of all, you need to do two passes over the file; one to determine where each Last line for each GX is, and then to add the LL there. However, I think if we knew what your process is and why you need to put that there, we might be able to help more.

Exo

I don't think you need 2 passes, in my answer I do it with 1. – Ohm

I see two passes in your answer, @user1803551. The first pass prints the line numbers. The second pass prints the lines. – Desperado

It's just to make it clearer for the OP. I could put the print in the first loop. Besides, you wanted 2 loops for determine where each Last line for each GX is and add the LL there. These two are done in one loop regardless of printing. – Ohm

I have tried using list but it is getting appended at the end of G2 .If i check the previous index value it is showing as '0' for everything @Ohm – Magdau

@Magdau Who told you to use a list? What value? This is not my answer. – Ohm

Answer 4

There is no need to keep a list of lines - this could be significant if you are processing very large files. What you need is to store the previous line as you are reading the file, and write it "one line behind".

You can remove customerList (which you're not using anyway) and recTypeList from findFilesForFolder, and just add a single string.

I've taken parts of your code and added some of my own lines to show what I mean:

        String previousLine = null;
        String[] previousSplit = null;
        while((line = in.readLine()) != null) {
            split = line.split("\\|");

            // ...

            if (previousLine != null) {
                bufferedWriter.write(previousLine);
                if (!previousSplit[2].equals(split[2])) {
                    bufferedWriter.write("LL");
                }
                bufferedWriter.newLine();

                // ...

            }
            previousLine = line;
            previousSplit = split;
        }
        bufferedWriter.write(previousLine);
        bufferedWriter.write("LL");
        bufferedWriter.newLine();

(The above code just illustrates the technique; you'll have to add some more processing to your own code to check the last column - the group - and do the above independently for each group.)

Desperado