java.lang.OutOfMemoryError: Java heap space for 100000 records

Question

Trying to write an excel file using the following code

public static void main(String[] args) 
    {
          XSSFWorkbook workbook = new XSSFWorkbook(); 

            XSSFSheet sheet = workbook.createSheet("Book Data");

            Map<String, Object[]> data = new TreeMap<String, Object[]>();
            data.put("1", new Object[] {"ID", "NAME", "LASTNAME"});
            for(int i=2;i<100000;i++)
            {               
                data.put(String.valueOf(i), new Object[] {i, "Name"+i, "LastName"+i});
            }

            Set<String> keyset = data.keySet();
            int rownum = 0;
            for (String key : keyset)
            {
                Row row = sheet.createRow(rownum++);
                Object [] objArr = data.get(key);
                int cellnum = 0;
                for (Object obj : objArr)
                {
                   Cell cell = row.createCell(cellnum++);
                   if(obj instanceof String)
                        cell.setCellValue((String)obj);
                    else if(obj instanceof Integer)
                        cell.setCellValue((Integer)obj);
                }
            }
            try
            {

                FileOutputStream out = new FileOutputStream(new File("D:\\nameandlname.xlsx"));
                workbook.write(out);
                out.close();
                System.out.println("nameandlname.xlsx written successfully.");
            } 
            catch (Exception e) 
            {
                e.printStackTrace();
            }
    }

VM arguments -Xms512M -Xmx1024M

eclipse.ini:

-startup
plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.1.R36x_v20100810
-product
org.eclipse.epp.package.jee.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
1024M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms512m
-Xmx1024m
-vm
C:/Program Files (x86)/Java/jdk1.6.0_21/bin/javaw.exe

And finally the error getting:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at org.apache.xmlbeans.impl.store.Saver$TextSaver.resize(Saver.java:1592)
    at org.apache.xmlbeans.impl.store.Saver$TextSaver.preEmit(Saver.java:1223)
    at org.apache.xmlbeans.impl.store.Saver$TextSaver.emit(Saver.java:1144)

Fragmental

Why would you increase the heap of eclipse and not that of your application? – Bidding

I don't know how to do that. Just following this to resolve this issue. Trying a sample java file to test. – Fragmental

Are you able to perhaps buffer and write your data over time, rather than storing -all- of it in memory and then writing? – Lundgren

@Rogue: I haven't tried that. How to do that? Any links? – Fragmental

Answer 1

What you need is SXSSF (Streaming Usermodel API).

SXSSF (package: org.apache.poi.xssf.streaming) is an API-compatible streaming extension of XSSF to be used when very large spreadsheets have to be produced, and heap space is limited.

There is an example on that page too: you need to substitute XSSFWorkbook with SXSSFWorkbook.

Toft