Close log files

Question

This project has been handed down to me, so I do not know much about it. There is a method where log (java.util.logging.Logger) is used and it creates two log files:

First file: fileName.log

Second file: fileName.log.lck

In Linux when I do lsof, I see these two files as open. How do I close these two files?

The reason why I want to close these files is this method is run multiple times a day and after couple of weeks the number of open files reaches a limit (around 1000), at which point our system stops working. When we restart our process ("Job Controller" which does the logging) the number of log files open goes to 0 and it works again.

This is what's been done to do logging

private static Logger log = Logger.getLogger(MyClass.class.getPackage().getName());
try{
    log.logp(Level.SEVERE, "com.MyClass", "run", "It failed");
}

This is what I tried to do to close the files in the finally block but it didn't work

finally{
    Handler[] handler =   log.getHandlers();
    for(Handler h: handler){
        h.close();
    }
}

Greywacke

why do you want to close these files? – Decani

Closing these files will probably cause problems for you.. – Lineation

What happenes is after couple of weeks when the number of open files reaches a certain limit(I think it's around 1000), our system just crashes. So when we restart the process(Job Controller), the number of files open goes back to 0 and everything works. Instead of restarting the process every so often manually, we want our code to do it. – Greywacke

@BrendanLong my boss believes that the problem is caused by the number of open log files. Why do you think closing these files will cause problems? – Greywacke

What are the filenames? Is there one for each day? I don't know why the application would keep so many open at once for typical logging. Looking at the Java spec and the OpenJDK source code, close() should work, but will throw an error if you try to log something again afterwards. – Nelsonnema

Doesn't sound like closing two files will fix the problem of hitting the open file limit. Or as Steve says, do these two files get opened, and stay open daily? – Sanctimony

I don't think this is where your problem is. In most applications, you'll only have a couple log files open at any time. I'd look for other cases in your application when you're not closing files (or even releasing their references, since files should get cleaned up eventually even if you don't remember to call close()). In case you actually do need a bunch of files open, it's possible to increase the limit. – Lineation

The actual log file name is JobController_runId_0.log and JobController_runId_0.log.lck ... (runId is some number) multiple jobs run everyday, so everytime a job runs, it creates log files. – Greywacke

@BrendanLong Everytime I run a job(or a process) it creates two log files and the number of files just keep on adding up. These log files show up when I do "lsof" – Greywacke

@Sanctimony at the end of a week, the number of open log files reaches over 1000 and that's when the system crashes. – Greywacke

So what didn't work about your finally block? That appears to be the correct way to do it. Are you getting an error or is it just not closing the files? – Lineation

@BrendanLong I'm not getting an error, it's just not closing the files. I count number of files open before and after I run the process to confirm. – Greywacke

According to answers on this question JUL isn't very good at handling multiple output files. It may be worthwhile to investigate Log4j or logback to see if they handle this better. – Lineation

Answer 1

First solution

If you do not want to modify your code use: How to send java.util.logging to log4j? java.util.logging.Logger to Logback using SLF4J?

I use log4j or logback. Both have Rolling File Appender (old files are removed) or Date/Time File appender.

Second solution

For logging the best usage is rolling file.

String filePattern = " fileName%.log";
int limit = 1000 * 1000; // 1 Mb
int numLogFiles = 3;
FileHandler fh = new FileHandler(filePattern, limit, numLogFiles);

// Add to logger
Logger logger = Logger.getLogger(MyClass.class.getPackage().getName());
logger.addHandler(fh);

I do not know if you can add globally file handler.

Spruik

Answer 2

I simply use:

LogManager.getLogManager().reset();

this will cancel all your log settings (log file path, file name pattern, formatter...) but it will stop using the logger close the lock file and release logger to

Shoe

It's very easy way! Thanks! – Eason

Answer 3

I'll leave it here for myself:

public class JQLogger {

    public static Logger getLogger(String logName) {
        if (LogManager.getLogManager().getLogger(logName) != null
                && LogManager.getLogManager().getLogger(logName).getHandlers().length > 0)
            return LogManager.getLogManager().getLogger(logName);

        String logPath = "log/";
        if (Files.notExists(Paths.get(logPath)))
            try {
                Files.createDirectories(Paths.get(logPath));
            } catch (IOException e) {
                e.printStackTrace();
            }
        Logger logger = Logger.getLogger(logName);
        logger.setLevel(Level.INFO);
        logger.setUseParentHandlers(false);

        ConsoleHandler consoleHandler = new ConsoleHandler();
        FileHandler fileHandler;
        try {
            fileHandler = new FileHandler(logPath + logger.getName() + ".%g.log", 52428800, 3, true);
            fileHandler.setFormatter(JQLogger.getFormatter());
            logger.addHandler(fileHandler);
        } catch (SecurityException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

        consoleHandler.setFormatter(JQLogger.getFormatter());
        logger.addHandler(consoleHandler);

        return logger;
    }

    private static Formatter getFormatter() {
        return new SimpleFormatter() {
            private String format = "[%1$tF %1$tT.%1$tL] [%2$-7s] %3$s %n";

            @Override
            public synchronized String format(LogRecord lr) {
                return String.format(format, new Date(lr.getMillis()), lr.getLevel().getLocalizedName(),
                        lr.getMessage());
            }
        };
    }

    public static void closeLogger(Logger logger) {
        for (Handler h : logger.getHandlers()) {
            h.close();
            logger.removeHandler(h);
        }
    }

}

Paramatta