Rolling log Files & removing old log files

Question

I am working on a Java SOAP based webservice application where I am writing stdout to a text file as log for our reference. That file is growing enormously, so I need to check for the size of the file... For example if the file size crosses 10 Mb, I have to create another file.

Like this, I have to create 10 files, rotating one after the other until ten files. After reaching ten files, I have to delete the starting files and start creating again...

How can I delete files after the no. of files will become 10?

Rajasthani

look at Log4j, it will do everything for you, once you set it up "properly" – Wildlife

Answer 1

I use logback to do this. The example below is a time based rolling policy. Depending upon how much data your outputting during your logs, this may work for you as-is.

Also, as a bonus, my config file tosses the log into HTML to make it easy to view for management types who want to look though the log file.

Relevant part of the config file:

 <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>logs\logFile.html</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <!-- daily rollover -- >
        <fileNamePattern>logs\logFile.%d{yyyy-MM-dd}.%i.html</fileNamePattern>
        <timeBasedFileNamingAndTriggeringPolicy
            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
            <!-- or whenever the file size reaches 10MB -- >
            <maxFileSize>10MB</maxFileSize>
        </timeBasedFileNamingAndTriggeringPolicy>
        <!-- keep 10 days worth of history -->
        <maxHistory>10</maxHistory>
    </rollingPolicy>

    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
        <charset>UTF-8</charset>
        <layout class="ch.qos.logback.classic.html.HTMLLayout">
            <pattern>%d{HH:mm:ss.SSS}%thread%level%logger%line%msg</pattern>
        </layout>           
    </encoder>
</appender> 

<root level="DEBUG">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="FILE" />
</root>

relevant Maven dependencies:

    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-core</artifactId>
        <version>1.0.12</version>
    </dependency>

    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.0.12</version>
    </dependency>

Eugenol

Wouldn't that keep fewer than 10 days of history if your logs get so big that they get split up into multiple files by the <maxFileSize>10MB</maxFileSize>? – Carrot

Answer 2

I see a lot of answers telling you to use Log4J, but you can use Java's own logger to do this by simply creating a FileHandler:

Handler handler =
    new FileHandler("%h/MyService-%g.log", 10 * 1024 * 1024, 10);
handler.setLevel(Level.ALL);
Logger.getLogger("").addHandler(handler);

Squarrose

thanks foe the answer....but i want to delete the older files....after it reaching 10 files......is this scenario can be handeled by filehandler.....please reply me back....i am doing this work from past one week...if possible expalin with code – Rajasthani

I haven't tried it, but my understanding of FileHandler's documentation is that "rotating set of files" means the same files are used over and over. Meaning, once MyService-9.log has reached its size limit, the logger truncates and overwrites MyService-0.log (assuming the FileHandler was constructed with a count of 10). So files are not deleted per se, but there will never be more files than the number you specify in the count argument. – Squarrose

Yes, I just tested this to see that it does replace the older logs, thank you. – Waterfall

You also need a true flag as fourth parameter on FileHandler constructor to append to the file (until specified size is reached) otherwise new file is created ignoring the size of previous file. – Bari

Answer 3

In log4j.xml you can try the following:

<appender name="fileappender" class="org.apache.log4j.RollingFileAppender">
      <param name="file" value="applog.log"/>
      <param name="Append" value="true" />
      <param name="MaxBackupIndex" value="10"/>

      <layout class="org.apache.log4j.PatternLayout">
         <param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n"/>
      </layout>
   </appender>

The value tells log4j.xml to only keep 10 rotated log files around.

Alternatively, if you are using a properties file (instead of the xml)

log4j.appender.File=org.apache.log4j.RollingFileAppender
log4j.appender.File.File=applog.log
log4j.appender.File.Append=true
log4j.appender.File.layout=org.apache.log4j.PatternLayout
log4j.appender.File.layout.ConversionPattern=%d{ABSOLUTE} %-5p [%c] %m%n
log4j.appender.[appenderName].MaxBackupIndex = 10

Lorolla

I speficied MaxBackupIndex as you wrote but got "log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.rolling.RollingFileAppender." – Barrios

ah, it looks like I'm using the RollingFileAppender from log4j extras which does not have this ... See #23945873 – Barrios

Answer 4

If you use java.util.logging.Logger, you can do it with FileHandler.

Source: kodejava

package org.kodejava.example.logging;

import java.util.logging.Logger;
import java.util.logging.FileHandler;
import java.util.logging.SimpleFormatter;
import java.io.IOException;

public class RollingLogFile {
    //
    // Set a small log file size to demonstrate the rolling log files.
    //
    public static final int FILE_SIZE = 1024;

    public static void main(String[] args) {
        Logger logger = Logger.getLogger(RollingLogFile.class.getName());

        try {
            //
            // Creating an instance of FileHandler with 5 logging files
            // sequences.
            //
            FileHandler handler = new FileHandler("myapp.log", FILE_SIZE, 5, true);
            handler.setFormatter(new SimpleFormatter());
            logger.addHandler(handler);
            logger.setUseParentHandlers(false);
        } catch (IOException e) {
            logger.warning("Failed to initialize logger handler.");
        }

        logger.info("Logging information message.");
        logger.warning("Logging warning message.");
    }
}

Edgeworth

Is it possible to do it using logging.properties file? – Kalfas

Answer 5

Most logging frameworks provide what you're looking for. In logback you should be able to achieve it by properly configuring a RollingFileAppender:

RollingFileAppender extends FileAppender with the capability to rollover log files. For example, RollingFileAppender can log to a file named log.txt file and, once a certain condition is met, change its logging target to another file.

and

RollingPolicy is responsible for the rollover procedure which involves file moving and renaming.

http://logback.qos.ch/manual/appenders.html

Noellenoellyn

i have to delete file after the no of log file created reaches to 10 – Rajasthani

and i have to write java code for this...i am new to webservices and java so please help me out – Rajasthani

Answer 6

Log4j can do this. Specifically the RollingFileAppender class.

Ouphe