How to delete old logs with log4j2

Question

( F.Y.I. I already searched out many documents in Internet. I'm using storm-0.10.0-beta1. Configuration file of log4j2 in Storm is worker.xml )

Now, I try to use log4j2.

I'm searching out way of deleting old logs but I cannot find out. Part of configuration is like below.

    <RollingFile name="SERVICE_APPENDER"
             fileName="${sys:storm.home}/logs/${sys:logfile.name}.service"
             filePattern="${sys:storm.home}/logs/${sys:logfile.name}.service.%d{yyyyMMdd}">
        <PatternLayout>
            <pattern>${pattern}</pattern>
        </PatternLayout>
        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
        <DefaultRolloverStrategy max="9"/>
    </RollingFile>

At first, I expected that log files which are older than 3 days are removed.

But, actually, it doesn't.

So, I wonder whether there is a way to remove old logs or not.

If there is a way which I didn't catch yet, please notify me.

Commonwealth

have you ever tried whats happening ? As far as I remember log4j already deleted outdated files, so I assume log4j2 will also. Simply set your rolloverstrategy to 2 and change your system date to try this ... – Lettie

@Lettie Thank you for your comments. Even if the current project show that I doesn't work, I will try again. – Commonwealth

Answer 1

Since 2.5, Log4j supports a custom Delete action that is executed on every rollover.

You can control which files are deleted by any combination of:

1. Name (matching a glob or a regex)
2. Age ("delete if 14 days old or older")
3. Count ("keep only the most recent 3")
4. Size ("keep only the most recent files up to 500MB")

Users who need even more fine-grained control over which files to delete can specify a script condition using any supported JSR-223 scripting language.

Please check out the documentation, it has three full examples that may be useful.

For your question, this snippet should work:

<RollingFile name="rollingFile" 
      fileName="/path/app.log"
      filePattern="/path/app.%d{yyyy-MM-dd}.log"
      ignoreExceptions="false">
. . .
      <DefaultRolloverStrategy>
        <!--
          * only files in the log folder, no sub folders
          * only rolled over log files (name match)
          * only files that are 4 days old or older
        -->
        <Delete basePath="${sys:storm.home}/logs/" maxDepth="1">
          <IfFileName glob="*.service.????????" />
          <IfLastModified age="4d" />
        </Delete>
      </DefaultRolloverStrategy>
 . . .

<RollingFile>

Finally, be careful! There is no way to recover files deleted this way. :-)

Frigidarium

Is there an option to achieve this via Log4j2 2.3 version? – Monarchal

No, I'm afraid not. The Delete action was added in 2.5... You can propose a backport fix (ideally provide a patch with unit tests) but it won't be a straightforward port since the Delete implementation uses the Java 7 Path API heavily... – Frigidarium

Answer 2

You can find more background information in this JIRA entry for log4j:

https://issues.apache.org/jira/browse/LOG4J2-524

It seems to be the case that auto deleting old log files does not work when you only use a TimeBasedTriggeringPolicy

Lettie

Thank you for sharing the link. Finally, I found out that there is no way to remove time-based old logs until now. According to the last comment at this link (issues.apache.org/jira/browse/LOG4J2-435), the jobs of providing deleting old time-based logs is in progress. Therefore, I decided to use size-based rollover strategy using index, %i, – Commonwealth

Answer 3

Looking here https://logging.apache.org/log4j/log4j-2.7/manual/appenders.html you can use the Delete policy within the DefaultRolloverStrategy

Whats great about it is that you can use logical ands and ors to determine when a log should be deleted.

Here's a sample I have:

    <RollingFile name="RLog" fileName="logpath/apps.log" filePattern="%d{yyyyMMdd-HH}-apps-%i.log">
        <PatternLayout pattern="%d{YYYY-MM-dd HH:mm:ss} [%-5level] %t - %c: %msg%n" />
        <Policies>
            <OnStartupTriggeringPolicy />
            <SizeBasedTriggeringPolicy size="20MB" />
        </Policies>
        <DefaultRolloverStrategy max="2">
            <Delete basePath="logpath" maxDepth="2">
                <IfFileName glob="*">
                <!-- Deletes log files older that match any of the conditions below. -->
                    <IfAny>
                        <IfAccumulatedFileCount exceeds="50" />
                        <IfLastModified age="P30D" />
                        <IfAccumulatedFileSize  exceeds="1GB" />
                    </IfAny>
                </IfFileName>
            </Delete>
        </DefaultRolloverStrategy>
    </RollingFile>

If you look above it will delete ANY file at the base logpath directory that meets any of the next conditions, those being if:

• the total # of files exceeds 50 OR
• if the file(s) are at least 30 days old OR
• The total file size of all the logs together exceeds 1GB

Mouthful