How to delete older rolled over log4j2 logs, keeping up to 10 files?

Question

What I want is:

• Maximum of 10 log files, in total
• Each log file, not more than 50MB in size.

Thus the logs folder never grows over (50MB *10 )= 500MB.

But it seems my log4j2 config is not properly done.

What is happening is:

• Logs do roll over after 50 MB
• But there are upto 10 logs kept per day
• Thus there is no limit of number of log files kept in log folder (since for eg, in 2 days, 20 logs of 50mb each have collected)

Here is the config:

<Configuration status="WARN">
    <Appenders>
        <RollingFile name="RollingFile" fileName="log/my.log" filePattern="log/my-%d{MM-dd-yyyy}-%i.log">
          <PatternLayout>
                <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
          </PatternLayout>
          <Policies>
                <OnStartupTriggeringPolicy />
                <TimeBasedTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="50 MB"/>
          </Policies>
          <DefaultRolloverStrategy max="10"/>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="RollingFile"/>
        </Root>
    </Loggers>
</Configuration>

What am I doing wrong?

Dicarlo

Answer 1

Since 2.5, Log4j supports a custom Delete action that is executed on every rollover.

You can control which files are deleted by:

1. Name (matching a glob or a regex)
2. Age ("delete if 14 days old or older")
3. Count ("keep only the most recent 3")
4. Size ("keep only the most recent files up to 500MB")

The above can be combined. Instead of only specifying a size condition to keep disk usage down to max 500MB, it's a good idea to also match the name so you don't inadvertently delete unrelated files.

Users who need even more fine-grained control over which files to delete can specify a script condition using any supported JSR-223 scripting language.

Please check out the documentation, it has three full examples that may be useful.

For your question, this snippet may work:

  <DefaultRolloverStrategy>
    <!--
      * only files in the log folder, no sub folders
      * only rolled over log files (name match)
      * either when more than 10 matching files exist or when the max disk usage is exceeded
    -->
    <Delete basePath="log" maxDepth="1">
      <IfFileName glob="my-??-??-????-*.log">
        <IfAny>
          <IfAccumulatedFileSize exceeds="500 MB" />
          <IfAccumulatedFileCount exceeds="10" />
        </IfAny>
      </IfFileName>
    </Delete>
  </DefaultRolloverStrategy>

As an aside, note that you can compress log files on rollover to make them take up less disk space.

Finally, be careful! There is no way to recover files deleted this way. :-)

Cloudberry

any way to do in log4j 2.1? – Hartley

Not as a Log4j function. You would need to script something that removes old logs in your operating system... – Cloudberry

I strongly recommend upgrading though. Any reason you can't upgrade? – Cloudberry

Only reason was to get approval from my manager :). Any way I upgraded to 2.7. Thanks – Hartley

Answer 2

The TimeBasedTriggeringPolicy works based of the filePattern. Basically, the smallest unit of time in the file pattern (%d) is the triggering time interval. In your case the value is 'dd' hence the policy is triggered every day.

The presence of %i in your filePattern keeps multiple log files for a day. I would recommend trying without the %i in the filePattern.

Straitjacket