How to retain log4j2's plugin configs after server restart or altering xml config file

Question

I have developed a custom rolling file appender and it works fine. the only problem is that it's getting re-initialized each time I restart the server or change the log4j2.xml file(any part of the file) and suddenly all the previous logs would get wiped off. I haven't observed such behavior with the shipped default appenders so I wonder what can I do to retain my configurations.

<CustomAppender name="CustomAppender"
                                     fileName="${log.file.directory}/file.log"
                                     filePattern="${log.file.directory}/file.log.%d{yyyy-MM-dd}-%i.gz"
                                     immediateflush="true"
                                     append="true">
        <CustomLayout/>
        <Policies>
            <SizeBasedTriggeringPolicy size="3 KB"/>
            <TimeBasedTriggeringPolicy interval="1"/>
        </Policies>
    </CustomAppender>

P.S. I have tried to make it a singleton but aside from that it didn't work I really don't want to keep it away from being reconfigured, I just want to retain my previously generated logs.

update

Apparently everytime the server shutdown or log4j2.xml file is being altered the manager rebuilds the appender from scratch, though built-in appenders retain their state even after restart or reconfiguration. They do this by overriding "stop" method from AbstractOutputStreamAppender. I did the same for my appender but it still doesn't behave as I expect.

@Override
public boolean stop(long timeout, TimeUnit timeUnit) {
    setStopping();
    final boolean stopped = super.stop(timeout, timeUnit, false);
    setStopped();
    return stopped;
}

And this is the manager I used in appender builder:

final RollingFileManager manager = RollingFileManager.getFileManager(fileName, filePattern, append,
                isBufferedIo, policy, strategy, advertiseUri, layout, bufferSize, isImmediateFlush(),
                createOnDemand, filePermissions, fileOwner, fileGroup, getConfiguration());
        if (manager == null) {
            return null;
        }
        manager.initialize();

Marrano

Can you share the details of your custom appender? Somehow I suspect it does not open files in append mode. – Anthropophagite

@PiotrP.Karwasz looking into log4j2's docs I found this: Most Appenders use Managers. A manager actually “owns” the resources, such as an OutputStream or socket. When a reconfiguration occurs a new Appender will be created. However, if nothing significant in the previous Manager has changed, the new Appender will simply reference it instead of creating a new one. So I think I should somehow make the appender to shutdown its manager in "stop" method like the built-in appenders – Marrano

Answer 1

When a reconfiguration takes place the following happens:

1. The new configuration file is read and converted to a Node tree.
2. The Plugins associated with the Nodes are accessed and their corresponding classes are created.
3. If the configuration was created successfully it is started.
4. The Loggers are updated to reference the new configuration.
5. The old configuration is stopped.

As you might imagine, since there are two configuration running at the same time this can lead to problems. For example, if both components try to use the same port then the second configuration will likely fail. To avoid these problems Appenders in Log4j 2 use Managers. The Manager will typically have a "name" that includes any items that, if they have the same value in the new configuration, indicate that the Manager should be reused. This allows things like OutputStreams to remain open during the reconfiguration. However, the side effect of this is that if a parameter is changed on an appender and is not part of the "name" then the change in value may be ignored since the Manager was not modified.

So when creating a new Appender, care must be given to the name passed to the Manager and the Appender should try to update the relevant values in the Manager when they are not part of the name. Also, the Manager must account for it being started and stopped multiple times. AbstractManager will actually handle this for you.

Ballad

I found out that the FileManager that I'm using differs from the one that's being used by my other built-in appenders, however, based on your answer, should I extend AbstractManger instead of RollingFileManager or I should somehow obtain that particular manager's name somehow and use it as my own manager? – Marrano

RollingFileManager extends FileManager which extends OutputStreamManager which extends AbstractManager. So all you have to do is make sure the Manager name contains the correct information. – Ballad