Log4perl: How do I dynamically load appenders at runtime?

Question

I'd like to have modules managing their logging at runtime, but without having everything referring to a single monolithic config file. When dealing with processes running under different permissions, I really don't want to deal with each process needing to be able to access every log on the system when they're only writing to a subset of them.

However, I'm not finding much documentation in the Log4perl manual on how to initialize additional appenders from a configuration file at runtime. http://metacpan.org/pod/Log::Log4perl::Appender references an add_appender method, but that works on instantiated appender objects instead of conf files. It also doesn't define the logger objects and the logger->appender relations.

I tried having each package init from its own conf, but that simply clobbers the existing config each time it's initalized. What I'd like to do is something along the lines of:

my $foo = Foo->new() ## Checks Log::Log4perl::initialized(), sees that it
                     ## hasn't been initalized yet, inits Log4perl from foo.conf
my $bar = Bar->new() ## Checks Log::Log4perl::initialized(), sees that it
                     ## has been initalized. Adds appenders and loggers defined
                     ## in bar.conf into the initialized configuration

How can I parse and add the configuration into the current config?

Edit: Probalem with using a package variable is that this is just a Moose role being consumed by various classes, pretty much just a MooseX::Role::Parameterized version of Ether's answer in Making self-logging modules with Log::Log4perl. Thus, my logger is getting composed into the library consuming it, and I don't have a global variable I can work on each time I use it.

Though..

If I declare a global variable outside of the MooseX::Role::Parameterized role block, would each and every class that consumes the role be using that same conf variable?

Bimbo

Anyone? I've glanced over the source for Log4perl a bit but I can't be the first person who's ever wanted to do this. I'm trying to make a Moose paramererized role I can apply to classes that allow me to pass a config file into the role, and it just Does The Right Thing by either initing the conf file or appending the conf file config to the current initialized Log4perl. – Bimbo

@Oesor: Package global is still global even if declared in Moose role. It should be easy to modify solution below to use roles (instead of BUILD mechanism you can use around BUILDARGS with very similar code). – Lanna

@Oesor: Updated code below per previous comment. – Lanna

Didn't realize that; figured anything using PACKAGE inside the role would refer to the package the role got composed into, not the role's package. – Bimbo

Actually -- one last question. This obviously relies on every initialization of Log4perl being through the logger role. Is there any way I can check the current source of the Log4perl initialization so that if I init it using a straight Log::Log4perl->init('foobar.conf') in a script then use a library using this logger role, I can append the library's init onto the current initialization? – Bimbo

It would be probably more difficult, one possible way would be to imitate Log::Log4perl::Config's _init method to do custom initialization. – Lanna

At minimum it is possible to detect cases when initialization was called outside Logger based classes - when %log_configs is empty and Log::Log4perl is initialized, you know that it happened and at least you can emit warning/error. – Lanna

Answer 1

While I was hoping to avoid it, if I parse the config files myself I can then access the configuration in perl via the API documented in http://search.cpan.org/perldoc?Log::Log4perl. Namely,

  ########################
  # Initialization section
  ########################
  use Log::Log4perl;
  use Log::Log4perl::Layout;
  use Log::Log4perl::Level;

     # Define a category logger
  my $log = Log::Log4perl->get_logger("Foo::Bar");

     # Define a layout
  my $layout = Log::Log4perl::Layout::PatternLayout->new("[%r] %F %L %m%n");

     # Define a file appender
  my $file_appender = Log::Log4perl::Appender->new(
                          "Log::Log4perl::Appender::File",
                          name      => "filelog",
                          filename  => "/tmp/my.log");

     # Define a stdout appender
  my $stdout_appender =  Log::Log4perl::Appender->new(
                          "Log::Log4perl::Appender::Screen",
                          name      => "screenlog",
                          stderr    => 0);

     # Have both appenders use the same layout (could be different)
  $stdout_appender->layout($layout);
  $file_appender->layout($layout);

  $log->add_appender($stdout_appender);
  $log->add_appender($file_appender);
  $log->level($INFO);

While the other method works, there's too many caveats for me to be comfortable using it (Gee I used this library, why'd my logging stop?) -- it's just too surprising for my tastes.

Instead, I think I'm going to see if I can't get from config file to Log::Log4perl state by looking through how to use Log::Log4perl::Config::PropertyConfigurator, which is delegated to by ->init when parsing a config file is needed. If I go over the data structure that returns, i can compare changes to the initialization on a logger-by-logger and appender-by-appender basis and modify the initialized state appropriately, handle namespace collisions properly, etc.

Bimbo

Answer 2

You can remember what config files was already loaded (%log_configs hash in code below). When new class arrives, you can reread all configs, merge it together and init Log::Log4perl again using string reference parameter to init.

I generally prefer having a single log configuration per application, because of easier maintenance and reload capability.

package Logger;
use Moose::Role;
use Log::Log4perl;

our %log_configs = ();

around BUILDARGS => sub {
    my $orig  = shift;
    my $class = shift;

    my $config_name = lc($class) . '.conf';

    # if the config is not integrated yet
    if(! defined $log_configs{$config_name}) {
        $log_configs{$config_name} = 1;

        # reload all configs including new one
        my $config_text = '';
        for my $file (sort keys %log_configs) {
            $config_text .= "\n" . do {
                local $/;   # slurp
                unless(open my $fh, "<", $file) {
                    warn "$file could not be open\n";
                    '';
                }
                else {
                    <$fh>
                }
            };
        }

        # refresh config
        Log::Log4perl::init(\$config_text);
    }

    return $class->$orig(@_);
};


package Foo;
use Moose;
with 'Logger';
use Log::Log4perl ':easy';

sub BUILD {
    ERROR 'Foo reporting';
}


package Bar;
use Moose;
with 'Logger';
use Log::Log4perl ':easy';

sub BUILD {
    INFO 'Bar reporting';
}


package main;

my $foo = Foo->new;
my $bar = Bar->new;

Lanna

That's more or less how I'd do it. Probably a good idea to wrap init in an eval and drop the most recent config file out if it dies in init. Nothing worse than a bad logging config file bringing unrelated applications crashing down. It wouldn't be hard to add a flag value to the %log_configs hash to indicate a bad config file. – Yapok

@Mark Tozzi - certainly proper error handling should be added if such approach is to be used in real application. Please consider the snippet as proof-of-concept. Another useful addition might be a property to specify name of log config, defaulting to class name, rather that this hard-coded sample (which btw does not work for namespaced classes like Foo::Bar). – Lanna