How to set source host address in Python Logging?

Question

There is a script, written in Python, which parse sensors data and events from number of servers by IPMI. Then it sends graph data to one server and error logs to the other. Logging server is Syslog-ng+Mysql

So the task is to store logs by owner, but not by script host.

Some code example:

import logging
import logging.handlers

loggerCent = logging.getLogger(prodName + 'Remote')
ce = logging.handlers.SysLogHandler(address=('192.168.1.11', 514), facility='daemon')
formatter = logging.Formatter('%(name)s: %(levelname)s: %(message)s')
loggerCent.setLevel(logging.INFO)

loggerCent.addHandler(ce)

loggerCent.warning('TEST MSG')

So I need to extend the code so I could tell to syslog-ng, that the log belongs to the other host. Or some other desigion.

Any ideas?

UPD:

So it looks like there is the way to use LogAdapter. But how can use it:

loggerCent = logging.getLogger(prodName + 'Remote')
ce = logging.handlers.SysLogHandler(address=('192.168.1.11', 514), facility='daemon')
logging.basicConfig(level=logging.DEBUG,
                        format='%(asctime)-15s %(name)-5s %(levelname)-8s host: %(host)-15s %(message)s')
loggerCent.addHandler(ce)
loggerCent2 = logging.LoggerAdapter(loggerCent,
                               {'host': '123.231.231.123'})
loggerCent2.warning('TEST MSG')

Looking for the message through TcpDump I see nothing about host in LoggerAdapter What am I doing wrong?

UPD2:

Well, I did't find the way to send host to syslog-ng. Though it is possible to send first host in chain, but I really can't find the way to sent it through Python Logging.

Anyway, I made parser in syslog-ng: CSV Parser

parser ipmimon_log {
        csv-parser(
    columns("LEVEL", "UNIT", "MESSAGE")
    flags(escape-double-char,strip-whitespace)
        delimiters(";")
        quote-pairs('""[](){}')
    );
};

log {
    source(s_net);
    parser(ipmimon_log);
    destination(d_mysql_ipmimon);
};

log {
    source(s_net);
    destination(d_mysql_norm);
    flags(fallback);
};

Then I send logs to the syslog-ng delimited by ;

Bordure

So you'll need to modify the logging server to be aware of the event originator identifier in the log messages pushed over the wire. But before we get there, where are you actually logging these messages? There are a number of approaches for adding context to logging messages, but depending upon where you are doing your logging, one of another might make more sense. – Furthermore

Sounds good. In fact, I need log different host or some some kind of tag in a different field. – Bordure

The only way to get the data out of the message at the receiver is by parsing the message string. SysLogHandler just pushes out a string with the Formatted message prepended by a syslog specific tag that encodes severity and facility and a terminator '/000' on the end to the remote machine. You'll need to define the format to serialize your metadata in to the message (in Format) yourself. You can get it there with a LoggerAdapter or some other mechanism. – Furthermore

Ok, looks like it can help. But I can't understand the way to use SysLogHandler and LogAdapter. – Bordure

Answer 1

edit-rewrite

You are missing the critical step of actually adding your Formatter to your Handler. Try:

loggerCent = logging.getLogger(prodName + 'Remote')
loggerCent.setLevel(logging.DEBUG)
ce = logging.handlers.SysLogHandler(address=('192.168.1.11', 514), facility='daemon')
formatter = logging.Formatter('%(host)s;%(message)s')
ce.setFormatter(formatter)
loggerCent.addHandler(ce)
loggerCent2 = logging.LoggerAdapter(loggerCent, {'host': '123.231.231.123'})
loggerCent2.warning('TEST MSG')

Note that you won't be running basicConfig any more, so you will be responsible for attaching a Handler and setting a log level for the root logger yourself (or you will get 'no handler' errors).

Furthermore

Oh wait, I reread your post a few times, and I think this isn't right... you log all the messages to the same logging server, but basically need to be able to provide custom metadata for your log messages to the logging server (the source machine from which the aggregation server received the IPMI info). Is that right? Do you have access to the message parser on the log server? – Furthermore

Yes, correct. And I do have access to the logger machine. Syslog-ng and Mysql. If it is possible to parse message, get a server id or address and put it to the separate field - that wold be great. – Bordure