Django server not sending logs to Logstash

Question

I am using ELK stack for centralised logging from my Django server. My ELK stack is on a remote server and logstash.conf looks like this:

input {
    tcp {
    port => 5959
    codec => json
  }
}
output {
  elasticsearch {
    hosts => ["xx.xx.xx.xx:9200"]
  }
}

Both services elasticsearch and logstash are working (checked using docker-compose logs logstash).

My Django server's settings file has logging configured as below:

LOGGING = {
  'version': 1,
  'handlers': {
        'logstash': {
            'level': 'INFO',
            'class': 'logstash.TCPLogstashHandler',
            'host': 'xx.xx.xx.xx',
            'port': 5959, # Default value: 5959
            'version': 0, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
            'message_type': 'django',  # 'type' field in logstash message. Default value: 'logstash'.
            'fqdn': True, # Fully qualified domain name. Default value: false.
            'tags': ['django.request'], # list of tags. Default: None.
        },
  },
  'loggers': {
        'django.request': {
            'handlers': ['logstash'],
            'level': 'DEBUG',
  },
}
}

I run my Django server and Logstash handler handles the logs as console shows no logs. I used the python-logstash library in Django server to construct the above conf, but the logs are not sent to my remote server.

I checked through many questions, verified that services are running and ports are correct, but I have no clue why the logs are not being sent to Logstash.

Thirion

Are you sure that "django.request" doesn't filter out all events? – Ochre

Django request is supposed to send all the requests logs – Thirion

Have you tried change the fqdn to False? – Enculturation

Yep tried but it did not work as well – Thirion

Answer 1

Looking at the configuration, logger "django.request" is set to level "DEBUG" and the handler "logstash" is set to level "INFO". My guess is that the handler will not process DEBUG messages. I'm not sure though.

Set the same level for the logger and the handler to test that it works.

What level to use depends on what you want from your logs. In this case I guess level INFO would suffice.

If not already take a look at Django logging

NOTE: From comments it seems not to solve the problem but I hope it is useful anyway.

UPDATE:

I tried the following configuration and it catches 404 and 500 errors in the "debug.log".

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
    'logfile': {
        'level': 'WARNING',
        'class': 'logging.FileHandler',
        'filename': os.path.join(PROJECT_DIR, 'debug.log'),
    },
},
'loggers': {
    'django.request': {
        'handlers': ['logfile'],
        'level': 'WARNING',
        'propagate': True,
    },
}}

With this test configuration the logstash handler should at least receive the message/logrecord. If no luck I suggest to try debug the logstash.TCPLogstashHandler and the SocketHandler (inherited by TCPLogstashHandler) to make sure they receive the emitted record.

Southeastwards

Your point seems to be correct so what would you suggest for solving this? – Thirion

I suggested to use level INFO. Depending on how log messages are created, sometimes it's good to be able to switch DEBUG logging on and off depending which environment is used. For instance django uses it for mail_admins. – Southeastwards

@ArpitSolanki django.request logging just WARNING and ERROR messages. docs.djangoproject.com/en/1.11/topics/logging/… – Pharmaceutical

@ArpitSolanki What messages are you trying to log? – Southeastwards

tried this answer with producing errors like 404, 500 but did not work – Thirion

@DanielBackman Thanks for your effort but it did not solve the problem. I can still award you the bounty if you add a small note to your answer that it did not solve the problem but helped in logging part of the code – Thirion

@ArpitSolanki I'll try to find why the logger is not working as expected. – Southeastwards

Answer 2

I faced the same issue but after scrambling around a lot, I made it work by changing django.request to django.server. I got the idea when I used only django as a logger name in the python code and then found out the actual logger_name from log data stored in elasticsearch. Below is the updated code

LOGGING = {
  'version': 1,
  'handlers': {
        'logstash': {
            'level': 'INFO',
            'class': 'logstash.TCPLogstashHandler',
            'host': 'xx.xx.xx.xx',
            'port': 5959, # Default value: 5959
            'version': 0, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
            'message_type': 'django',  # 'type' field in logstash message. Default value: 'logstash'.
            'fqdn': True, # Fully qualified domain name. Default value: false.
            'tags': ['django.request'], # list of tags. Default: None.
        },
  },
  'loggers': {
        'django.server': {  # Here is the change
            'handlers': ['logstash'],
            'level': 'DEBUG',
      }
  },
}

Refer to this for more detail on django logging https://docs.djangoproject.com/en/1.11/topics/logging/#id3

Dibbuk

so later should I do this before logging logging.getLogger('django.server') ? the . is confusing me – Duotone

Answer 3

Your logging configuration is correct. You need to mention index name in elasticsearch config part of logstash conf. Update your logstash config to

input {
    tcp {
    port => 5959
    codec => json
  }
}
output {
  elasticsearch {
    hosts => ["xx.xx.xx.xx:9200"]
    manage_template => false
    index => "djangologs"
  }
}

If you are using google cloud or AWS, open port/update firewall rules.

Periphrasis

Answer 4

Thank you Tasneem,

This worked on my machine:

mysite/settings.py

LOGGING = {
  'version': 1,
  'handlers': {
      'logstash': {
          'level': 'DEBUG',
          'class': 'logstash.TCPLogstashHandler',
          'host': 'xx.xx.xx.xx',
          'port': 9600, # Default value: 5959
          'version': 1, # Version of logstash event schema. Default value: 0 (for backward compatibility of the library)
          'message_type': 'django',  # 'type' field in logstash message. Default value: 'logstash'.
          'fqdn': False, # Fully qualified domain name. Default value: false.
          'tags': ['django.request'],# list of tags. Default: None.
      },
  },
  'loggers': {
      'django.server': {
          'handlers': ['logstash'],
          'level': 'DEBUG',
      },
  },
}

logstash# cat logstash-django.conf

input {
  tcp {
    port => 9600
    codec => json
  }
}
output {
    elasticsearch {
      hosts => ["https://your.elasticsearch.com:port"]
      user => ["xx"]
      password => ["xx"]
      index => "djangotest"
      document_type => "%{type}"
    }
   stdout {
      codec => rubydebug {metadata => true }
  }
}
   

Timbal