Centos/Linux setting logrotate to maximum file size for all logs

Question

we use logrotate and it runs daily ... now we have had some situations where logs have grown significantly (read: gigbaytes) and killing our server. So now we would like to set a maximum filesize to the logs ....

can I just add this to the logrotate.conf?

size 50M

and would it then apply to all log files? Or do I need to set this on a per log basis?

Or any other advice?

(ps. I understand that if you want to be notified is the log grows like described and what we want to do is not ideal - but it is better than not being able to logon anymore because there is no space available)

thanks, Sean

Twosided

Answer 1

It specifies the size of the log file to trigger rotation. For example size 50M will trigger a log rotation once the file is 50MB or greater in size. You can use the suffix M for megabytes, k for kilobytes, and G for gigabytes. If no suffix is used, it will take it to mean bytes. You can check the example at the end. There are three directives available size, maxsize, and minsize. According to manpage:

minsize size
              Log  files  are  rotated when they grow bigger than size bytes,
              but not before the additionally specified time interval (daily,
              weekly,  monthly, or yearly).  The related size option is simi-
              lar except that it is mutually exclusive with the time interval
              options,  and  it causes log files to be rotated without regard
              for the last rotation time.  When minsize  is  used,  both  the
              size and timestamp of a log file are considered.

size size
              Log files are rotated only if they grow bigger then size bytes.
              If size is followed by k, the size is assumed to  be  in  kilo-
              bytes.  If the M is used, the size is in megabytes, and if G is
              used, the size is in gigabytes. So size 100,  size  100k,  size
              100M and size 100G are all valid.
maxsize size
              Log files are rotated when they grow bigger than size bytes even before
              the additionally specified time interval (daily, weekly, monthly, 
              or yearly).  The related size option is  similar  except  that  it 
              is mutually exclusive with the time interval options, and it causes
              log files to be rotated without regard for the last rotation time.  
              When maxsize is used, both the size and timestamp of a log file are                  
              considered.

Here is an example:

"/var/log/httpd/access.log" /var/log/httpd/error.log {
           rotate 5
           mail [email protected]
           size 100k
           sharedscripts
           postrotate
               /usr/bin/killall -HUP httpd
           endscript
       }

Here is an explanation for both files /var/log/httpd/access.log and /var/log/httpd/error.log. They are rotated whenever it grows over 100k in size, and the old logs files are mailed (uncompressed) to [email protected] after going through 5 rotations, rather than being removed. The sharedscripts means that the postrotate script will only be run once (after the old logs have been compressed), not once for each log which is rotated. Note that the double quotes around the first filename at the beginning of this section allows logrotate to rotate logs with spaces in the name. Normal shell quoting rules apply, with ,, and \ characters supported.

Duckboard

Nice. A small note, I discovered that maxsize and minsize doesn't work well with old distros (like centos5 and ubuntu 10.10 ... the latter is EOL anyway). – Muff

A little off-topic, but @Muff is right; maxsize was added in version 3.8.1. To help find which version of your OS has which version of logrotate, try version specific man page for debian, version map & same for centos. – Samaria

zeeshan, the data size suffix, e.g. 1G is that one gigabyte or one gibibyte? I know there's a difference between between what hardware manufacturers market for drive sizes and what your filesystem uses. Do you know which unit logrotate uses? – Phytosociology

Answer 2

As mentioned by Zeeshan, the logrotate options size, minsize, maxsize are triggers for rotation.

To better explain it. You can run logrotate as often as you like, but unless a threshold is reached such as the filesize being reached or the appropriate time passed, the logs will not be rotated.

The size options do not ensure that your rotated logs are also of the specified size. To get them to be close to the specified size you need to call the logrotate program sufficiently often. This is critical.

For log files that build up very quickly (e.g. in the hundreds of MB a day), unless you want them to be very large you will need to ensure logrotate is called often! this is critical.

Therefore to stop your disk filling up with multi-gigabyte log files you need to ensure logrotate is called often enough, otherwise the log rotation will not work as well as you want.

On Ubuntu, you can easily switch to hourly rotation by moving the script /etc/cron.daily/logrotate to /etc/cron.hourly/logrotate

Or add

*/5 * * * * /etc/cron.daily/logrotate

To your /etc/crontab file. To run it every 5 minutes.

The size option ignores the daily, weekly, monthly time options. But minsize & maxsize take it into account.

The man page is a little confusing there. Here's my explanation.

minsize rotates only when the file has reached an appropriate size and the set time period has passed. e.g. minsize 50MB + daily If file reaches 50MB before daily time ticked over, it'll keep growing until the next day.

maxsize will rotate when the log reaches a set size or the appropriate time has passed. e.g. maxsize 50MB + daily. If file is 50MB and we're not at the next day yet, the log will be rotated. If the file is only 20MB and we roll over to the next day then the file will be rotated.

size will rotate when the log > size. Regardless of whether hourly/daily/weekly/monthly is specified. So if you have size 100M - it means when your log file is > 100M the log will be rotated if logrotate is run when this condition is true. Once it's rotated, the main log will be 0, and a subsequent run will do nothing.

So in the op's case. Specficially 50MB max I'd use something like the following:

/var/log/logpath/*.log {
    maxsize 50M
    hourly
    missingok
    rotate 8
    compress
    notifempty
    nocreate
}

Which means he'd create 8hrs of logs max. And there would be 8 of them at no more than 50MB each. Since he's saying that he's getting multi gigabytes each day and assuming they build up at a fairly constant rate, and maxsize is used he'll end up with around close to the max reached for each file. So they will be likely close to 50MB each. Given the volume they build, he would need to ensure that logrotate is run often enough to meet the target size.

Since I've put hourly there, we'd need logrotate to be run a minimum of every hour. But since they build up to say 2 gigabytes per day and we want 50MB... assuming a constant rate that's 83MB per hour. So you can imagine if we run logrotate every hour, despite setting maxsize to 50 we'll end up with 83MB log's in that case. So in this instance set the running to every 20 minutes or less should be sufficient.

Ensure logrotate is run every 20 mins. In this case, in 20mins the log file might be < 50MB, but in 40min, over 50MB in which case it will rotate. If we rotate every 30mins, in 30mins it's < 50MB (no rotation), in 1hr it may have built to > 80MB already in which case the file will be rotated but much bigger than we were expecting. So make sure you take rotation time into account.

*/20 * * * * /etc/cron.daily/logrotate 

Blacking

can you explain liitle more about "The size option ignores the daily, weekly, monthly time options". – Phylis

is it almost same as maxsize? – Phylis

Ok, added explanation for size – Blacking

Thanks for explaination. It is much better than the original one in man page. – Cock

a bit confusing i tried to add like you gave in your example but dont knowwhy its not executing. but when i tried manually its working. – Disfavor

@BhargavPatel - if it's Ubuntu it should work. Other Linux Variants may be set up differently. Or possibly your crond daemon process is not running. – Blacking

Note that the logrotate cron script originally found in /etc/cron.daily/logrotate may contain the following lines: # skip in favour of systemd timer, if [ -d /run/systemd/system ]; then, exit 0, fi. If it does, and /run/systemd/system is a directory on your system, then you need to comment out/delete those lines, otherwise moving /etc/cron.daily/logrotate to /etc/cron.hourly isn't going to make any difference. See this answer for more information – Acropolis

Great answer but there is a subtle mistake. If the logs are growing at 83MB per hour then the attempt to rotate at 30 minutes will find a 42MB log and so it will not rotate. This is actually the Nyquist sampling theoreom, we need to set the period to 20 minutes so that at 40 minutes (second attempt) it will see a 55MB log and do the rotation. – Chiliasm

@Chiliasm - you're right! fixed – Blacking

why's this upvoted? i could not learn a thing by reading this. – Fenland

Answer 3

To simplify the explanation further:

Logrotate size parameter is only applied when logrotate runs.

So for example, if you set your logrotate to run every hour and when size reaches 5MB. If the file reaches over 5MB before an hour is reached - the file will in effect grow to be bigger than 5MB because logrotate was never called on the file.

It is imperative that logrotate is run on the file frequently enough to check its size. Therefore when using the size parameter in logrotate, just let the timing of the logrotate be handled by something else. (i.e. cron/script). This means you can omit specifying time in your logrotate config.

For example if I want to rotate a file at size 5MB - how quick that file reaches that size will determine how often logrotate should run. Suppose it takes about 10 minutes on average to get to 5MB, firstly the rotate settings at minimum has:

/var/log/path/the.log {
           rotate 1 (#number of rotations)
           size 5M
       }

create directory /etc/custom-rotate.d

Save the above in /etc/custom-rotate.d/customlog

Permissions: sudo chmod 644 /etc/custom-rotate.d/customlog

Create a config file:

cat << EOF | sudo tee /etc/custom-rotate.conf
# packages drop custom log rotation information into this directory
include /etc/custom-rotate.d
EOF

Permission: sudo chmod 644 /etc/custom-rotate.conf

Then have a cron to run every, say 5 minutes (giving space for possible anomalies) to check the size.

sudo su

crontab -e

Add entry:

*/5  *  *  *  * /usr/sbin/logrotate /etc/custom-rotate.d/customlog

Reload cron:

 sudo service cron reload

Therefore every 5 minutes logrotate will run and if the size is greater than 5M it will rotate the logs.

Quaggy

Tip: If you want to be sure that the cron is running say: */10 * * * * /usr/sbin/logrotate /etc/custom-rotate.d/customlog > root/customlog.log -> if the file customlog.log exisits it means your cron ran - alternatively just tail -f var/log/syslog – Quaggy