Where to configure internal tomcat7 stdout/stderr log files
Asked Answered
S

5

20

I'm using Apache Tomcat 7.0.40 with Log4j config according to http://tomcat.apache.org/tomcat-7.0-doc/logging.html

Everything's working as expected, except that some log files are created, which are actually not configured in my log4j.properties:

log4j.rootLogger=INFO, CATALINA

# Define all the appenders
log4j.appender.CATALINA=org.apache.log4j.RollingFileAppender
log4j.appender.CATALINA.File=${catalina.base}/logs/catalina.log
log4j.appender.CATALINA.MaxFileSize=3MB
log4j.appender.CATALINA.MaxBackupIndex=10
log4j.appender.CATALINA.Append=true
log4j.appender.CATALINA.Encoding=UTF-8
log4j.appender.CATALINA.layout = org.apache.log4j.PatternLayout
log4j.appender.CATALINA.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.LOCALHOST=org.apache.log4j.RollingFileAppender
log4j.appender.LOCALHOST.File=${catalina.base}/logs/localhost.log
log4j.appender.LOCALHOST.MaxFileSize=3MB
log4j.appender.LOCALHOST.MaxBackupIndex=10
log4j.appender.LOCALHOST.Append=true
log4j.appender.LOCALHOST.Encoding=UTF-8
log4j.appender.LOCALHOST.layout = org.apache.log4j.PatternLayout
log4j.appender.LOCALHOST.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.MANAGER=org.apache.log4j.RollingFileAppender
log4j.appender.MANAGER.File=${catalina.base}/logs/manager.log
log4j.appender.MANAGER.MaxFileSize=3MB
log4j.appender.MANAGER.MaxBackupIndex=10
log4j.appender.MANAGER.Append=true
log4j.appender.MANAGER.Encoding=UTF-8
log4j.appender.MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.HOST-MANAGER=org.apache.log4j.RollingFileAppender
log4j.appender.HOST-MANAGER.File=${catalina.base}/logs/host-manager.log
log4j.appender.HOST-MANAGER.MaxFileSize=3MB
log4j.appender.HOST-MANAGER.MaxBackupIndex=10
log4j.appender.HOST-MANAGER.Append=true
log4j.appender.HOST-MANAGER.Encoding=UTF-8
log4j.appender.HOST-MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.HOST-MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding=UTF-8
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

# Configure which loggers log to which appenders
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost]=INFO, LOCALHOST
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager]=\
  INFO, MANAGER
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager]=\
  INFO, HOST-MANAGER

The files which get created although they are NOT in my configuration are:

  • commons-daemon.yyyy-MM-dd.log
  • tomcat7-stderr.yyyy-MM-dd.log
  • tomcat7-stdout.yyyy-MM-dd.log

As you see they use a DailyRollingFileAppender, which is not desired, as I need a maintenance free system. So I would prefer a RollingFileAppender with maxBackupIndex to prevent unlimited growth of the logs.

So where can I configure the stdout, stderr and commons-daemon logs? Is there some configuration somewhere or can I override the config with my own Log4j config? Thx

Sheepshank answered 5/11, 2013 at 10:53 Comment(4)
you want to get rid of those logs ?Stunning
not necessarily. I want to change them to rollover a defined number of times, so that I don't have 365 logs after a year. But for the interest - how to get rid of them at all?Gorski
Did you find any solution to your problem?Swifter
Check this link, it's works for me on tomcat 8.5 Tomcat log file locationTeofilateosinte
J
20

I foresee us having a similar requirement/facing same issue some time later in future on one of our systems. So was keen on finding some solution/workaround myself.

Looked into the commons-daemon first to see if there is any configuration we can do to achieve a maxbackupindex on the logs; but the search didn't help much.

However i found two approaches that people take for dealing with "tomcat logs crossing limits".

First Approach: Configure internally in Tomcat

  1. Find context.xml in Tomcat in the path :-

    YourTomcatInstallDir\conf\context.xml  
    
  2. Edit the context tag to add swallowOutput="true" like below:-

    <Context swallowOutput="true">
    

    (This swallows all your stdout/stderr and redirects to your underlying logging system.)

  3. In your log4j.properties; add config to redirect your org.apache.catalina logs to your own log file.

    Your logger will look like this:-

    log4j.logger.org.apache.catalina=INFO, YourAppender
    

    (On YourAppender now you can set MaxBackupIndex and MaxFileSize to achieve your defined number of rollovers and a limit on the log size. )

If the above approach doesn't work; you can try using an external utility as mentioned in the second approach

Second Approach: Configure externally with "logrotate"

There's a simple tool called logrotate available to achieve the desired effect on logs you do not have control over. Here are some links to get you started.

The usage of this tool is pretty straightforward and a quick run through these links should get you what you need.

Janik answered 25/9, 2014 at 17:40 Comment(0)
S
14

I had the same issue and finally stumbled on the solution. Tomcat 7 for windows has a tomcat properties application called Monitor Tomcat. On the logging tab you need to delete the "auto" value in the redirect stdout and stderr input boxes. Just leave it blank and it won't create those files. Then you can use log4j to create rolling log files for those critical files for future debug purposes.

Hope that helps.

Shimberg answered 9/12, 2013 at 17:1 Comment(4)
I was not able to find a solution to the commons-daemon log file but since it seems to only log entries related to the starting and stopping of the service it's impact is negligible in my instance.Shimberg
Can you clarify the last part, "you can use log4j to create rolling log files for those critical files.."? How can I redirect stdout and stderr to rolling log files?Becket
This answer really helpful to mute the tomcat's logging while you run tomcat as a window service.Crystlecs
My Windows service tomcat8-stdout.date.log files all had one line, "Commons Daemon procrun stdout initialized" and the tomcat8-stderr.date.log files provided similar information to the catalina.date.log files, like filtering the same stuff to INFO or higher and formatting it differently. Suppressing both by deleting auto was a nice clean-up. Thank you.Pul
S
9

Disable log file creation inside Procrun GUI

Short version:

So where can I configure the stdout, stderr and commons-daemon logs?

Inside the Procrun GUI.

Is there some configuration somewhere or can I override the config with my own Log4j config?

No, I wish, but I don't think so. But here's a dirty hack to stop Procrun from logging altogether: Set Log Path: to NOTADRIVE: (including the trailing :). Then none of the three log files (commons-daemon, stderr, stdout) will be created.

Medium version: I'm not super happy with the solution but for the Tomcats I have to administer I've set up several things:

  • I've minimized what goes into catalina.out/stdout/stderr files from my webapps: I've removed all console logging from the webapps' own logging configuration. (After all: no need to have the same logs both inside the webapps' own Log4J-managed logs AND in catalina.out/stderr/stdout?)
  • I've minimized what goes into catalina.out/stdout/stderr files from Tomcat itself: As per Apache's recommendation I've removed ConsoleHandler from logging.properties. (The reasoning is the same: no need to log things twice.)
  • I've (for now) decided to keep Procrun's logs (commons-daemon, stdout, stderr). But I hope to somehow limit the size they can grow to by using a combination of a weekly restart along with a cleanup job. The automated weekly restart ensures that Procrun will start a fresh (and timestamped) set of these three files. And the automated weekly delage32 job cleans up these old files after a while.
  • I've considered using LogRotateWin but couldn't get it work with files locked for access. (Not even with the copytruncate directory that is supposed to do just that.) Maybe I'll give it another try in the future.
  • Another thing not tried yet: Tomcat's swallowOutput attribute ** Update: 2022-03-30, I've tried it and it didn't work at all for my webapps. Also the description on Apache sounded scary. So I've settled for NOT using swallowOutput.

Long version see below. (This is a rambling deep dive. Consider yourself warned.)


Tomcat as a Windows service

The default Windows installer will register Tomcat as a service.

The internal mechanism for this is called the Commons Daemon Service Manager.

There are Unix and Windows versions of this. And the Win32 version is called "Procrun".

Procrun has a GUI and strictly speaking this GUI is called "Prunmgr". -- But I will just gloss over this and pretend it's all called "Procrun".

Procrun does NOT use the standard Windows Event Logging Framework. Instead Procrun will create some log files. Let's start the GUI and see what they are.

Starting the Procrun GUI

The Windows icon is just labeled Monitor Tomcat.

In a default install this is a shortcut that starts "C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin\Tomcat7w.exe" //MS//Tomcat7

And starting this will take you directly to Procrun's Logging tab.

"Monitor Tomcat" icon

Procrun's "Logging" tab

Here's a screenshot of Procrun's "Logging" tab after fresh install with all defaults: "Logging" tab of Procrun app

(Install details: apache-tomcat-7.0.40.exe on Win10 x64.)

Procrun can generate several files:

  1. The commons-daemon log file.
  2. The Pid file. We don't care about this right now.
  3. The Redirect Stdout log file.
  4. The Redirect Stderror log file.

Procrun makes 3 log files.

So there are 3 log files. And Procrun will only start a new one when you restart Tomcat.

So Procrun will NOT create a new file if the old log file gets too large. Also Procrun will not do anything about old log files lying around in the logs directory and making the logs directory itself too large.

Let's look at the individual log files and what they do: The commons-daemon log file, the Redirect Stdout log file and the Redirect Stderror log file.

commons-daemon log file

This is file contains information about the workings of Procrun itself. Example:

PS C:\> Get-Content -Path "C:\Program Files\Apache Software Foundation\Tomcat 7.0\logs\commons-daemon.2020-09-08.log"

[2020-09-08 16:06:28] [info]  [ 4276] Commons Daemon procrun (1.0.15.0 64-bit) started
[2020-09-08 16:06:28] [info]  [ 4276] Running 'Tomcat7' Service...
[2020-09-08 16:06:28] [info]  [17304] Starting service...
[2020-09-08 16:06:29] [info]  [17304] Service started in 1135 ms.
[2020-09-08 16:09:24] [info]  [11520] Stopping service...
[2020-09-08 16:09:25] [info]  [11520] Service stop thread completed.
[2020-09-08 16:09:25] [info]  [ 4276] Run service finished.
[2020-09-08 16:09:25] [info]  [ 4276] Commons Daemon procrun finished

The default log level is info.

So the path and file name are constructed like this: Log path + "\" + Log prefix + "." + date formatted as YYYY-MM-DD + ".log".

"Log prefix" empty

If you leave Log prefix empty, then it will default to commons-daemon. And if you close Procrun and reopen it, then Log prefix will be set to commons-daemon. In other words: You can not leave this empty. And if you try, then Procrun will silently change it back.

"Log prefix" absolute path

If you point Log path to an inexistent directory, then ProcRun will try to create this directory.

"Log prefix" anything other than an absolute path

If you put in IDONTCARE, then this will happen:

PS C:\> handle.exe commons-daemon
Nthandle v4.22 - Handle viewer
Copyright (C) 1997-2019 Mark Russinovich
Sysinternals - www.sysinternals.com

Tomcat7.exe        pid: 4880   type: File           1B8: C:\Windows\System32\IDONTCARE\commons-daemon.2020-09-08.log


PS C:\> handle -nobanner IDONTCARE
Tomcat7.exe        pid: 4880   type: File           1B8: C:\Windows\System32\IDONTCARE\commons-daemon.2020-09-08.log
Tomcat7.exe        pid: 4880   type: File           1BC: C:\Windows\System32\IDONTCARE\tomcat7-stdout.2020-09-08.log
Tomcat7.exe        pid: 4880   type: File           1D4: C:\Windows\System32\IDONTCARE\tomcat7-stdout.2020-09-08.log
Tomcat7.exe        pid: 4880   type: File           1D8: C:\Windows\System32\IDONTCARE\tomcat7-stderr.2020-09-08.log
Tomcat7.exe        pid: 4880   type: File           1DC: C:\Windows\System32\IDONTCARE\tomcat7-stderr.2020-09-08.log

Yes. It created a directory called IDONTCARE inside C:\Windows\System32. That can't be a good idea.

But if you put in I\DONT\CARE, then NO file will be created. Then none of the three log files will be created.

PS C:\> handle -nobanner commons-daemon
No matching handles found.

The same is true for putting in something that has a colon in it like "NOTADRIVE:".

"Log path" empty

If you leave Log path empty, then Procrun will silently default to %SYSTEMROOT%\System32\LogFiles\Apache. (See below.) I was surprised by this. (But it is actually the documented default behavior.)

PS C:\> handle.exe commons-daemon
Nthandle v4.22 - Handle viewer
Copyright (C) 1997-2019 Mark Russinovich
Sysinternals - www.sysinternals.com

Tomcat7.exe        pid: 7504   type: File            88: C:\Windows\System32\LogFiles\Apache\commons-daemon.2020-08-31.log

Redirect Stdout log file and Redirect Stderr log file

I will only describe Stdout here. The behavior for Stderr is identical.

"Redirect Stdout" auto

The default setting for Redirect Stdout is auto. Then the path and filename are constructed like this: Log path + "" + your-service-name-here + "-stdout." + date formatted as YYYY-MM-DD + ".log".

Capitalization of "auto" does not seem to matter. "auto" in all lower case will work just the same as "AUTO" in all upper case.

Example: With the default install, your service name will be "Tomcat7". And if you start Tomcat on Septemter 8th 2020, then the full name will be: C:\Program Files\Apache Software Foundation\Tomcat 7.0\logs\tomcat7-stdout.2020-09-08.log

"Redirect Stdout" empty

If you leave Redirect Stdout empty, then no Stdout log file will be created. (I double checked with handle.exe.) This behavior is different from the behavior for the commons-daemon log file.

"Redirect Stdout" absolute path

If you set Redirect Stdout to an absolute path, then the log file will be created there.

If the directory does not exist, then it will not be created. This behavior is different from the behavior for the commons-daemon log file.

If the filename part of the absolute path is "auto" then this carries no special meaning. It will NOT create timestamped file. It will just create a file with name "auto".

Relative Path (Words other than "auto")

If you set Redirect Stdout to "mystdout", then a file will be created with full path: "C:\Program Files\Apache Software Foundation\Tomcat 7.0\mystdout". Note that this is outside of the Log Path directory. This behavior is different from the behavior for the commons-daemon log file.

The path that Procrun uses here comes from Procrun | "Startup" | "Working Path:"

(The default is C:\Program Files\Apache Software Foundation\Tomcat 7.0)

(This path is stored in the Windows registry here: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\Tomcat7\Parameters\Start => WorkingPath -- This is on my Win10 x64 Machine. Not sure why WOW6432Node is used.)

Workarounds

There are several workarounds that come to my mind:

  1. Make a hacky workaround: Set Procrun's Log Path: to NOTADRIVE:. Then none of the three log files will be created.

  2. Make a less hacky workaround: Disable creation of stdout/stderr files (see below). You will STILL need a way to clean up the old "commons-daemon" files. (I know of no way to disable creation of those files. Just emptying the text-field will NOT do the trick.) But at least those should be small in size.

  3. Set up automated restart of Tomcat and/or the whole machine. This will create a file timestamped with the restart date. You will STILL need a way to clean up the old commons-daemon/stdout/stderr files. But then at least they won't be locked as "in use" and you can just delete them, maybe with something like DelAge.

  4. Use an external Logrotate-for-Windows utility (such as LogRotateWin to both rotate the current file and clean up the old rotated files. For an example see this answer: https://serverfault.com/questions/358172/equivalent-of-logrotate-for-windows/1032685#1032685

  5. Do not use Procrun and start your Windows Tomcat in some other way. This will create none of the commons-daemon/stdout/stderr files. -- "Procrun" is just one of several existing (supposedly) general purpose java service wrappers. I write "supposedly" here, because "general purpose" seems to have been a design goal for the "commons daemon", but in reality I've never seen it used anywhere other than with Tomcat. -- While googling around for "java service wrapper" I found several projects but I haven't tried any of them.

I think the official position can roughly be summarized as "What? Your catalina/stdout/stderr files are getting too large? Well, you shouldn't be logging anything to those anyway." In other words: If you're using Procrun at all, then the only officially supported option seems to be number 2 (and 4).

Further reading

Schizothymia answered 29/1, 2019 at 14:57 Comment(0)
Z
1

On windows you'll have to go to tomcat/bin/service.bat and edit PR_LOGPATH variable (all of them). And probably reinstall service by "service remove" and "service install" On linux systems solution must be very similar

Zoometry answered 22/8, 2014 at 8:57 Comment(0)
A
0

For those who just want to customize the location where these 3 logs are created:

If you run Tomcat as a service and you want to customize where the commons-daemon, stderr and stdout files reside, then you can start your service with the command line parameter --LogPath or set on the server machine an environment variable PR_LOGPATH with your custom path.

This is documented also here together with other params: https://tomcat.apache.org/tomcat-8.0-doc/windows-service-howto.html

Architrave answered 3/4, 2019 at 20:54 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.