Re: [SAGE] monitoring Tomcat

[Gabriel Krabbe <gabe@rtfs.de>]

Note: My Tomcat experience is on Solaris/SPARC.

On Wed, Sep 15, 2004 at 12:31:48PM -0400, Lance A. Brown wrote:
> 
> OS: Red Hat Enterprise Linux WS release 3 (Taroon Update 2)
> Tomcat: 4.1.30
> Java: Sun j2sdk 1.4.2_03
> Apache: 2.0.46 (RPM 2.0.46-38.ent)
> 
> We are currently running with the following options:
> 
> -Xrunhprof:file=$hprof  -Xmx2368m -Xms512m
> 
> $hprof is a filename for the java.hprof output.
> 
> We find that any large[r] -Xmx value causes java to error out
> immediately at startup.
> 
> I found some web pages talking about examining the hprof output to help 
> determine where memory allocation is going, but am not having my luck 
> with it.

There really shouldn't be any reason to have Xms a different value than
Xmx. Generally, the sizing decisions made by the VM for the various
memory areas are pretty damn good, so all you're really doing is
introducing the chance that, when the VM actually does want to grow
towards your maximum specified size, the memory might not be available
in the system, as it wasn't reserved. 
See also http://java.sun.com/docs/hotspot/gc1.4.2/

The profiling output is something that some gurus might find useful, but
only once you've established that you do indeed have a memory leak, and
then it's the developers' job to find out where - if they want to
profiling data, then provide it, but until then, it's a major
performance/stability issue, or so my experiments with it appeared.

You'll definitely want to collect information about the garbage
collection. That's where the parameter "-verbose:gc" comes in - at no
performance cost, it'll tell you all about minor and major collections.
The major ones are intersting; check your stdout for "Full GC"
information, the format of that line is self-documenting. Ignore
"unloading..." messages as unnecessary detail. Basically, if the second
number, i.e. the memory in use after the GC pass, steadily rises until the
number is just under the "Xmx" max heap size and then you crash with an
OutOfMemory, it's a memory leak within the application. If not, the GC
data will provide you with a bit of an insight into the tradeoff between
number of Full GC passes and duration of a single Full GC pass. 
See also http://java.sun.com/docs/hotspot/VMOptions.html

As for the 2368m being the maximum possible potential heap size: Yes.
See also http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5031531
See also http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4669640
See also http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4435069
(Free Reg. Reqd. for those three)

Putting it together, I'd suggest you use the options:
   -server -verbose:gc -Xms2000m -Xmx2000m 

Of course, it may well be that you've solved the problem by now and it
was none of the above. As far as I'm concerned, Java is the Windows of
programming languages: It's easy to get people started with it, it's
damn hard to get them to understand the underlying concepts with it, and
as far as troubleshooting is concerned, you're screwed.


Gabe