[SAGE] SUMMARY: circular log trimming tool (long)

[dnb@ccs.neu.edu (David N. Blank-Edelman)]

Howdy-

  Sorry it has taken so long to write this up. I wanted to make sure
  to feed back a summary to the list of the information I
  collected. Many thanks to everyone who responded to my question. If
  I omit your name below, please accept my apology.

ORIGINAL QUESTION:

  I was looking for a program that could take a single log file and
  keep it under a constant size by trimming older lines off the
  top. This request was different from the usual log wrangling
  programs that perform what John Sellens succinctly called "cascading
  rename of log files." Our goal was to keep only a single copy of
  each log file that was made to stay under a certain size.

WHAT THE ORIGINAL QUESTION OMITTED:

  I should have specified the OS I needed this for (since some of the
  answers pointed to program that shipped with a specific
  distribution). I also should have indicated just why I wanted
  something like this. Just for the record: 

    - I was looking for a program to run under Solaris. 

    - We are implementing a service for our students so they can play
      around with CGI-scripts et. al. We have a program that splits
      the Apache error and access logs into separate log files per
      user (extract_UserDir_logs.c by Dan Harkless, at
      http://harkless.org/dan/software/). I didn't want each user to
      have N log files to look at, just two, kept to a finite
      size. That's why the usual approach to logfile rotation was not
      desired here.

ANSWERS THAT HIT THE MARK

  trimlog (http://www.securityfocus.com/tools/118) suggested by
  Ken McKinlay and Shane B. Milburn was exactly what we were looking
  for. It took a few minor tweaks to get running under a modern
  Solaris, but it works great.

ANSWERS THAT DIDN'T HIT THE MARK BUT WERE STILL COOL

  logrotate
  (ftp://ftp.redhat.com/pub/redhat/linux/rawhide/SRPMS/SRPMS/logrotate-3.6.8-1.src.rpm
  was the latest source distribution I could find, hope you have rpm
  on hand to unpack it, grr.)

     Several folks pointed me at this package (Nick Stoughton, Doug
     Hughes), but as far as I can tell it can do virtually anything
     one would want to do having to do with logfile rotation, but it
     didn't know how to trim files in the manner I needed. Doug
     actually noted this deficiency and suggested having it's
     post-rotate command fire off another program which would do the
     actual trimming.  Only the plus side, we are planning to use it
     for another project (see below).

  newsyslog (solaris ports: http://anders.fix.no/solaris/ and here:
  http://www.weird.com/~woods/projects/newsyslog.html) 

     Found in *BSD, this was the other hometown favorite (Julian
     C. Dunn, chris@telerama, David H. Wolfskill, Chuck
     Yerkes). Again, as far as I can tell (and I could be wrong), it
     too just does a great job of rotation and does not have the
     ability to trim logs by itself.

     One big plus (from my point of view) vs. logrotate is that it's
     config is kept in essentially table form making it easy to tell
     at a glance just which files are being rotated and when. It's not
     as featureful as logrotate so I'm going to steal this config file
     format and post-process it into a logrotate config file upon
     demand.

  Rot[t]Log (http://www.nongnu.org/rottlog/) pointed to by Ade Rixon
  is a bash-based system very similar to logrotate.

ANSWERS THAT WERE FARTHER OFF THE MARK BUT EVEN COOLER

  cronolog (http://www.cronolog.org) suggested by John Sellens is a
  cool program that acts as a filter for log data and "does the right
  thing with it at that moment". By this I mean it takes input from
  some source and writes that input to the right file based on a file
  template. In some ways I find it cooler to put the data in the right
  place to start with rather than have to rotate it there at some
  later time.

  rotatelogs (ships with apache, doc here:
  http://httpd.apache.org/docs/programs/rotatelogs.html) suggested by
  shades2@iinet works on a similar principle but is a little less
  flexible.

So again, many thanks to the people mentioned and anyone I have
forgotten. 

        -- dNb