Re: Converting squid logs to common log format

Martin,
> The script is available from:
> <URL:http://www.unimelb.edu.au/pwebstats/squid2common.pl>
We've been using an earlier version with pwebstats for a while now, thanks
for making your scripts available.

A couple of small changes,

The line,
        if($host =~ /\d\d\.\d\d/)
needs to be changed to
         if($host =~ /\d+\.\d+\.\d+\.\d+/)

It misses doing a host address lookup on addresses of the form
        222.1.222.1
And uses whatever name was translated last time round the loop.

Additionally
        elsif( $type eq "TCP_HIT" || ($type eq "TCP_IFMODSINCE" && $size >= 220) ){

should include udp hit objects,
        elsif( $type eq "TCP_HIT" || $type eq "UDP_HIT_OBJ" ||
                ($type eq "TCP_IFMODSINCE" && $size >= 220)){

A lot of sites are doing more than 4G a day so %ld overflows in the
subroutine commas.

 - Mark.

Here's a context diff:

*** squid2common.pl.orig Fri Aug 23 16:31:41 1996
--- squid2common.pl Fri Aug 23 16:47:26 1996
***************
*** 167,173 ****
          if( $hosts{$host} ) { $name = $hosts{$host};}
          else
          {
! if($host =~ /\d\d\.\d\d/)
                  {
                          @address = split(/\./,$host);
                          $addpacked = pack('C4',@address);
--- 167,173 ----
          if( $hosts{$host} ) { $name = $hosts{$host};}
          else
          {
! if($host =~ /\d+\.\d+\.\d+\.\d+/)
                  {
                          @address = split(/\./,$host);
                          $addpacked = pack('C4',@address);
***************
*** 192,198 ****
                  $sizeproxied += $size;
                  $totalproxied += 1;
          }
! elsif( $type eq "TCP_HIT" || ($type eq "TCP_IFMODSINCE" && $size >= 220) ){
              if( $size != 0 ) {
                          $line_new = "$hosts{$host} - - [$www_date] \"$htype $url HTTP/1.0\" 200 $size\n";
                          if( ! $onefile ) { print CACHE_FILE "$line_new"; }
--- 192,199 ----
                  $sizeproxied += $size;
                  $totalproxied += 1;
          }
! elsif( $type eq "TCP_HIT" || $type eq "UDP_HIT_OBJ" ||
! ($type eq "TCP_IFMODSINCE" && $size >= 220) ){
              if( $size != 0 ) {
                          $line_new = "$hosts{$host} - - [$www_date] \"$htype $url HTTP/1.0\" 200 $size\n";
                          if( ! $onefile ) { print CACHE_FILE "$line_new"; }
***************
*** 246,252 ****
  
  sub commas {
          local($_)=@_;
! $_ = sprintf "%ld", $_;
          1 while s/(.*\w)(\w\w\w)/$1,$2/;
          $_;
  }
--- 247,253 ----
  
  sub commas {
          local($_)=@_;
! $_ = sprintf("%.0f", $_);
          1 while s/(.*\w)(\w\w\w)/$1,$2/;
          $_;
  }
Received on Thu Aug 22 1996 - 23:50:24 MDT