[squid-users] Log entry in access_log cut off when exceeding a certain length..

From: Essad Korkic <essad.korkic_at_gmail.com>
Date: Thu, 13 Sep 2012 16:24:40 +0200

Hi All,

I have an issue with the access_log of squid.

It seems that a standard access_log entry cannot exceed a certain length.

During some logfile analysis I noticed I had similar usernames, but
not quite, as if they were not complete.
After digging a bit deeper I found that if users browse to very long
URL's, the log entry' is being cut off.

For example:

1346672509.258 104 10.22.111.22 TCP_MISS/200 1911 GET
http://ad-emea.doubleclick.net/adj/mobile.pkw.gebraucht.bmw/_315;sz=120x600,160x600,200x600,250x600,300x600,336x280,2x2;price=09;typ=01;ch=01;ccm=0;cap=0;reg=13;km=01;fuel=01;gear=01;ac=01;restr=0;s=0;intid=0;advid=0;tsn=0;hsn=0;sch=0;a=01;art=0;pr=34117;kw=0;ma=0;ez=01.2012;regy=2012;con=0;site=01;cm=bmw;mod=315;mwst=01;lang=de;city=000;custid=0;us=51;us=52;us=53;us=55;us=56;us=57;us=58;us=59;us=60;us=62;us=63;us=64;us=65;us=66;us=67;us=69;us=71;us=72;us=73;us=74;us=75;us=76;us=150;mg=21;hfd=0;p=01;cl=0;ab=0;tile=2;n=001;n=004;n=009;n=015;n=016;n=018;n=020;n=022;n=027;n=030;n=035;n=039;n=043;n=047;n=049;n=051;n=053;n=055;n=057;n=059;n=061;n=063;n=065;n=067;n=069;n=071;n=075;n=077;n=079;n=081;n=083;n=085;n=087;n=089;n=092;n=093;n=095;n=098;n=099;n=101;n=103;n=105;n=107;n=109;n=111;n=114;n=115;n=117;n=119;n=121;n=125;n=127;n=129;n=131;n=135;n=136;n=138;n=999;l=01;!c=des_pl01;oba=29068061;ord=5247789107326666
myusername_at_REALM.DOMA

Especially these ad-url's are really annoying..

But while analyzing logs, you can see that the username is cut off.

I've counted some of these lines, and all were above 997 characters.
So I'm guessing that there is something in the squid code which cut's
off these long log lines.

This is my logformat: "%ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un
%Sh/%<A %mt"
(common)

I don't think there is a way to specify a max length for an URL,
except for example using "strip_query_terms", but due to auditing
requirements, this is not really an option.
And it would not help in the URL mentioned above, as it does not
contain question marks.

Has anyone encountered this as a problem?

I've attached a few of the log entry's that are sanitized and very long...

Thanks,
Essad Korkic

Received on Thu Sep 13 2012 - 14:24:46 MDT

This archive was generated by hypermail 2.2.0 : Tue Sep 18 2012 - 12:00:02 MDT