[squid-users] Squid ICAP TCP Zero Window Size Issues on Solaris

From: Niall O'Cuilinn <nocuilinn_at_amdocs.com>
Date: Fri, 8 Apr 2011 15:06:15 +0300

Hi all,

We have recently been having trouble with large POST requests causing threads to get blocked on our ICAP Server. We are using Squid 3.0 STABLE 15

We have tracked the problem down to TCP Zero Window Size issues when the ICAP Server is trying to write the POST body back to Squid in the REQMOD response.

It seems that Squid stops processing the ICAP response but never properly closes the connection. The issue occurs for POST requests to unavailable web servers. Our guess is that Squid determines that the remote host is unavailable and decides not to waste resources processing the ICAP response.

We can only reproduce the issue on Solaris.

We tried upgrading to Squid 3.1.12 but found a similar but different issue, where for the same large POST requests the TCP Zero Window issue occurs on the connection between the web client and Squid, rather than between the ICAP Server and Squid.

We have opened two bugs relating to the issue, bug 3191 for the original 3.0.15 issue and 3190 for the 3.1.12 issue. I attached logs and tcp trace to bug 3190 but had problems uploading the same for bug 3191 - possibly due to the size of the logs (although the smaller trace wont attach for me either.

I will reply to this email with the same attachments

Any help or advice much appreciated,

Thanks
Niall

-----Original Message-----
From: Niall O'Cuilinn
Sent: 13 April 2010 15:09
To: 'squid-users_at_squid-cache.org'
Subject: Squid 3.1 ICAP Issue with REQMOD 302

Hi,

I have recently moved from Squid 3.0 to Squid 3.1. I am trying to integrate it with an ICAP server.

I am having a problem where Squid 3.1  is rejecting some responses from the ICAP server which Squid 3.0 accepted.

The response in question is a REQMOD response where the ICAP server is returning a HTTP 302 response rather than modifying the original HTTP request.

Here is the ICAP request and response:

ICAP Request from Squid:

REQMOD icap://10.1.1.25:1344/reqmod ICAP/1.0\r\n
Host: 10.1.1.25:1344\r\n
Date: Mon, 12 Apr 2010 14:25:39 GMT\r\n
Encapsulated: req-hdr=0, null-body=398\r\n
Allow: 204\r\n
\r\n
GET http://c.proxy.com/www.test.com/ HTTP/1.1\r\n
Host: c.proxy.com\r\n
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-gb,en;q=0.5\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
Pragma: no-cache\r\n
Cache-Control: no-cache\r\n
\r\n

Response from ICAP Server:

ICAP/1.0 200 OK\r\n
Date: Mon, 12 Apr 2010 14:25:15 GMT\r\n
Connection: keep-alive\r\n
ISTag: "ReqModService"\r\n
Encapsulated: res-hdr=0,null-body=160\r\n
\r\n
HTTP/1.x 302 Found\r\n
content-type: text/html\r\n
location: https://localhost:8443/mib/authentication\r\n
\r\n
\r\n

Squid displays an ICAP error in the browser and states that an illegal response was received from the ICAP server.

Any ideas what might be wrong? Although the ICAP server worked correctly with Squid 3.0 I am open to the possibility that the issue is with the ICAP response and that the old Squid was simply more tolerant than v3.1.

Thanks in advance,
Niall

Niall Ó Cuilinn 
Product Development
ChangingWorlds - A Unit of Amdocs Interactive
t: +353 1 4401268 | niall.ocuilinn_at_changingworlds.com

AMDOCS > CUSTOMER EXPERIENCE SYSTEMS INNOVATION

This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp
Received on Fri Apr 08 2011 - 12:06:36 MDT

This archive was generated by hypermail 2.2.0 : Fri Apr 08 2011 - 12:00:03 MDT