Re: [squid-users] [Help] Reverse Proxy: suspend bulk requests for invalid urls

From: Le Trung Kien <kienlt_at_vietnamnet.vn>
Date: Fri, 13 May 2011 10:55:33 +0700

Hi,
On the original servers, I'm using IIS6.0 and the 404b.html is the
page returned when client requests non-existing pages.
I attempt to add a header like this on that page:

<HEAD><TITLE>The page cannot be found</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=Windows-1252">
<META HTTP-EQUIV="Cache-Control" Content="Private">
<META HTTP-EQUIV="Expires" Content="300">
</HEAD>

This header is the same on all pages generated by our web applications
and could be cached.
However, I test and see that our squid still doesn't cache that 404b.html page

squidclient -m HEAD http://invalid_URL

HTTP/1.0 404 Not Found
Content-Length: 1731
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Fri, 13 May 2011 03:41:10 GMT
X-Cache: MISS

squidclient -m HEAD http://existing_URL

HTTP/1.0 200 OK
Date: Fri, 13 May 2011 03:34:22 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
X-Powered-By: UrlRewriter.NET 2.0.0
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 121485
Age: 424
X-Cache: HIT

I realize that the header squidclient receives when requesting an
invalid URL is less and our squid still MISS.

Kien Le.

On Thu, May 12, 2011 at 1:57 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>
> On 12/05/11 17:10, Le Trung Kien wrote:
>>
>> I realized that the server reply both 403 and 404.
>> About 404, but I don't know how to cache 404 File Not Found reply from
>> original servers, should I add a default error page on web application
>> for invalid URLs ?
>> I tested and saw that cache misses on those URLs because we don't have
>> a default error page now :
>>
>> 404 TCP_MISS:FIRST_UP_PARENT
>>
>> Kien Le
>>
>
> Default page or not, Squid does not mind. All it needs is an Expires: header at least a few seconds in the future.
>
> Personally, I use a script which detects and sets teh header if its an unknown URL (5 seconds caching, the client could be about to create it) or one of the permanently dead ones (1 year caching with sometimes info saying where the new one is).
>
> How you do it depends on the server capabilities and website needs.
>
> Amos
> --
> Please be using
>  Current Stable Squid 2.7.STABLE9 or 3.1.12
>  Beta testers wanted for 3.2.0.7 and 3.1.12.1
Received on Fri May 13 2011 - 03:55:40 MDT

This archive was generated by hypermail 2.2.0 : Fri May 13 2011 - 12:00:03 MDT