Re: [squid-users] ecap adapter munging cached body

On 22/01/11 19:22, Jonathan Wolfe wrote:
> First, request and reply headers for a cached version, as gzip isn't included in the Accept-Encoding header.
>
> Request:
>
> GET /styles/media.css HTTP/1.1
> User-Agent: httperf/0.9.0
> Host: www.example.com
> Accept-Encoding: deflate
>
> Reply:
>
> HTTP/1.0 200 OK
> Cache-Control: max-age=86400
> Expires: Sun, 23 Jan 2011 06:06:04 GMT
> Vary: Accept-Encoding
> Last-Modified: Thu, 30 Jul 2009 11:30:14 GMT
> Mime-Version: 1.0
> Date: Sat, 22 Jan 2011 06:06:04 GMT
> Server: AOLserver/4.5.1
> Content-Type: text/css; charset=utf-8
> Content-Length: 2654
> Age: 1
> X-Cache: HIT from www.example.com
> X-Cache-Lookup: HIT from www.example.com:80
> Via: 1.0 www.example.com (squid/3.1.10)
> Connection: keep-alive
>
>
> Now, request and reply headers when accepting gzip.
>
> Request:
>
> GET /styles/media.css HTTP/1.1
> User-Agent: httperf/0.9.0
> Host: www.example.com
> Accept-Encoding: gzip,deflate
>
> Reply:
>
> HTTP/1.0 200 OK
> Cache-Control: max-age=86400
> Expires: Sun, 23 Jan 2011 06:06:08 GMT
> Vary: Accept-Encoding
> Last-Modified: Thu, 30 Jul 2009 11:30:14 GMT
> Mime-Version: 1.0
> Date: Sat, 22 Jan 2011 06:06:08 GMT
> Server: AOLserver/4.5.1
> Content-Type: text/css; charset=utf-8
> Content-Encoding: gzip
> X-Cache: MISS from www.example.com
> X-Cache-Lookup: MISS from www.example.com:80
> Via: 1.0 www.example.com (squid/3.1.10)
> Connection: keep-alive
>
> I see that the gzipped version doesn't reply with a Content-Length header.

Not nice, but fine. The connection should get closed after the object
despite the keep-alive. If not that is another bug.

>
> Tried with Firefox for a more standard request, exactly the same response headers.
>
> Regards,
> Jonathan Wolfe

Ah, sorry. I'm not sure what has happened to my mind these last few
days. I think that is a perfectly normal and working set of transactions
with nothing to do with the ecap module.

Vary in Squid is currently treated as an exact-match text key. So when
asked for a "gzip,deflate" variant Squid does not have enough smarts to
serve the "deflate" variant. So it MISSes and gets a fresh one, which
may or may not be gzipped, but is served gzipped to the client anyway.

When passing the second request through squid twice in a row does the
reply change from a MISS to a HIT? or stay a MISS?

Squid caches only one of N variants so the expected behviour is that
each new variant is a MISS but becomes a HIT on repeated duplicate
requests until a new variant pushes it out of cache.

I think one of two things are needed to make this work nicely:
   (1) request modification to normalize and add "gzip" to the
Accept-Encoding header prior to the cache variant lookup
   (2) de-zipping for clients who can't accept gzip
   (3) zipping for clients who can but are sent a un-zipped version

I gather that (3) is occurring. But some variant has entered your cache
due to (1) not being done.

NOTE: patches very welcome to make Squid treat "deflate" as a subset of
"gzip,deflate" when finding variants.
   It's a bit trickier than it sounds though. One will have to alter the
variant key syntax to store the variant with the Content-Encoding type
instead of the full Accept-Encoding header. Then check for all options
in the client Accept-Encoding header when looking up the hash.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.10
   Beta testers wanted for 3.2.0.4