Re: developing a module for POST caching on squid

Matteo Campanella wrote:
> Hello, I have found an old post about how to modify squid to cache POST
> responses (copied below) and I would like to know a couple things.

The post you found is a mix of someones earlier request identical to
yours. And Henriks expert reply covers exactly what need to be done.
Nothing has changed since then.

> I was thinking to implement POST caching by letting the POSTed body to
> contribute to the construction of the MD5 key in
> storeKeyPublicByRequestMethod function, simply by adding one more call to
> SQUID_MD5Update using the body as parameter - is it the right place where to
> calculate the store key?

To thing you need to think about a bit more...

The use-cases that need to be considered when moving request bodies include:
  * MB of data in POSTs is quite common
  * 'infinite' length are even more common.
  * requests lasting several hours or days are also known.

Handling all of the above in a pre-calculated MD5 is not nice. Thus the
buffering requirement.

The question of really: does the POST need to actually reach the web server?
  - YES: then it cannot be cached.
  - NO: then why is a POST being used at all?

RFC 2616 permits the use of a GET request with body data (must not
affect the reply) plus the same dynamic URL and/or headers sent in the POST.

>
> In order to do what reported above, I need the body bytes to be available
> inside the storeKeyPublicByRequestMethod function; how can I do that? I've
> read the httpSendRequestEntry() function but honestly it did not make things
> clearer :)
>
>
> Looking foward for your response, I wish you merry xmas and happy new year!
>
> ciao
> Matteo
>

[attributions corrected below]
> -------------------------------->
> Subject: Re: Where to deal with POST? Actions From: Henrik
> Nordstrom
> (hn..._at_squid-cache.org) Date: Nov 6, 2003 12:41:48 am List: org.squid-
> cache.squid-dev
>

On Thu, 6 Nov 2003, Zhou Hong wrote:
>
> I am now working on to add function module of caching web services to squid
> 2.5.
> Since the soap requests of web service use POST and squid marks the POST
> request nocache, I have to find where to modify the manipulation of POST
>

Henrik wrote:
> Squid does not cache POST replies as it is uncertain under which conditions
> a POST reply may be cached. Most uses of POST depends heavily on the
> contents of the POST request entity.
>
> If your goal is to make POST cacheable then there is a couple of things you
> need to do, assuming this caching is dependent on the POST request data:
>
> * Modify how request entities are forwarded, temporarily buffering the
> entity so you can process the request entity before the request is
> forwarded.
>
> * Extend the public storeKey hashed data with information as required to
> give each POST request/reply which should be cached uniquely a unique store
> key.
>
> * Remove the flag which makes POST requests uncacheable (look for
> METHOD_POST).
>
> I also encounter the following question when looking into codes, and need
> your help,
>

Zhou Hong wrote:
> 1) Where to get POST request content body?
>

Henrik wrote:
> See httpSendRequestEntry() for an example.
>

Zhou Hong wrote:
> 2) Besides url, which other things are selected to form the hash key.
>

Henrik wrote:
> See storeKeyPublicByRequest()
>

Zhou Hong wrote:
> 3) What is the difference between storeSetPrivateKey and storeSetPublicKey,
> I mean What scenarios do we use private key and public key respective in.
>

Henrik wrote:
> Every object in Squid must have a unqiye key identifying the object to
> Squid.
> Objects which should not be shared with other clients use private keys,
> shareable objects use public keys.
>
> All objects start out private and when Squid determines the response may be
> cached the key is changed to the public key.
>

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21
   Current Beta Squid 3.1.0.15