Re: range request cache

On 22/02/2012 1:09 p.m., Zhu, Shan wrote:
> I have an urgent need for caching requested ranges so I want to do a quick "hack" on this topic before the new feature becomes available.
>
> What I want to achieve is for Squid to cache a range request without pre-fetching and caching the whole object, so that if the cached range is requested again it can be served from the cache.
>
> What I want to do is to change the URL with range request into a unique file name, and once the response is received from the back-end server, the response can be cached as a single object.
> The workflow should be like this,
>
> (1) Change URL with range request into something like: "[original URL]_[range start]_[range end]", internally to Squid only.
>
> (2) Squid checks cache to see if it is cached. If yes, Squid responds to the client with the cached object. If no, go to Step 3.
>
> (3) When it is a cache miss in Step 2, Squid forwards the original URL to the back-end server, with range request, as normal. (Not to pre-fetch of = the whole object file.)
>
> (4) Squid receives the response from back-end server, for the requested range only, recognizes it as the response corresponding to the changed URL = "[original URL]_[range start]_[range end]".
>
> (5) Squid caches the range according to the changed URL "[original URL]_[range start]_[range end]", like a single object file.
>
> (6) Squid responds to the client for the URL with range request, as normal.
>
> Here I may have simplified the problem and omitted the time-stamp issue, etc.
>
> Is this doable? How difficult would it be? Can I get any suggestion on how to proceed? I am starting from scratch on the source code change.

Doable yes. Have a look at how object variants have a special cached
object acting as a pointer to the real stored variant. This can be used
to store the range reply as a variant.

This has a few problems with requests invalidating cached content. But
gets you past the first hurdle of having it cache without breaking HTTP.

As a followup once that is going you may want to look at merging two
responses. And possibly generating background requests to fill in the
object gaps for future clients.

Amos
Received on Thu Feb 23 2012 - 15:15:27 MST