Re: squid-3 development: stmem, endOffset(), range requests

On Tue, 2004-08-24 at 10:37 +0800, Adrian Chadd wrote:
> On Mon, Aug 23, 2004, Henrik Nordstrom wrote:
>
> > >.. now, at this point, the whole point behind content_length + hdr_sz
> > >is the size of the _reply_. endOffset() is now _not_ the size of the
> > >reply anymore. It is the highest position of the object we know about.
> >
> > What is the difference?
>
> > So you say the difference is that before endOffset did indicate the amount
> > of data known for the object, but now there may be holes?
>
> endOffset() now returns the highest byte of the object with ranges.
>
> in 2.5, a request like this:
>
> ranges: 1-1000,2000-3000
>
> the reply from the server would be 2kb (1-1000 and 2000-3000).
> Whatever it is in 2.5 would return 2000 when the whole request was read.
>
> In squid-3, it'd return 3000.
>
> This could cause quite a few problems in the store code.

I did a sweep for uses of that function... I thought I'd caught them all
and sanitised etc. Range requests should not be going to the store
anyway at this point...

> > So how is the code now supposed to know if all needed data is available in
> > order to fix the lines above?
>
> Thats what I'm going to need to think about. I'll try to figure out
> what the current uses of endOffset() in the store and http.cc code
> is and see what I'll need to do.
>
> The 'hack' would be to write a routine to return how many bytes are
> currently in the squid-3 stmem/MemObject. This may be enough for
> the store code which uses it to swap objects out but I want to make
> the http usage a bit nicer.

Eww. range requests really shouldn't be hitting disk. range combining is
not cooked yet.

> I suggest fixing this up and trying to get a stable looking devel release out
> before we run around and try to replace anything like the callback methods.

Of course - notice I mentioned squid 3.1, not 3.0 for that :}.

Rob