Re: Download hanging

On Mon, 7 Aug 2006, Henrik Nordstrom wrote:

> Steven, do you think this could be the cause of Bug #1304?
>
> Never got to the bottom with that bug and it seemed to disappear, but it
> feels like this could be the root cause...
>

This definitely looks like the same bug to me.

I did have one other thought. If you set max-object-size on a cache_dir,
it effectively allows an object to use up to max-object-size woth of RAM
before it will start to swap out. I can understand this being needed if
there is no content-length header, but should the following code also do a
check against the content-length header if the header has given us a size?

store.c near line 200:

    if ((e->store_status != STORE_OK) && (swapout_size <
store_maxobjsize)) {
        /*
         * NOTE: the store_maxobjsize here is the max of optional
         * max-size values from 'cache_dir' lines. It is not the
         * same as 'maximum_object_size'. By default, store_maxobjsize
         * will be set to -1. However, I am worried that this
         * deferance may consume a lot of memory in some cases.
         * It would be good to make this decision based on reply
         * content-length, rather than wait to accumulate huge
         * amounts of object data in memory.
         */
        debug(20, 5) ("storeSwapOut: Deferring starting swapping out\n");
        return;
    }

i.e. should it look like:
if ((e->store_status != STORE_OK) && (swapout_size < store_maxobjsize) &&
(e->mem_obj->reply->content_length < 0)) {

If e->mem_obj->reply->content_length is > 0, I don;t believe that we need
to wait for the object to grow too large before we can make a decision on
where to store it.

Steven
Received on Mon Aug 07 2006 - 17:51:46 MDT