[squid-users] ESI content missing

We're been using ESI to include dynamic content into otherwise static
pages, but one of our web content authors tried using it to include
some static HTML and ran into a bit of a problem when refreshing a
page in his browser. I think I've tracked it down and found a way to
fix it, but I'd like some feedback.

For example, if we have two pages say 'master.html' includes
'included.html' with a directive such as:
<esi:include src="http://$(HTTP_HOST)/included.html" onerror="continue" />

- If 'master.html' is older than 'included.html' then everything works
as we expect, we get the included content inserted into master.html.

- If 'master.html' is newer than 'included.html' and the included page
isn't freshly cached then everything works.

- If 'master.html' is newer than 'included.html' and the included page
is fresh in the squid cache then we get nothing in the page where the
included content should exist.

- Clearing the browser cache before refreshing or using shift-refresh
in the browser sends an unconditional request and the included content
is always present.

What seems to be happening is that the forwarded request includes most
of the headers from the original request: in particular it includes
the 'If-Modified-Since' header. When squid tries to fetch the included
page it generates an internal request for the page but because the
master page is newer than the included page it gets a 304 response and
no data. There is also something else going on that I haven't fully
understood: when squid forwards the request to the backend server we
get the content even with a 304 response, but when it satisfies the
request internally it doesn't work.

I tried modifying ESIInclude.cc to strip the IMS header and we now
reliably get the content we expect:

void
ESIInclude::prepareRequestHeaders(HttpHeader &tempheaders, ESIVarState *vars)
{
   tempheaders.update (&vars->header(), NULL);
   tempheaders.removeHopByHopEntries();
   tempheaders.delById(HDR_IF_MODIFIED_SINCE); // I added this line
}

(alternatively adding HDR_IF_MODIFIED_SINCE to HopByHopHeadersArr has
the same effect). I guess this isn't the place for a proper fix as it
means more data pulled from the backend server, but I couldn't work
out where the actual problem might be.

My question is whether this is indeed a bug or whether I've maybe just
got something wrong in the squid config?
We're using Squid 3.0 stable 14 but I checked with stable 24 and get
the same results.
Received on Mon Mar 08 2010 - 15:55:38 MST