Re: [RFC] Have-Digest and duplicate transfer suppression

On 08/10/2011 05:08 PM, Robert Collins wrote:
> On Thu, Aug 11, 2011 at 10:59 AM, Alex Rousskov
> <rousskov_at_measurement-factory.com> wrote:
>> On 08/10/2011 04:18 PM, Robert Collins wrote:
>>> How is case B different to If-None-Match ?
>>
>> The origin server may not supply enough information for If-None-Match
>> request to be possible OR it may lie when responding to If-None-Match
>> requests.
>
> The parent squid could handle the case when the origin lies though,
> couldn't it ?

Yes, sometimes.

> So:
> client -> child -> parent -> origin
> if client asks for url X, child has an old copy, child could add
> If-None-Match, parent could detect that origin sends the same bytes
> (presumably by spooling the entire response) and then satisfy the
> If-None-Match, and child can give an unconditional reply to client,
> which hadn't had the original bytes.

Without duplicate transfer suppression, the parent Squid cannot detect
that the bytes are the same if the origin server lies (sends a different
ETag for the same response body). Parent Squid cannot compute ETags.
Computing Content-MD5 may be possible, I guess. Computing Have-Digest
values or similar is always possible, by definition.

> That doesn't help with the not-enough-information case, which is I
> presume the lack of a strong validator.

Exactly. The parent Squid cannot detect that the child Squid has the
same bytes unless there is a matching strong validator. Have-Digest or
similar is a way to add such a validator when the origin does not send
one OR when the origin lies and the parent Squid cannot detect a lie.

> So, perhaps we could consider this 'how can intermediaries add strong
> validators' - if we do that, and then (in squid - no http standards
> violation) - honour If-None-Match on those additional validators, it
> seems like we'll get the functionality you want, it a slightly more
> generic (and thus reusable) way ?

Good idea!

I agree that Have-Digest or similar custom "digest" entity-tag can be
used with If-None-Match. You gain the advantages of the existing
If-None-Match mechanism. You also gain problems associated with
overloading an end-to-end mechanism with hop-by-hop optimizations: I am
sure some servers will get confused when they suddenly receive our
Have-Digest entity tags in If-None-Match requests (because the request
did not go through the parent capable of stripping those custom entity
tags -- the parent got replaced, overloaded, bypassed, etc).

Both If-None-Match and Transfer-Encoding (or similar) schemes are HTTP
compliant. Both extend an existing HTTP mechanism.

I am not sure whether the pros outweigh the cons in this case, but I
think using If-None-Match should be seriously considered. Regardless of
this decision, the most complex and performance-expensive elements of
the duplicate suppression proposal remain the same though.

Thank you,

Alex.
Received on Thu Aug 11 2011 - 04:16:29 MDT