Re: How to buffer a POST request

Well, I've got a proof of concept which works well but its -very-
ugly. This is one of those things may have been slightly easier to do
in Squid-3 with Alex's BodyPipe changes. I haven't stared at the
BodyPipe code to know whether its doing all the right kinds of
buffering for this application.

The problem is that Squid-2's request body data pipeline doesn't do
any of its own buffering - it doesn't do anything at all until a
consumer says "give me some more request body data please" at which
point its copied out of conn->in.buf (the client-side incoming socket
buffer), consumed, and passed onto the caller.

I thought about a "clean" implementation which would involve the
request body pipeline code consuming socket buffer data until a
certain threshold is reached, then feeding that back up to the request
body consumer but I decided that was too difficult for this particular
contract.

Instead, the "hack" here is to just keep reading data into the
client-side socket buffer - its already doing double duty as a request
body buffer anyway - until an ACL match fires to begin forwarding. Its
certainly not clean but it seems to work in local testing. I haven't
yet tested connection aborts and such to make sure that connections
are properly cleaned up.

I'll look at posting a patch to squid-dev in a day or two once my
client has had a look at it.

Thanks,

Adrian

2008/8/8 Adrian Chadd <adrian_at_squid-cache.org>:
> Well I'm still going through the process of planning out what changes
> need to happen.
>
> I know what changes need to happen long-term but this project doesn't
> have that sort of scope..
>
>
>
> Adrian
>
> 2008/8/8 Mark Nottingham <mnot_at_yahoo-inc.com>:
>> You said you were doing it :)
>>
>>
>> On 08/08/2008, at 4:40 PM, Adrian Chadd wrote:
>>
>>> Way to dob me in!
>>>
>>>
>>> Adrian
>>>
>>> 2008/8/8 Mark Nottingham <mnot_at_yahoo-inc.com>:
>>>>
>>>> I took at stab at:
>>>> http://wiki.squid-cache.org/Features/RequestBuffering
>>>>
>>>>
>>>> On 22/07/2008, at 4:40 PM, Henrik Nordstrom wrote:
>>>>
>>>>> It's not a bug. A feature request in the wiki is more appropriate.
>>>>>
>>>>> wiki.squid-cache.org/Features/
>>>>>
>>>>> Regards
>>>>> Henrik
>>>>>
>>>>> On mån, 2008-07-21 at 17:50 -0700, Mark Nottingham wrote:
>>>>>>
>>>>>> I couldn't find an open bug for this, so I opened
>>>>>> http://www.squid-cache.org/bugs/show_bug.cgi?id=2420
>>>>>>
>>>>>>
>>>>>> On 11/06/2008, at 3:29 AM, Henrik Nordstrom wrote:
>>>>>>
>>>>>>> On ons, 2008-06-11 at 12:51 +0300, Mikko Kettunen wrote:
>>>>>>>
>>>>>>>> Yes, I read something about this on squid-users list, there seems
>>>>>>>> to be
>>>>>>>> 8kB buffer for this if I understood right.
>>>>>>>
>>>>>>> The buffer is bigger than that. But not unlimited.
>>>>>>>
>>>>>>> The big change needed is that there currently isn't anything delaying
>>>>>>> forwarding of the request headers until sufficient amount of the
>>>>>>> request
>>>>>>> body has been buffered.
>>>>>>>
>>>>>>> Regards
>>>>>>> Henrik
>>>>>>
>>>>>> --
>>>>>> Mark Nottingham mnot_at_yahoo-inc.com
>>>>>>
>>>>
>>>> --
>>>> Mark Nottingham mnot_at_yahoo-inc.com
>>>>
>>>>
>>>>
>>
>> --
>> Mark Nottingham mnot_at_yahoo-inc.com
>>
>>
>>
>
Received on Tue Sep 09 2008 - 07:34:35 MDT