Re: wish for an advice

At 17:26 06/08/2007, Henrik Nordstrom wrote:
>On fre, 2007-08-03 at 15:37 +0200, Jiri Kuthan wrote:
>
>> I'm wondering if you could do me a massive favor and exchange some experience you
>> might have gathered with building squid regarding TCP scalability.
>
>We can try..

Hi Henrik,

thank you very much indeed.

>> We've been writing a SIP proxy server, SER (www.iptel.org/ser), which typically operates
>> in UDP mode. Now in the IETF, TCP is gaining more momentum. SER supports it but compared
>> to UDP it scales rather poorly -- at the moment we can't get more than 50k connections
>> out of it.
>
>The primary thing to keep in mind is to make use of a decent event
>notificatoin mechanism, i.e. epoll/kqueue/devpoll. Without it
>performance will quickly suffer when the amount of connections
>increases.

We do epoll, that shall be fine then.

>To make life easy one may use libevent, which provides all the needed
>framework.
>
>When using a good event notification the performance is again more
>related to the amount of messages one has to process and not related to
>the amount of concurrent connections.
>
>> Could you share with me, how many TCP connections can squid handle, if you have any
>> performance measurment results, tuning tips, etc?
>
>Unlike SIP HTTP is still dominated by relatively shortlived connections,
>so we don't get that many concurrent connections. A very buzy server is
>more in the range of 5-10K connections.

I think that's what makes the big difference. We do have permanent
connections (i.e. connections stay as long alive as long a phone
is registered with a server). Otherwise clients could not receive
asynchronous messages from behind NATs. One alternative we are considering
is to make ourselves more similar to web apps in that we generate
a redirect reply over a short-lived connection and hand-off
the connection to client-server. (In our use case, which is load-
balancing, we dont have to stay in the path except for the
initial message.)

>But have done lab tests with several hundreds of thousand idle
>connections, to verify that the event notification works. It does, but
>Squid's connection oriented buffer management makes it consume quite a
>bit of memory...

ok, that's a word :-)
If we would like to get some lab intelligence, is all we have to do
install squid and buy a lot of memory, or is there some other black
magic? Have you tried more than 4 GB?

Thanks a lot!

-jiri

>Regards
>Henrik
>

--
Jiri Kuthan            http://iptel.org/~jiri/