Re: [squid-users] Squid performance profiling

On 06/21/2013 04:34 AM, Ahmed Talha Khan wrote:
>>> Then the question becomes why squid is slowing down?
>>
>> I think there are 2.5 primary reasons for that:
>>
>> 1) Higher concurrency level ("c" in your tables) means more
>> waiting/queuing time for each transaction: When [a part of] one
>> transaction has to wait for [a part of] another before being served,
>> transaction response time goes up. For example, the more network sockets
>> are "ready" at the same time, the higher the response time is going to
>> be for the transaction which socket happens to be the last one ready
>> during that specific I/O loop iteration.

> Are these queues maintained internally inside squid? What can be done
> to reduce this?

Resource contention is a property of the workload. While optimizations
targeted at making Squid spend fewer resources (e.g., CPU cycles) per
transaction or adding more resources (e.g., CPU cores) effectively
reduce contention as a side effect, there is nothing you can do to
reduce it directly.

>> 2a) Squid sometimes uses hard-coded limits for various internal caches
>> and tables. With higher concurrency level, Squid starts hitting those
>> limits and operating less efficiently (e.g., not keeping a connection
>> persistent because the persistent connection table is full -- I do not
>> remember whether this actually happens, so this is just an example of
>> what could happen to illustrate 2a).
>
> Can you point me to some of the key ones and their impact? So that I
> can test by changing
> these limits and seeing if it enhances/degrades the performance. Also,
> any tweaks in
> the network stack that might help with that. I am primarily interested
> in enhancing the SSL performance.

I doubt #2a is a bottleneck here so if you want to optimize Squid, I
would recommend looking at other areas. Please note that I was simply
answering your "why Squid slows down" question, not outlining an
optimization plan.

I am not sure there is consensus among Squid developers regarding the
best optimization plan, but I can offer a few starting points:

* To enhance SSL performance, try hardware SSL accelerators. OpenSSL
supports some. Or are you already using them? Also, session caching may
be important. We will be posting a patch adding SSL session cache for
SMP Squid shortly. Sorry, I cannot answer most of your earlier questions
about session caching without additional research/checking, but I can
tell you that SMP session caching currently does not work at all (but
will work, at least for some forms of sessions, with the pending patch).

* To enhance overall Squid performance for small responses, I would
recommend focusing on reducing re-parsing and memory copying. The
StringNG project underpins many of the latter optimizations.

* To enhance overall Squid performance for large responses, especially
cached ones, I would focus on eliminating linear scans of in-memory pieces.

There are many other problematic areas and OS tweaks, of course. Some
are known and some are yet to be discovered. Their importance varies
depending on your workload and environment. If these changes, tweaks,
and optimizations were easy (or were backed by a lot of demand), we
would have done them already.

>> 2b) Poor concurrency scale. Some Squid code becomes slower with more
>> concurrent transactions flying around because that code has to iterate
>> more structures while dealing with more collisions and such.
>>
>
> Well all that can be done on this front is that I have to wait for
> the changes to go in.

Waiting is one of at least three options you have:
http://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F

> Do you see any wrong conclusion that I might have made in
> interpreting these results?

I believe you shared test results and asked good questions about them
pretty much without interpreting those results. Please correct me if I
am wrong.

> Why would it not do in my test setup? I does use all CPU cores to the
> fullest in the case of HTTPS, but not in the case
> of HTTP as i pointed out earlier

I have already speculated about that. In summary, your HTTP workload is
probably not giving Squid enough CPU work to do. HTTPS adds more CPU
work, of course. Needless to say, it is rather risky for me to criticize
a test based on a high-level description and a results table. I hope you
will forgive me if my guess ends up being wrong!

Thank you,

Alex.
Received on Fri Jun 21 2013 - 15:01:44 MDT