On 07/17/2014 02:49 AM, Martin Sperl wrote:
> This is why I have been mentioning all the pools that show similar (wavy)
> memory increase pattern. There must be one of those that is the root of
> all the others.
Unfortunately, I do not share your certainty. As I see it, at least two
theories more-or-less fit your data:
A) MemObjects are leaking.
B) Your memory cache is growing, explaining some or all of the pools
growth and muddying the waters along the way. Something unrelated to
MemObjects is leaking. That something may or may not be pooled.
My bet is on (B).
If you do not mind reminding me, does accounted-for memory pools growth
correspond to the actual/total Squid memory growth? Or is the "pooled"
vs "total/real" gap widening with time?
> the strange thing is that if you look at the
> distribution of vm_objects, then you see that they have expired long ago
> (16267 days ago to be exact, so with EX:-1 - 42511 exactly).
> If these have been expired, then why would they get loaded into memory?
because you have free cache space and Squid can still serve that cached
data (after revalidation)? In general, there is no benefit from purging
something from a non-full cache, especially if that something is reusable.
I have not checked the code and responses producing those "16267 days"
stats so I treat that number with a grain of salt. :-)
> Well: SUM(inmem_hi) is memory for payload (possibly without headers) against
> which we compare the cache_mem against.
>
> But if the squid process consumes 9GB, then there must be more than a factor
> of 2 of overhead so that we get to those 9GB.
Yes if (A) above is correct. If (B) above is correct, that "overhead" is
a leak of something other than memory cache entries.
> Report-file(with Date+Time) psmemory MemObjects kb/MemObject
> report-20140708-234225.txt 1795932 123979 6.03
...
> report-20140717-000001.txt 10662148 845404 11.37
> Which shows that the average size/memObject is increasing constantly!
Yes, and that is one of the reasons I bet on (B). (B) is a lot less
surprising than a constantly growing MemObject overhead that (A) has to
explain :-). Running with a small cache size (i.e., with a full cache)
would confirm that.
> Another point here more on the reporting/debugging side:
> Total space in arena: -2034044 KB
Yes, this is a known problem mostly outside of Squid control:
http://www.squid-cache.org/mail-archive/squid-users/201402/0296.html
Squid should probably stop using that API [when it can detect that it is
broken].
And yes, I know that my response does not really help you. I just wanted
to make sure you consider both (A) and (B) theories in your investigation.
Alex.
P.S. Another suggestion that you probably cannot use in your environment
is to run trunk or a trunk-based branch. Perhaps the leak you are after
has been fixed. And if not, somebody may be more motivated to help you
find it in trunk.
Received on Thu Jul 17 2014 - 21:03:48 MDT
This archive was generated by hypermail 2.2.0 : Fri Jul 18 2014 - 12:00:04 MDT