RE: RES: [squid-users] Squid box dropping connections

From: Jenny Lee <bodycare_5_at_live.com>
Date: Thu, 17 Nov 2011 22:31:50 +0000

----------------------------------------
> From: listas.nata_at_cnett.com.br
> To: bodycare_5_at_live.com; squid-users_at_squid-cache.org
> Date: Thu, 17 Nov 2011 15:55:20 -0300
> Subject: RES: [squid-users] Squid box dropping connections
>
> Hello Jenny,
>
> Thanks for your answer. Sorry I haven't wrote but my hashsize is already in
> the same value as conntrack_max. I have some out of memory in dmesg:
>
> Nov 17 15:43:13 02 kernel: Out of socket memory
 
 
Well, there you go. Here is your problem. You will need to decrease your hashsize. I suggest you experiment with conntract max and hashsize nad buckets and watch for errors like these.
 
There are couple of good docs out there explaining kernel memory use with conntrack.
 
 

> And in cache.log I was not able to find any CommBind. I am reading about
> this port ranges (ephemeral). I think my squid is using too many sockets:
>
> sockets: used 16662
> TCP: inuse 28433 orphan 12185 tw 2191 alloc 28787 mem 18786
> UDP: inuse 8 mem 0
> RAW: inuse 1
> FRAG: inuse 0 memory 0
>
> And it has about 16k files open right now. I will try to find a way to make
> more ports available. Thanks!
 
You can check available port range with:
cat /proc/sys/net/ipv4/ip_local_port_range

And increase it with:
echo "1024 65535" > /proc/sys/net/ipv4/ip_local_port_range
 
 
This is for RHEL6, I don't recall if it is the same for RHEL5.
 
Here is a small perl script to log these for post-mortem review. Put it to cron, run every minute as root. Then you can review later.
 
Your orphans don't look good to me. However, you have nolocalbind and you are using tproxy.
 
I am neither linux, nor perl, nor tproxy, nor tcp expert. Just someone trying to solve her problems. So approach all these with caution, I take no responsibility.
 
Good luck!
 
Jenny
 
 
 
#!/usr/bin/perl

$ct = `cat /proc/sys/net/netfilter/nf_conntrack_count`;
chomp $ct;
@ss = `ss -s`;

foreach (@ss) {
        if (/TCP:\s+(\d+)\s+\(estab\s+(\d+),.+orphaned\s+(\d+),.+timewait\s+(\d+).+ports\s+(\d+)/) {
                $tcp = $1; $est = $2; $orp = $3; $tw = $4; $ports = $5;
        }
}

$file = "/var/log/tcp.log";
$date=localtime();
 

open(OUT, ">>$file");
print OUT "$date: CT:$ct TCP:$tcp EST:$est ORP:$orp TW:$tw PORTS:$ports\n";
close OUT;
Received on Thu Nov 17 2011 - 22:31:57 MST

This archive was generated by hypermail 2.2.0 : Fri Nov 18 2011 - 12:00:03 MST