Re: [squid-users] Squid and OS tuning

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 05 Apr 2007 08:26:48 +0700
zen <zen@tk-pttuntex.com> wrote:

> Tek Bahadur Limbu wrote:
>
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >On Wed, 04 Apr 2007 13:27:05 +0700
> >zen <zen@tk-pttuntex.com> wrote:
> >
> >
> >
> >>Dear Users.
> >>i'm planning to build a proxy server and it will be serving 10k of
> >>users/clients,
> >>it's a AMD X2 3800 AM2 and currently installed RAM are 4Gigs, and 2
> >>SATA HDD 300GB
> >>and running FreeBSD 6.2STABLE am64 build.
> >>i would like to ask for guidance how to build and optimized this proxy
> >>server ( OS base or Squid base )
> >>kindly one of you share the tips/tricks regarding this problems.
> >>
> >>
> >>
> >
> >Hi Zen,
> >
> >Do you want to run Squid in transparent mode? If so, which firewall (ipfw/pf/ipf) will you be using?
> >
> >Try to do download the source package and compile it manually rather than using FreeBSD's ports.
> >
> >You can using the following compilation options:
> >
> >- --bindir=/usr/local/sbin \
> >- --sysconfdir=/usr/local/etc/squid \
> >- --datadir=/usr/local/etc/squid \
> >- --libexecdir=/usr/local/libexec/squid \
> >- --localstatedir=/usr/local/squid \
> >- --enable-removal-policies=heap,lru \
> >- --enable-async-io \
> >- --enable-storeio=diskd,aufs,coss,ufs,null \
> >- --enable-time-hack \
> >- --enable-snmp \
> >- --enable-kqueue \
> >- --with-large-files \
> >- --prefix=/usr/local \
> >- --disable-ident-lookups
> >
> >For full compilation options, see:
> >
> >#~ ./configure --help | less
> >
> >Also, try to use the Squid-2.6 version. It's quite stable and extremely CPU friendly.
> >
> >In your Kernel, you can use the following options:
> >
> >options SMP
> >
> >options SHMSEG=128
> >options SHMMNI=256
> >options SHMMAX=33554432 # max shared memory segment size (bytes)
> >options SHMALL=16384 # max amount of shared memory (pages)
> >options MSGMNB=16384 # max # of bytes in a queue
> >options MSGMNI=48 # number of message queue identifiers
> >options MSGSEG=768 # number of message segments
> >options MSGSSZ=64 # size of a message segment
> >options MSGTQL=4096 # max messages in system
> >
> >
> >options IPFIREWALL #Only for IPFW firewall
> >options IPFIREWALL_VERBOSE #Only for IPFW enable logging to syslogd(8)
> >options IPFIREWALL_FORWARD #Only for IPFW enable transparent proxy support
> >
> >
> >For some optimization and also since your users/clients base is about 10000, you might want to try the following values.
> >
> >/boot/loader.conf:
> >
> >kern.maxusers=0
> >kern.ipc.nmbclusters=65536
> >kern.maxfiles=16384
> >kern.maxproc=8192
> >kern.ipc.somaxconn: 8192
> >kern.ipc.maxsockets=16384
> >
> >
> >/etc/sysctl.conf:
> >
> ># TCP/IP optimization
> >net.inet.tcp.recvspace=65535
> >net.inet.tcp.sendspace=65535
> >kern.maxfilesperproc=8192
> >
> >
> >For squid.conf:
> >cache_effective_user nobody
> >cache_effective_group nobody
> >
> >cache_mem 128 MB
> >cache_swap_low 90
> >cache_swap_high 98
> >
> >http_port 3128 transparent # Only if you want Squid to run transparently
> >
> >maximum_object_size 131072 KB
> >maximum_object_size_in_memory 16 KB
> >
> >tcp_recv_bufsize 65535 bytes
> >ipcache_size 4096
> >fqdncache_size 4096
> >
> >emulate_httpd_log on
> >
> >client_persistent_connections off
> >half_closed_clients off
> >
> >cache_dir diskd /cache1 100000 64 512 Q1=72 Q2=64
> >cache_dir diskd /cache2 100000 64 512 Q1=72 Q2=64
> >
> >hierarchy_stoplist cgi-bin ?
> >
> >acl QUERY urlpath_regex cgi-bin \?
> >no_cache deny QUERY
> >
> >negative_dns_ttl 2 minutes
> >
> >#Default pattern is good enough
> >refresh_pattern ^ftp: 1440 20% 10080
> >refresh_pattern ^gopher: 1440 0% 1440
> >refresh_pattern . 0 20% 4320
> >
> >
> >Also regarding DISKD storage system, see the great FAQ below:
> >
> >http://wiki.squid-cache.org/SquidFaq/CyclicObjectStorageSystem
> >
> >Even though, it is quite outdated, it offers enough insight about DiskD and FreeBSD in general.
> >
> >
> >Note: This values should normally work but what works for me does not necessarily mean that it will work for all.
> >
> >Also I welcome any suggestions and feedback for the above mentioned values of mine. I may have miss something myself.
> >
> >Thanking you...
> >
> >
> >
> >>TIA
> >>
> >>Zen
> >>
> >>
> >>
> >
> >
> >- --
> >
> >
> >With best regards and good wishes,
> >
> >Yours sincerely,
> >
> >Tek Bahadur Limbu
> >
> >(TAG/TDG Group)
> >Jwl Systems Department
> >
> >Worldlink Communications Pvt. Ltd.
> >
> >Jawalakhel, Nepal
> >
> >http://www.wlink.com.np
> >-----BEGIN PGP SIGNATURE-----
> >Version: GnuPG v1.4.2.2 (FreeBSD)
> >
> >iD8DBQFGE5cLVrOl+eVhOvYRAtwKAJ4ptACbnnkxKlhoYj/h/V6O92sYiQCgmd8Z
> >TlzhH69L16FH+0iyz0Nh9NI=
> >=G/3Y
> >-----END PGP SIGNATURE-----
> >
> >
> >
> thanks for sharing the tips,
> but onething i wanna ask you,
> is option SMP on the kernel side necessary?
> because Hendrik said squid only know/work with single proc.

Well I also don't think the SMP Option is necessary with Squid being a single proc as Hendrik mentioned.

But it might help other applications and processes. Furthermore, if Squid's next version improves support for multiple CPUs, then you don't need to re-compile your kernel!

By the way, you may need to include the following compilation option in Squid if you will be using ipf:

- --enable-ipf-transparent

>
>
> TIA
>
> Zen
>

- --

With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (FreeBSD)

iD8DBQFGFM0HVrOl+eVhOvYRApQfAJ4nqyhjeuD9pYtd59JuusNSzCwNPACghjey
Db86ovT62APL62gA1Arzkow=
=iFL5
-----END PGP SIGNATURE-----
Received on Thu Apr 05 2007 - 04:29:21 MDT