Hi All,
I was running multiple instances of squid 3.0 Stable 25 on the same server successfully. However i intend to run squid 2.7 & 3.1.3 on the same server now reason being 2.7s enhance support of dynamic content caching. (Earlier the main intention to use multiple instances was to give fault tolerance to cache failure )
My question is that if this possible? If there be any special changes i be requiring?
copy of squid instance 2 which i will be using for caching please peruse it in the context of youtube/facebook caching specifically. If you notice any other drawback/discrepancy please do guide about it as well i would be really really thankful.
( i have also altered the client_side.c as per the guide available on squid cache web site)
---------------------------------------------------------------------------------------------------------------------------------
visible_hostname squidlhr1_at_virtual.local
unique_hostname squidlhr1cache
pid_filename /var/run/inst2squid.pid
http_port 1975
icp_port 0
snmp_port 7172
access_log /var/logs/inst2access.log squid
cache_log /var/logs/inst2cache.log
cache_store_log /var/logs/inst2store.log
cache_effective_user proxy
cache_mgr squidadmins_at_virtual.local
# If peering with ISA then following options will be required. Otherwise not
#cache_peer 10.1.82.205 parent 8080 0 default no-digest no-query no-delay
#never_direct allow all
# Hard disk size 71gb SAS 15k dedicated for caching. Operating system is on RAID1.
cache_dir aufs /cachedisk1/var/spool/squid 50000 128 256
coredump_dir /cachedisk1/var/spool/squid
cache_swap_low 75
#should be 1/4 of the physical memory installed in the system
cache_mem 1000 MB
range_offset_limit -1 KB
maximum_object_size 4 GB
minimum_object_size 10 KB
quick_abort_min -1 KB
# not yet sure that what options during compilation should be provided and if i have defined this directive correctly
cache_replacement_policy heap
#---------Refresh Pattern Portion--------------------------
# Custom Refresh patterns will come first
#specific for youtube custom refreshpatterns belowones....
refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload
# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
# This portion is not understood yet well what does it mean?
# Let the clients favorite video site through with full caching
# - they can come from any of a number of youtube.com subdomains.
# - this is NOT ideal, the 'merging' of identical content is really needed here
acl youtube dstdomain .youtube.com
cache allow youtube
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl CONNECT method CONNECT
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager
acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id)
# storeurl rewrite helper program
storeurl_rewrite_program /usr/local/etc/squid/storeurl.pl
storeurl_access allow store_rewrite_list
storeurl_access deny all
storeurl_rewrite_children 1
storeurl_rewrite_concurrency 10
#Allow access from localhost only
http_access allow localhost
http_access deny all
-----------------------------------------------------------------------------------------
This is the script i be looking forward to use as per configuration guide.
------------------------------------------------------------------------------------------
#your perl location in here, mine is #!/bin/perl
$|=1;
while (<>) {
@X = split;
$x = $X[0];
$_ = $X[1];
} elsif (m/^http:\/\/([0-9.]{4}
|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=
[0-9]*).*?\&(id=[a-zA-Z0-9]*)/) {
print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $2 .
"&" . $3 . "\n";
} else {
print $x . $_ . "\n";
}
}
----------------------------------------------------------------------------------------
Just for the completion sake only here is the copy of my squid.conf that is user facing.......However if somebody could give suggestions over it as well will definately be really thankful.
# This is the configuration file for the instance1 which is serving the user requests by forwarding it to the local parent peer. All the logic of Authentication/Access control is build here. Name this file squidinst1.conf
#---Adminsitrative Section-----
visible_hostname squidLhr1
unique_hostname squidlhr1main
pid_filename /var/run/inst1squid.pid
http_port 8080
icp_port 0
snmp_port 3161
access_log /var/logs/inst1access.log squid
cache_log /var/logs/inst1cache.log
cache_store_log /var/logs/inst1store.log
cache_effective_user proxy
cache_mgr squidAdmins_at_virtual.local
#-------------cache peer defined mulitple instance setup---------------------
cache_peer 127.0.0.1 parent 3128 0 default no-digest no-query proxy-only
# uncomment the below line if the proxy server has no direct connection to the internet and peer is the only way to go....
#never_direct allow all
#prefer_direct off (Must use this directive in the multi instances setup where proxy server has a direct connection)
# ---------------LAT configuration------------------------------
# ------------------testing yet pending-------------------------
#Define Local Servers
#acl localservers dstdomain .mcb.com.pk (preferred method)
#acl localservers dst 10.0.0.0/8 (Second choice)
# Local server should never be forwarded to neighbour/peers and they should never be cached.
#always_direct allow localservers
#cache deny LocalServers
#------------------------------------------------------------------
# -------------Cache Directory Section--------------------------
cache_dir aufs /var/spool/squid 10000 16 256
coredump_dir /var/spool/squid
cache_swap_low 75
cache_replacement_policy lru
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
#----------cache_manager and Local host section--------------------
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
#-----------------Defining ports section--------------------
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
#acl Safe_ports port 70 # gopher (Commented as DENY THIS PORT IS RECOMMENDED)
#acl Safe_ports port 210 # wais (Commented as DENYING of THIS PORT IS RECOMMENDED FROM SECURITY POINT OF VIEW)
acl Safe_ports port 1025-65535 # unregistered ports
#acl Safe_ports port 280 # http-mgmt (Commented as DENYING of THIS PORT IS RECOMMENDED FROM SECURITY CONTEXT)
#acl Safe_ports port 488 # gss-http (Commented as DENYING of THIS PORT IS RECOMMENDED FROM SECURITY CONTEXT)
#acl Safe_ports port 591 # filemaker (Commented as DENYing of THIS PORT recommended AS THIS SERVICE IS UNREQUIRED)
#acl Safe_ports port 777 # multiling http (DENYTHIS of this PORT hence commented)
acl CONNECT method CONNECT
#---------------defining Windows Update Section---------------
acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
###------working hours---------------
acl whbl time MTWHF 09:00-12:59
acl whal time MTWHF 13:00-17:00
#-----------------------ACL Section---------------------------
acl BIP dst "/etc/squid/blockedipdst"
###Definition of MP3/MPEG
acl FTP proto FTP
acl MP3url urlpath_regex \.mp3(\?.*)?$
acl Movies rep_mime_type video/mpeg
acl MP3s rep_mime_type audio/mpeg
###Definition of Flash Video
acl deny_rep_mime_flashvideo rep_mime_type video/flv
###Definition of Porn
#acl Sex urlpath_regex sex
acl pornSites url_regex "/etc/squid/pornlist"
####Definition of YouTube.
## The videos come from several domains
acl youtube_domains dstdomain .youtube.com .googlevideo.com .ytimg.com
###Definition of FaceBook
acl facebook_sites dstdomain .facebook.com
#### Definition of MSN Messenger
acl msn urlpath_regex -i gateway.dll
acl msnd dstdomain messenger.msn.com gateway.messenger.hotmail.com
acl msn1 req_mime_type application/x-msn-messenger
####Definition of Skype
acl numeric_IPs url_regex ^(([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)|(\[([0-9af]+)?:([0-9af:]+)?:([0-9af]+)?\])):443
acl Skype_UA browser ^skype^
##Definition of Yahoo! Messenger
acl ym dstdomain .messenger.yahoo.com .psq.yahoo.com
acl ym dstdomain .us.il.yimg.com .msg.yahoo.com .pager.yahoo.com
acl ym dstdomain .rareedge.com .ytunnelpro.com .chat.yahoo.com
acl ym dstdomain .voice.yahoo.com
acl ymregex url_regex yupdater.yim ymsgr myspaceim
## Other protocols Yahoo!Messenger uses ??
acl ym dstdomain .skype.com .imvu.com
###Definition for Disallowing download of executables from web#####
acl downloads url_regex "/etc/squid/restrictdownload"
###Definiton of Torrentz####
acl torrentSeeds urlpath_regex \.torrent(\?.*)?$
###Definition of Rapidshare###
acl dlSites dstdomain .rapidshare.com .rapidsharemegaupload.com .filespump.com
#acl localnet src "/etc/squid/localnet.conf"
#localnet comprised of Fullaccessusers, power users, Fully restricted users
#Allowing windows update access must be defined before any authentication/authorization is defined.
http_access allow CONNECT wuCONNECT localnet
http_access allow windowsupdate localnet
# access for localhost
http_access allow CONNECT wuCONNECT localhost
http_access allow windowsupdate localhost
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager
# Deny request to unknown ports
http_access deny !Safe_ports
# Deny request to other than SSL ports
http_access deny CONNECT !SSL_ports
#Allow access from localhost
http_access allow localhost
# Denied access to the pornsites
http_access deny PornSites
---------------------Other Access controls based on the user groups-----------------------------
#http_access deny PUsr msnd
#http_access deny RUsr msnd
#http_access deny PUsr msn
#http_access deny RUsr msn
#http_access deny PUsr msn1
#http_access deny RUsr msn1
#http_access deny PUsr numeric_IPs
#http_access deny RUsr numeric_IPs
#http_access deny PUsr Skype_UA
#http_access deny RUsr Skype_UA
#http_access deny PUsr ym
#http_access deny RUsr ym
#http_access deny PUsr ymregex
#http_access deny RUsr ymregex
#http_reply_access deny Movies RUsr
#http_reply_access deny MP3s RUsr
#http_access deny RUsr FTP
#http_access deny RUsr MP3url
#http_access deny RUsr BIP
#http_access deny RUsr downloads
#http_reply_access deny deny_rep_mime_flashvideo RUsr
#http_access deny RUsr facebook_sites
#http_access deny RUsr youtube_domains
#http_access allow PUsr !workinghours youtube_domains
#http_access allow PUsr !workinghours facebook_sites
#http_access allow PUsr !workinghours BIP
#http_access allow FcUsr
#http_access allow PUsr
#http_access allow RUsr
---------------------End of Other Access controls based on the user groups-----------------------------
#---------Defining user based Authentication/Authorization-----------------
auth_param negotiate program /usr/libexec/squid/squid_kerb_auth
auth_param negotiate children 10
auth_param negotiate keep_alive on
# basic auth ACL controls to make use of it are.
#acl auth proxy_auth REQUIRED
#http_access deny !auth
#http_access allow auth
#Groups fom Mailserver Domain_child1:
external_acl_type squid_kerb_ldap_mailserver1 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR1_at_MAILSERVER.VIRTUAL.LOCAL
external_acl_type squid_kerb_ldap_mailserver2 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR2_at_MAILSERVER.VIRTUAL.LOCAL
external_acl_type squid_kerb_ldap_mailserver3 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR3_at_MAILSERVER.VIRTUAL.LOCAL
acl ldap_group_check_mailserver1 external squid_kerb_ldap_mailserver1
acl ldap_group_check_mailserver2 external squid_kerb_ldap_mailserver2
acl ldap_group_check_mailserver3 external squid_kerb_ldap_mailserver3
# external_acl_type squid_kerb_ldap_bt1 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR1_at_CH2.VIRTUAL.LOCAL
# external_acl_type squid_kerb_ldap_bt2 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR2_at_CH2.VIRTUAL.LOCAL
# external_acl_type squid_kerb_ldap_bt3 ttl=3600 negative_ttl=3600 %LOGIN /usr/libexec/squid/squid_kerb_ldap -g INETGRLHR3_at_CH2.VIRTUAL.LOCAL
#acl ldap_group_check_bt1 external squid_kerb_ldap_bt1
#acl ldap_group_check_bt2 external squid_kerb_ldap_bt2
#acl ldap_group_check_bt3 external squid_kerb_ldap_bt3
http_access deny ldap_group_check_mailserver2 msnd
#http_access deny ldap_group_check_bt2 msnd
http_access deny ldap_group_check_mailserver3 msnd
#http_access deny ldap_group_check_bt3 msnd
http_access deny ldap_group_check_mailserver2 msn
#http_access deny ldap_group_check_bt2 msn
http_access deny ldap_group_check_mailserver3 msn
#http_access deny ldap_group_check_bt3 msn
http_access deny ldap_group_check_mailserver2 msn1
#http_access deny ldap_group_check_bt2 msn1
http_access deny ldap_group_check_mailserver3 msn1
#http_access deny ldap_group_check_bt3 msn1
http_access deny ldap_group_check_mailserver2 numeric_IPs
#http_access deny ldap_group_check_bt2 numeric_IPs
http_access deny ldap_group_check_bt2 numeric_IPs
#http_access deny ldap_group_check_bt2 numeric_IPs
http_access deny ldap_group_check_mailserver2 Skype_UA
#http_access deny ldap_group_check_bt2 Skype_UA
http_access deny ldap_group_check_mailserver3 Skype_UA
#http_access deny ldap_group_check_bt3 Skype_UA
http_access deny ldap_group_check_mailserver2 ym
#http_access deny ldap_group_check_bt2 ym
http_access deny ldap_group_check_mailserver3 ym
#http_access deny ldap_group_check_bt3 ym
http_access deny ldap_group_check_mailserver2 ymregex
#http_access deny ldap_group_check_bt2 ymregex
http_access deny ldap_group_check_mailserver3 ymregex
#http_access deny ldap_group_check_bt3 ymregex
###----Most Restricted settings Exclusive for Normal users......###
http_access deny ldap_group_check_mailserver3 Movies
#http_access deny ldap_group_check_bt3 Movies
http_access deny ldap_group_check_mailserver3 MP3s
#http_access deny ldap_group_check_bt3 MP3s
http_access deny ldap_group_check_mailserver3 FTP
#http_access deny ldap_group_check_bt3 FTP
http_access deny ldap_group_check_mailserver3 MP3url
#http_access deny ldap_group_bt3 MP3url
http_reply_access deny ldap_group_check_mailserver3 deny_rep_mime_flashvideo RUsr
#http_reply_access deny ldap_group_check_bt3
http_access deny ldap_group_check_mailserver3
#http_access deny ldap_group_check_bt3
http_access deny ldap_group_check_mailserver3
#http_access deny ldap_group_check_bt3
http_access deny ldap_group_check_mailserver3
#http_access deny ldap_group_check_bt3
http_access deny ldap_group_check_mailserver3 downloads
#http_access deny ldap_group_check_bt3 downloads
#http_access deny RUsr torrentSeeds
http_access deny ldap_group_check_mailserver3 torrentSeeds
http_access deny ldap_group_check_bt3 torrentSeeds
#http_access deny RUsr dlSites
http_access deny ldap_group_check_mailserver3 dlSites
#http_access deny ldap_group_check_bt3 dlSites
###----End of Most Restricted settings Exclusive for Normal users......###
http_access allow ldap_group_check_mailserver2 !workinghours youtube_domains
#http_access allow ldap_group_check_bt2 !workinghours youtube_domains
http_access allow ldap_group_check_mailserver2 !workinghours facebook_sites
#http_access allow ldap_group_check_bt2 !workinghours facebook_sites
http_access allow ldap_group_check_mailserver2 BIP !workinghours
#http_access allow ldap_group_check_bt2 BIP !workinghours
http_access allow ldap_group_check_mailserver1
#http_access allow ldap_group_check_bt1
http_access allow ldap_group_check_mailserver2
#http_access allow ldap_group_check_bt2
http_access allow ldap_group_check_mailserver3
#http_access allow ldap_group_check_bt3
http_access deny all
_________________________________________________________________
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
https://signup.live.com/signup.aspx?id=60969
Received on Wed May 19 2010 - 12:20:59 MDT
This archive was generated by hypermail 2.2.0 : Wed May 19 2010 - 12:00:06 MDT