Re: [squid-users] WCCP

Ross Kovelman wrote:
>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>> Date: Tue, 27 Oct 2009 12:17:12 +1300
>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>> Subject: Re: [squid-users] WCCP
>>
>> On Wed, 21 Oct 2009 12:20:00 -0400, Ross Kovelman
>> <rkovelman_at_gruskingroup.com> wrote:
>>>> From: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>> Date: Mon, 19 Oct 2009 22:35:36 -0400
>>>> To: Amos Jeffries <squid3_at_treenet.co.nz>
>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>> Subject: Re: [squid-users] WCCP
>>>>
>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>> Date: Tue, 20 Oct 2009 13:20:27 +1300
>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>> Subject: Re: [squid-users] WCCP
>>>>>
>>>>> On Mon, 19 Oct 2009 20:06:55 -0400, Ross Kovelman
>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>> Date: Tue, 20 Oct 2009 12:40:02 +1300
>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>
>>>>>>> On Mon, 19 Oct 2009 18:26:18 -0400, Ross Kovelman
>>>>>>> <rkovelman_at_gruskingroup.com> wrote:
>>>>>>>>> From: Amos Jeffries <squid3_at_treenet.co.nz>
>>>>>>>>> Date: Tue, 20 Oct 2009 11:04:42 +1300
>>>>>>>>> To: Ross Kovelman <rkovelman_at_gruskingroup.com>
>>>>>>>>> Cc: "squid-users_at_squid-cache.org" <squid-users_at_squid-cache.org>
>>>>>>>>> Subject: Re: [squid-users] WCCP
>>>>>>>>>
>>>>>>>>> On Mon, 19 Oct 2009 14:21:44 -0400, Ross Kovelman wrote:
>>>>>>>>>>> From: Amos Jeffries
>>>>>>>>>>>
>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>> From: Amos Jeffries:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ross Kovelman wrote:
>>>>>>>>>>>>> I am going to be using WCCP. I did another reconfigure with
>>>>>>>>>>>>> the
>>>>>>>>>>>>> --enable
>>>>>>>>>>>>> WCCP option. How can I check that it is on and running? The
>>>>> next
>>>>>>>>>>>>> step I
>>>>>>>>>>>>> need to do is upgrade to version 2 since the Cisco only
>>>>>>> communicates
>>>>>>>>>>>>> on
>>>>>>>>>>>>> version 2. I tried to do the patch < upgrade patch but then
>> I
>>>>> get
>>>>>>> a
>>>>>>>>>>>>> response with path to upgrade and I am not sure where the
>> file
>>>>> is
>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> patch.
>>>>>>>>>>>>> There is zero need to patch for support WCCPv2. It's been
>> built
>>>>>>> into
>>>>>>>>>>>>> Squid for many years now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Run "./configure --help".
>>>>>>>>>>>>> * If it lists "--disable-wccpv2" there is no need to do
>>>>> anything.
>>>>>>>>>>>>> * If it lists "--enable-wccpv2" , add that to your build
>>>>> options.
>>>>>>>>>>>>> * If it does not mention "wccpv2" at all upgrade your Squid
>>>>>>>>> version.
>>>>>>>>>>>>> Then setup squid.conf with the relevant wccp2_* options.
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://www.squid-cache.org/Doc/config/ or the wiki example
>>>>> configs
>>>>>>>>> have
>>>>>>>>>>>>> details on those.
>>>>>>>>>>>> Thanks again.
>>>>>>>>>>>> Running the ./configure --help only says this:
>>>>>>>>>>>> --disable-wccp Disable Web Cache Coordination V1
>>>>> Protocol
>>>>>>>>>>>> --disable-wccpv2 Disable Web Cache Coordination V2
>>>>> Protocol
>>>>>>>>>>>> When I did the install I ran the ./configure --enable wccp
>>>>>>>>>>>> option.
>>>>> I
>>>>>>>>>>>> didn't
>>>>>>>>>>>> say --enable-wccpv2, does this matter? I also have this in the
>>>>>>>>> config:
>>>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>>>> wccp2_return_method 1
>>>>>>>>>>>>
>>>>>>>>>>>> I am running Squid Web Proxy 2.7.STABLE5.
>>>>>>>>>>> Okay. Thats fine.
>>>>>>>>>>>
>>>>>>>>>>> The ./configure results mean that both WCCP versions are built
>>>>>>>>>>> into
>>>>>>>>>>> Squid by default unless you explicitly say --disable. Nothing
>>>>>>>>>>> extra
>>>>>>>>>>> needed to build them.
>>>>>>>>>>>
>>>>>>>>>>> The config options you have there are already WCCPv2-only
>> options
>>>>> for
>>>>>>>>>>> Cisco. Nothing new needed there either.
>>>>>>>>>>>
>>>>>>>>>>> If thats not working its a config error somewhere.
>>>>>>>>>>>
>>>>>>>>>> I am getting this in my cache log:
>>>>>>>>>>
>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 3128, FD 20.
>>>>>>>>>> commBind: Cannot bind socket FD 21 to *:3128: (48) Address
>> already
>>>>> in
>>>>>>>>> use
>>>>>>>>>> Accepting proxy HTTP connections at 0.0.0.0, port 80, FD 21.
>>>>>>>>>> commBind: Cannot bind socket FD 22 to *:80: (48) Address already
>> in
>>>>>>> use
>>>>>>>>>
> http://wiki.squid-cache.org/SquidFaq/TroubleShooting#Cannot_bind_socket_FD_NN>>
>>> _
>>>>>>>>> to_.2A:8080_.28125.29_Address_already_in_use
>>>>>>>>>
>>>>>>>>> I would suspect this as part of the problem. The WCCP router will
>> be
>>>>>>>>> trying to contact whatever software is already running on port
>> 3128,
>>>>>>> not
>>>>>>>>> the Squid you are starting with WCCP config.
>>>>>>>>>
>>>>>>>>>> Accepting ICP messages at 0.0.0.0, port 3130, FD 22.
>>>>>>>>>> WCCP Disabled.
>>>>>>>>>> Accepting WCCPv2 messages on port 2048, FD 23.
>>>>>>> To answer your earlier question:
>>>>>>> the above two lines means WCCPv1 is disabled, WCCPv2 is being
>> used.
>>>>>>>>>> Initialising all WCCPv2 lists
>>>>>>>>>>
>>>>>>>>>> As from my other posting I need WCCP enabled but it is showing
>>>>>>> disabled.
>>>>>>>>>> Any reason why? How can I resolve this. Below is my lines in
>>>>> config
>>>>>>>>>> wccp2_router 192.168.16.1
>>>>>>>>>> wccp2_forwarding_method 1
>>>>>>>>>> wccp2_return_method 1
>>>>>>>>> The above are only the config of how squid sends packets to the
>>>>> Cisco.
>>>>>>>>> WCCP requires configuration Cisco, the squid box OS and firewall,
>>>>>>>>> and
>>>>>>>>> routing tables. Any one of which could be the problem.
>>>>>>>>> The tutorials and troubleshooting info we have at present is a
>>>>>>>>> little
>>>>>>>>> spread out and disjointed. What how-to are you working from?
>>>>>>>>>
>>>>>>>>> Amos
>>>>>>>> Amos,
>>>>>>>> I just did a TCP dump and I think my problem is the GRE packet. It
>>>>>>>> is
>>>>>>>> being
>>>>>>>> listed I think as unknown. Shouldn't squid be able to pick the
>>>>>>>> packet
>>>>>>> up
>>>>>>>> and open it? The Cisco sees squid and relays the information good
>>>>>>>> but
>>>>>>> it
>>>>>>>> is
>>>>>>>> stopping at the squid box. Any ideas? I am just google'ing around
>> no
>>>>>>> set
>>>>>>>> how to.
>>>>>>> Okay. I've polished up our exemplar configs a little:
>>>>>>> http://wiki.squid-cache.org/Features/Wccp2
>>>>>>> (some way to go though).
>>>>>>>
>>>>>>> There are four parts to WCCP systems:
>>>>>>>
>>>>>>> 1) WCCP capture and redirect
>>>>>>>
>>>>>>> 2) gre tunnel between the Cisco and Squid boxes
>>>>>>>
>>>>>>> 3) squid box firewall settings and NAT capture of received gre
>>>>>>> packets
>>>>>>>
>>>>>>>
> http://wiki.squid-cache.org/ConfigExamples/Intercept#Traffic_Interception_cap>>
>>> t
>>>>>>> ure_into_Squid
>>>>>>>
>>>>>>> 4) squid.conf settings to make Squid contact the cisco router
>>>>>>>
>>>>>>> Amos
>>>>>>>
>>>>>> From what I have read and what you show only for the PIX and ASA
>> should
>>>>> be
>>>>>> the same. The Pix is actually correct for the ASA, although that is
>>>>> what
>>>>>> Cisco told me to do.
>> Hmm, I was worried a bit by this. Then realized what the problem was.
>> The difference appears to have been only a security ACL added to the ASA
>> config and the screwy wrapping.
>>
>> Thanks for that hint.
>>
>>>>>> As far as:
>>>>>> wccp2_router - My cisco router address
>>>>>> wccp2_forwarding_method - I took this out of my config as GRE is
>>>>>> default
>>>>>> wccp2_return_method - same as forward
>>>>>> wccp2_assignment_method - nothing in config
>>>>>> wccp2_service - nothing in config
>>>>>>
>>>>>> Am I missing something? If I have my cisco config turned on for WCCP
>>>>> and
>>>>>> squid running no one can browse the web. If I turn squid off and
>> leave
>>>>>> wccp
>>>>>> running on the Cisco browsing web is perfect. No issues. Anything
>> else
>>>>> to
>>>>>> check?
>>>>> ... rp_filter settings on the Squid box are turned off.
>>>>>
>>>>> ... iptables does REDIRECT or DNAT capture of the packets to the Squid
>>>>> http_port marked with "transparent"
>>>>>
>>>>>> bert:~ administrator$ sudo tcpdump -n -i en1 ip proto gre
>>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>> decode
>>>>>> listening on en1, link-type EN10MB (Ethernet), capture size 96 bytes
>>>>>> 15:00:33.599161 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>> gre-proto-0x883e
>>>>>> 15:00:34.715585 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 60:
>>>>>> gre-proto-0x883e
>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>> gre-proto-0x883e
>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>> gre-proto-0x883e gre-proto-0x883e
>>>>>> 15:00:34.805734 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>> gre-proto-0x883e
>>>>>> 15:00:34.808181 IP 192.168.xx.1 > 192.168.xx.xxx: GREv0, length 56:
>>>>>> gre-proto-0x883e
>>>>>>
>>>>>> Does that help? Let me know what you need from me so we can resolve
>>>>> this.
>>>>>> I did mask off my IP but the IP prior to the > is the ASA and the
>>>>> numbers
>>>>>> after is the squid server
>>>>>>
>>>>>> Thanks
>>>> Amos,
>>>>
>>>> I have this in my sysctl config:
>>>> net.ipv4.ip_forward =1
>>>> net.ipv4.conf.all.rp_filter = 0
>>>>
>>>> That should take care of the rp_filter. Although how can I check that
>> I
>>>> don't know. I am also running transparent so I assume that iptables
>>>> thing
>>>> you wrote I do not need to do?
>>>>
>>>> Thanks
>>>>
>>>>
>>> I am starting to look more into this and what I see is this on the
>> firewall
>>> log:
>>> Oct 21 12:03:37 bert ipfw: 12313 Accept P:47 192.168.xx.1
>> 192.168.xx.xxx
>>> in
>>> via en1
>>>
>>> P47 is GRE so I can see that the GRE packet from the ASA is passed and
>>> accepted to the squid server. I do not think Squid knows how to either
>>> decipher the GRE packet and or when it tries to send the information
>> back
>>> out its not going back to the client or ASA. How can I resolve this?
>> Aha, you have an ASA. Somehow I missed that detail earlier. This is the
>> specific ASA config details we have so far:
>> http://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoAsaWccp2
>>
>> Check that you have the squid bypass in the config. Thats one of the
>> critical parts.
>>
>> Good tracking so far.
>>
>> It's the OS business to unwrap the GRE packet into a normal TCP packet
>> before passing it to Squid. I'm not sure how ipfw ensures that. modprobe
>> ip_gre?
>>
>> The next bit will be to see if Squid receives the packet at all. With
>> debug_options ALL,6 or so cache.log should record a connection accepted
>> from the client and show what happens to it.
>>
>> Amos
>>
> Amos,
>
> Got it working, but I am having some timeout issues when browsing all
> websites. Do you know why or know what I can look for? I do see the ASA
> and squid server communicating now.
>
> Thanks

Not a clue. I'd guess some delays on the network. Perhaps during DNS
lookups.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE19
   Current Beta Squid 3.1.0.14