Re: [squid-users] Help! one more time on on Squid3.HEAD(20110307), TPROXY4 and Iptables 1.4.9 + ebtables

From: Jim Binder <jbinder_at_cyphort.com>
Date: Wed, 16 Mar 2011 02:03:30 -0700

Amos,

Back at it again tonight -- So, when you did this (and I'm assuming you have -- maybe incorrectly ); ) how many nics did you have enabled.

Also, for grins, I just to ubuntu 11.04 with same config and tested with both 2.7Stable9 and 3.HEAD and still get it to work.

it's running on

Linux ubuntu 2.6.38-5-generic #32-Ubuntu SMP Tue Feb 22 16:09:46 UTC 2011 i686 i686 i386 GNU/Linux
2011/03/16 00:45:57.905 kid1| Accepting spoofingHTTP Socket connections at FD 17 on [::]:3129
2011/03/16 01:48:41.219 kid1| The AsyncCall httpAccept constructed, this=0x89a5b58 [call7]
2011/03/16 01:48:41.220 kid1| The AsyncCall httpAccept constructed, this=0x89a6cc8 [call9]
2011/03/16 01:48:41.222 kid1| AcceptLimiter.cc(40) kick: size=0
2011/03/16 01:48:41.222 kid1| AcceptLimiter.cc(40) kick: size=0
2011/03/16 01:48:41.223 kid1| AcceptLimiter.cc(40) kick: size=0
2011/03/16 01:48:41.223 kid1| AsyncJob constructed, this=0x89a7538 type=Comm::TcpAcceptor [job1]
2011/03/16 01:48:41.223 kid1| AcceptingHTTP Socket connections at FD 15 on [::]:3128
2011/03/16 01:48:41.223 kid1| AsyncJob constructed, this=0x876e4b0 type=Comm::TcpAcceptor [job2]
2011/03/16 01:48:41.223 kid1| Accepting spoofingHTTP Socket connections at FD 16 on [::]:3129
2011/03/16 01:48:41.223 kid1| Comm::TcpAcceptor status in: FD 15, [::] [ job1]
2011/03/16 01:48:41.224 kid1| TcpAcceptor.cc(80) start: FD 15, [::] [ job1] AsyncCall Subscription: 0x89a5bd8*1
2011/03/16 01:48:41.224 kid1| Comm::TcpAcceptor status out: FD 15, [::] [ job1]
2011/03/16 01:48:41.224 kid1| Comm::TcpAcceptor status in: FD 16, [::] [ job2]
2011/03/16 01:48:41.224 kid1| TcpAcceptor.cc(80) start: FD 16, [::] [ job2] AsyncCall Subscription: 0x89a5c40*1
2011/03/16 01:48:41.224 kid1| Comm::TcpAcceptor status out: FD 16, [::] [ job2]
root_at_ubuntu:/usr/local/squid/var/logs# tail -f cache.log | grep -i accept

and the stats seem to indicate the same thing.

root_at_ubuntu:/usr/local/squid/var/logs# iptables -t raw -nvL ; iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 822K packets, 1112M bytes)
 pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 81 packets, 22834 bytes)
 pkts bytes target prot opt in out source destination
Chain PREROUTING (policy ACCEPT 822K packets, 1112M bytes)
 pkts bytes target prot opt in out source destination
    0 0 DIVERT tcp -- * * 0.0.0.0/0 0.0.0.0/0 socket
   19 1140 TPROXY tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 TPROXY redirect 0.0.0.0:3129 mark 0x1/0xffffffff

Chain INPUT (policy ACCEPT 1261 packets, 167K bytes)
 pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 1 packets, 328 bytes)
 pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 81 packets, 22834 bytes)
 pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 83 packets, 23194 bytes)
 pkts bytes target prot opt in out source destination

Chain DIVERT (1 references)
 pkts bytes target prot opt in out source destination
    0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1
    0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0

It has to be a routing issue but I'm just not seeing the socket pop for any kind of read (and nothing is showing up on lo via tcp dump).

0: from all lookup local
32765: from all fwmark 0x1 lookup 100
32766: from all lookup main
32767: from all lookup default

root_at_ubuntu:/usr/local/squid/var/logs# ip ro li t all
local default dev lo table 100 scope host
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.66
172.16.1.0/24 dev eth0 proto kernel scope link src 172.16.1.137 metric 1
169.254.0.0/16 dev eth0 scope link metric 1000
default via 192.168.1.254 dev br0
default via 172.16.1.2 dev eth0 proto static
local 172.16.1.137 dev eth0 table local proto kernel scope host src 172.16.1.137
broadcast 192.168.1.0 dev br0 table local proto kernel scope link src 192.168.1.66
local 192.168.1.66 dev br0 table local proto kernel scope host src 192.168.1.66
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 172.16.1.255 dev eth0 table local proto kernel scope link src 172.16.1.137
broadcast 192.168.1.255 dev br0 table local proto kernel scope link src 192.168.1.66
broadcast 172.16.1.0 dev eth0 table local proto kernel scope link src 172.16.1.137
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
fe80::/64 dev eth0 proto kernel metric 256
fe80::/64 dev eth2 proto kernel metric 256
fe80::/64 dev eth1 proto kernel metric 256
fe80::/64 dev br0 proto kernel metric 256
unreachable default dev lo table unspec proto kernel metric -1 error -101 hoplimit 255
local ::1 via :: dev lo table local proto none metric 0 hoplimit 0
local fe80::20c:29ff:fee1:1307 via :: dev lo table local proto none metric 0 hoplimit 0
local fe80::20c:29ff:fee1:1307 via :: dev lo table local proto none metric 0 hoplimit 0
local fe80::20c:29ff:fee1:1311 via :: dev lo table local proto none metric 0 hoplimit 0
local fe80::20c:29ff:fee1:13fd via :: dev lo table local proto none metric 0 hoplimit 0
ff00::/8 dev eth0 table local metric 256
ff00::/8 dev eth2 table local metric 256
ff00::/8 dev eth1 table local metric 256
ff00::/8 dev br0 table local metric 256
unreachable default dev lo table unspec proto kernel metric -1 error -101 hoplimit 255

James S. Binder
Vice President, Engineering
Cyphort Inc.,

jbinder_at_cyphort.com
408.761.1403 (cell)

This information contained in this e-mail message and any attachments thereto, is intended only for the personal and confidential use of the recipient(s) named above. This message may be under the terms of a Mutual Non-Disclosure Agreement communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify use immediately by e-mail and delete this original message.

On Mar 15, 2011, at 3:02 AM, Amos Jeffries wrote:

> On 15/03/11 20:22, Jim Binder wrote:
>> Trying this one more time to see if anyone might know what's wrong in getting my transparent bridging with squid to work.
>> Config... pings work thought the box (the bridge is working however; the 3129 socket never pops with an HTTP request)
>>
>> Admin on Eth1, Internet on eth0 and Inside (client) interface on eth2. Br0 used as the bridge.
>>
>> Running Fedora core 14 (but went back as fare as 12 and couldn't get it to work)
>>
>> Squid Cache: Version 3.HEAD-20110307
>> configure options: '--enable-ecap' '--enable-icap-client' '--enable-linux-netfilter' --enable-ltdl-convenience
>>
>> iptables-1.4.9-1.fc14.i686
>> kernel-2.6.35.11-83.fc14.i686
>> ebtables-2.0.9-5.fc13.i686
>>
>> Went as far to turn on dynamic debug logging and I don't see what's wrong but the connect never seems to get made to the 3129 socket.
>>
>> [ 214.914113] TRACE: mangle:PREROUTING:rule:2 IN=eth2 OUT= MAC=00:40:f4:cd:01:70:00:50:56:36:df:78:08:00 SRC=192.168.1.91 DST=192.168.1.88 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=3380 DF PROTO=TCP SPT=48255 DPT=80 SEQ=1363486620 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A02522AA80000000001030306)
>> [ 214.914155] xt_TPROXY: redirecting: proto 6 c0a80158:80 -> 00000000:3129, mark: 1
>> [ 217.920783] TRACE: raw:PREROUTING:policy:3 IN=eth2 OUT= MAC=00:40:f4:cd:01:70:00:50:56:36:df:78:08:00 SRC=192.168.1.91 DST=192.168.1.88 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=3381 DF PROTO=TCP SPT=48255 DPT=80 SEQ=1363486620 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A025236680000000001030306)
>> [ 217.920846] TRACE: mangle:PREROUTING:rule:2 IN=eth2 OUT= MAC=00:40:f4:cd:01:70:00:50:56:36:df:78:08:00 SRC=192.168.1.91 DST=192.168.1.88 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=3381 DF PROTO=TCP SPT=48255 DPT=80 SEQ=1363486620 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0 OPT (020405B40402080A025236680000000001030306)
>> [ 217.920891] xt_TPROXY: redirecting: proto 6 c0a80158:80 -> 00000000:3129, mark: 1
> <snip>
>> [root_at_fw01 ~]#
>> [root_at_fw01 ~]# ip route list table all
>> local default dev lo table 100 scope host
>
> Tried with "table 100" created on eth0 and eth2 ?
>
> That seems to be needed recently.
>
> Everything else looks okay to me. Down to the packets hitting the TPROXY and DIVERT rules.
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE9 or 3.1.11
> Beta testers wanted for 3.2.0.5
Received on Wed Mar 16 2011 - 09:03:36 MDT

This archive was generated by hypermail 2.2.0 : Wed Mar 16 2011 - 12:00:03 MDT