Re: [squid-users] Squid TProxy Problem

Amos,
Sorry for the typo; here are the rules:

ip rule add fwmark 1 lookup 100
ip -f inet route add local 0.0.0.0/0 dev lo table 100
ip -f inet route add local 0.0.0.0/0 dev eth0 table 100

Warm Regards,
Ali Majdzadeh Kohbanani

2011/6/11 Ali Majdzadeh <ali.majdzadeh_at_gmail.com>:
> Dear Amos,
> Hi
> As the documentation suggests, I have used the following rules, but
> except the first one, others fail:
>
> ip rule add fwmark 1 lookup 100
> ip -f inet route add local 0.0.0.0/0 dev lo table 100
> ip -f inet route add local 0.0.0.0/0 dev eth0 table 10
>
> Any ideas?
>
> Warm Regards,
> Ali Majdzadeh Kohbanani
>
>
> 2011/6/8 Ali Majdzadeh <ali.majdzadeh_at_gmail.com>
>>
>> Amos,
>> Thanks for your reply. As you had depicted in the diagrams, I think
>> you meant that the clients and the Squid box are both connected to the
>> gateway through the switch, didn't you? If it is so, yes, they are
>> connected, but the default gateway for the clients is set to the IP
>> address of the Squid box.
>> So, you mean we should insert a special firewall rule in our gateway
>> in order to detect and bypass the Squid outward traffic by its MAC
>> address, is that true? Does this method still preserves the clients'
>> IP addresses?
>> Sorry for my elementary questions and thanks in advance for your helpful notes.
>>
>> Warm Regards,
>> Ali
>>
>> 2011/6/8 Ali Majdzadeh <ali.majdzadeh_at_gmail.com>:
>> > Amos,
>> > Thanks for your reply. As you had depicted in the diagrams, I think
>> > you meant that the clients and the Squid box are both connected to the
>> > gateway through the switch, didn't you? If it is so, yes, they are
>> > connected, but the default gateway for the clients is set to the IP
>> > address of the Squid box.
>> > So, you mean we should insert a special firewall rule in our gateway
>> > in order to detect and bypass the Squid outward traffic by its MAC
>> > address, is that true? Does this method still preserves the clients'
>> > IP addresses?
>> > Sorry for my elementary questions and thanks in advance for your helpful notes.
>> >
>> > Warm Regards,
>> > Ali
>> >
>> > 2011/6/8 Amos Jeffries <squid3_at_treenet.co.nz>:
>> >> On 08/06/11 22:53, Ali Majdzadeh wrote:
>> >>>
>> >>> Amos,
>> >>> Hi
>> >>> Thanks for your reply. The Squid box has only one NIC and it is
>> >>> connected to the internet via it's default gateway, I think I should
>> >>> have corrected our network diagram as follows:
>> >>> Internet<-> Gateway<-> Squid<-> Clients
>> >>> Does this configuration make any difference?
>> >>
>> >> That diagram is no different, but a 1-NIC squid box would be:
>> >>
>> >> Internet<->Gateway<->Clients.
>> >> \<->Squid
>> >>
>> >> or:
>> >>
>> >> Internet<->Gateway<--switch-->Clients.
>> >> \<->Squid
>> >>
>> >>
>> >> That makes a difference.
>> >>
>> >> If you bump cache.log up to ALL,5 during a test connection. You may see
>> >> traffic arrive but then hang while connecting out.
>> >>
>> >> If you do see that behaviour in cache.log, the problems is at the gateway
>> >> end. It MUST be able to detect and bypass the Squid outward traffic by MAC
>> >> address or tcp_outgoing_tos instead of IP address.
>> >>
>> >> Amos
>> >>
>> >>> Thanks again for your reply. I will try to reconfigure the whole
>> >>> solution from scratch to find out where I go wrong.
>> >>>
>> >>> Warm Regards,
>> >>> Ali Majdzadeh Kohbanani
>> >>>
>> >>> 2011/6/8 Amos Jeffries<squid3_at_treenet.co.nz>:
>> >>>>
>> >>>> On 08/06/11 01:15, Ali Majdzadeh wrote:
>> >>>>>
>> >>>>> Amos,
>> >>>>> The configuration is as follows:
>> >>>>> Internet<-> Squid<-> Clients
>> >>>>>
>> >>>>> Would you please clarify what you mean by declaring "routing packets
>> >>>>> to the squid box"?
>> >>>>
>> >>>> That the packets actually do get passed/routed through the squid box and
>> >>>> not
>> >>>> via some other possible route.
>> >>>>
>> >>>>> Does the above configuration conform to the
>> >>>>> so-called declaration?
>> >>>>
>> >>>> If those are physical wires or even just logical routing table entries,
>> >>>> yes
>> >>>> it does.
>> >>>>
>> >>>>> If it is so, what should be done to solve the
>> >>>>> issue?
>> >>>>
>> >>>> Your packet counter incrementing is a good sign that the ruting layer is
>> >>>> okay.
>> >>>>
>> >>>>> Thanks again.
>> >>>>> By the way, we have compiled libcap from source and it is the latest
>> >>>>> version of the library.
>> >>>>
>> >>>> Okay. That should do :).
>> >>>>
>> >>>>
>> >>>>> 2011/6/6 Ali Majdzadeh<ali.majdzadeh_at_gmail.com>
>> >>>>>>
>> >>>>>> Amos,
>> >>>>>> Sorry, the packet counter increments, I made a mistake, but still no
>> >>>>>> logs either in access.log nor in cache.log.
>> >>>>
>> >>>>
>> >>>> Given that you have a recent libcap. That means we must suspect the
>> >>>> kernel
>> >>>> handling once TPROXY marks the packets.
>> >>>>
>> >>>> The "table 100" bit of the config has given a lot of people trouble.
>> >>>> AFAIK
>> >>>> "normally" you only have one such table entry and for TPROXY its internal
>> >>>> to
>> >>>> the kernel with the "lo" interface. BUT, some people have had to
>> >>>> configure
>> >>>> other interfaces to get it working.
>> >>>>
>> >>>> Try to add a table 100 (or whatever you called it) entry for each NIC the
>> >>>> box has. If your kernel accepts them check access.log again.
>> >>>>
>> >>>> If your kernel denies multiple tables, erase the existing one and try
>> >>>> creating one for each NIC. Repeating until you find one that works.
>> >>>>
>> >>>> OR, if that still fails. We have to get help from Balabit and/or
>> >>>> Netfilter
>> >>>> and figure WTF is happening.
>> >>>>
>> >>>> Amos
>> >>>>
>> >>>>>>
>> >>>>>> Warm Regards,
>> >>>>>> Ali Majdzadeh Kohbanani
>> >>>>>>
>> >>>>>> 2011/6/6 Ali Majdzadeh<ali.majdzadeh_at_gmail.com>:
>> >>>>>>>
>> >>>>>>> Amos,
>> >>>>>>> Hi
>> >>>>>>> The packet counter on -j TPROXY does not increment. So, why clients
>> >>>>>>> are able to surf the web?
>> >>>>>>>
>> >>>>>>> Warm Regards,
>> >>>>>>> Ali Majdzadeh Kohbanani
>> >>>>>>>
>> >>>>>>> 2011/6/6 Ali Majdzadeh<ali.majdzadeh_at_gmail.com>
>> >>>>>>>>
>> >>>>>>>> Amos,
>> >>>>>>>> Hi
>> >>>>>>>> Thanks for your reply. Ragarding the documentation, I have inserted
>> >>>>>>>> the following routing rules:
>> >>>>>>>> ip rule add fwmark 1 lookup 100
>> >>>>>>>> ip route add local 0.0.0.0/0 dev lo table 100
>> >>>>>>>> Now, access.log is populated with proper logs, but clients can not
>> >>>>>>>> surf the web, I mean the proxy server is unable to forward http
>> >>>>>>>> responses to clients' browsers. When the client enters for example
>> >>>>>>>> www.google.com, the connection to the http server is established but
>> >>>>>>>> the process halts at Waiting for www.google.com and after a while
>> >>>>>>>> Squid reports the unablility to retreive the requested URL.
>> >>>>>>>> By the way, we have disabled selinux.
>> >>>>>>>> Any ideas?
>> >>>>>>>>
>> >>>>>>>> Warm Regards,
>> >>>>>>>> Ali Majdzadeh Kohbanani
>> >>>>>>>>
>> >>>>>>>> 2011/6/6 Amos Jeffries<squid3_at_treenet.co.nz>:
>> >>>>>>>>>
>> >>>>>>>>> On 06/06/11 06:32, Ali Majdzadeh wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hello All,
>> >>>>>>>>>> I have setup the following configuration:
>> >>>>>>>>>> Squid (3.1.12) (--enable-linux-netfilter passed as the one and only
>> >>>>>>>>>> configure option)
>> >>>>>>>>>> Kernel (2.6.38.3)
>> >>>>>>>>>> iptables (1.4.11)
>> >>>>>>>>>>
>> >>>>>>>>>> I have added the following two directives in squid.conf:
>> >>>>>>>>>> http_port 3128
>> >>>>>>>>>> http_port 3129 tproxy
>> >>>>>>>>>>
>> >>>>>>>>>> Also, I have configured iptables with the following rules:
>> >>>>>>>>>> iptables -t mangle -N DIVERT
>> >>>>>>>>>> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
>> >>>>>>>>>> iptables -t mangle -A DIVERT -j MARK --set-mark 1
>> >>>>>>>>>> iptables -t mangle -A DIVERT -j ACCEPT
>> >>>>>>>>>> iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
>> >>>>>>>>>> --tproxy-mark 0x1/0x1 --on-port 3129
>> >>>>>>>>>>
>> >>>>>>>>>> Everything work as expected, I mean, the users can surf the web and
>> >>>>>>>>>> the proxy server is transparent. The problem is that actually there
>> >>>>>>>>>> is
>> >>>>>>>>>> no caching. I mean, both cache.log and access.log files are empty.
>> >>>>>>>>>> On
>> >>>>>>>>>
>> >>>>>>>>> That would be transparency to the point of not going through the
>> >>>>>>>>> proxy.
>> >>>>>>>>> access.log should have entries for each request.
>> >>>>>>>>>
>> >>>>>>>>>> the other hand, if I manually set the proxy configuration in
>> >>>>>>>>>> clients'
>> >>>>>>>>>> browsers (the IP address of the squid server and port number 3128)
>> >>>>>>>>>> everything is OK; the log files are incremented and objects are
>> >>>>>>>>>> cached.
>> >>>>>>>>>>
>> >>>>>>>>>> Have anyone faced the same issue?
>> >>>>>>>>>
>> >>>>>>>>> Some. Its usually boiled down to missing out some details omitted.
>> >>>>>>>>> building
>> >>>>>>>>> against libcap2 or routing packets to the squid box for example.
>> >>>>>>>>>
>> >>>>>>>>> Are the packet counters on that -j TPROXY rule showing captures?
>> >>>>>>>>>
>> >>>>>>>>> Did you follow the rest of the feature config?
>> >>>>>>>>> ie the special sub-routing table? OS packet filtering toggles?
>> >>>>>>>>> selinux
>> >>>>>>>>> updated to allow tproxy?
>> >>>>>>>>>
>> >>>>>>>>> Is this box even routing or bridging port 80 traffic for the
>> >>>>>>>>> network?
>> >>>>>>>>>
>> >>>>>>>>> Amos
>> >>>>>>>>> --
>> >>>>>>>>> Please be using
>> >>>>>>>>> Current Stable Squid 2.7.STABLE9 or 3.1.12
>> >>>>>>>>> Beta testers wanted for 3.2.0.8 and 3.1.12.2
>> >>>>>>>>>
>> >>>>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Please be using
>> >>>> Current Stable Squid 2.7.STABLE9 or 3.1.12
>> >>>> Beta testers wanted for 3.2.0.8 and 3.1.12.2
>> >>>>
>> >>
>> >>
>> >> --
>> >> Please be using
>> >> Current Stable Squid 2.7.STABLE9 or 3.1.12
>> >> Beta testers wanted for 3.2.0.8 and 3.1.12.2
>> >>
>> >
>
Received on Sat Jun 11 2011 - 11:45:39 MDT