Re: [squid-users] Squid 3.3 Reverse Proxy Mode - 502 Errors when uploading files larger than 6MB

From: Eliezer Croitoru <eliezer_at_ngtech.co.il>
Date: Sat, 23 Nov 2013 01:51:56 +0200

Hey,

I would try to test it per case to understand the issue.
lets take a simple apache server using a POST form to upload a file:
upload_max_filesize = 10M
post_max_size = 10M

Set the above in php.ini.

Build a php form that will make it possible to upload a file.
Now for couple cases which are http.
I will take as an example a mp3 file of a recording.

I am using 3.4.0.2 on CentOS to test the case.
default settings of squid.
The proxy is in a forward proxy mode.
And it works just fine.
192.168.10.1 - - [23/Nov/2013:01:28:38 +0200] "POST /tests/uploader.php
HTTP/1.1" 200 30 "http://www1.ngtech.co.il/tests/1.html" "Mozilla/5.0
(X11; Ubuntu; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0"

Which has uploaded an mp3 file which is 11M file.

I will now add to the proxy a "cache_peer" which is not a https one
which in turn:
1385163442.173 880 192.168.10.100 TCP_MISS/200 299 POST
http://www1.ngtech.co.il/tests/uploader.php - FIRSTUP_PARENT/127.0.0.1
text/html

Which is clear from that the POST is OK.

 From the logs it seems like the connection was reset by peer in a read
failure.

Is this only a SSL site?
If it can be tested on a HTTP and non SSL connection it can help to
minimize the cause of the problem.

I would also try to remove all the request_header_access that you use
unless you know that there is a need for them.

Another thing is that it is not clear to me where is the source of the
incoming connection?
Is it from a part of the LAN or a WAN?

Thanks,
Eliezer

On 21/11/13 18:50, techguy005-me_at_yahoo.com wrote:
> I am currently running Squid 3.3.9 and 3.3.10 on RedHat
> Enterprise Server 5 in a reverse-proxy set-up and have an issue when attempting
> to upload a file (i.e. .xls) that is larger than 6MB. A 502 error is
> thrown by Squid. The backend parent web server is an IIS 7.5 Win2k8 R2.
> The oddity is the site functions just fine under the old Squid 2.6.22
> version. However after upgrading to 3.3.9 (and subsequently 3.3.10 to see
> if this error resolves), the upload issue of files larger than 6MB still
> exists.
>
> The 502 Error from the access.log reads:
> https://site.com/products/application/SomeThing/Batch.aspx -
> FIRSTUP_PARENT/192.168.1.5 text/html
> 1384868971.758 134798 192.250.4.3 TCP_MISS/502 5065 POST
>
> I turned up the debug level to: debug_options ALL,333
>
> This produced the output below in the cache.log file
> which showed a “(104) Connection reset by peer” error:
>
> 013/11/20 07:58:09.714 kid1| http.cc(1104)
> persistentConnStatus: persistentConnStatus: clen=0
> 2013/11/20 07:58:10.167 kid1| comm.cc(145)
> commHandleRead: comm_read_try: FD 10, size 16383, retval -1, errno 104
> 2013/11/20 07:58:10.167 kid1| AsyncCall.cc(85)
> ScheduleCall: IoCallback.cc(127) will call
> HttpStateData::readReply(local=192.250.4.3:55985 remote=192.168.1.5:443 FD 10
> flags=1, errno=104, flag=-1, data=0x10edade8, size=0, buf=0x10ee9ff0) [call264]
> 2013/11/20 07:58:10.167 kid1| AsyncCallQueue.cc(51)
> fireNext: entering HttpStateData::readReply(local=192.250.4.3:55985
> remote=192.168.1.5:443 FD 10 flags=1, errno=104, flag=-1, data=0x10edade8,
> size=0, buf=0x10ee9ff0)
> 2013/11/20 07:58:10.168 kid1| http.cc(1172) readReply:
> local=192.250.4.3:55985 remote=192.168.1.5:443 FD 10 flags=1: read failure:
> (104) Connection reset by peer.
> 2013/11/20 07:58:10.168 kid1| AsyncJob.cc(131) callEnd:
> HttpStateData::readReply(local=192.250.4.3:55985 remote=192.168.1.5:443
> flags=1, errno=104, flag=-1, data=0x10edade8, size=0, buf=0x10ee9ff0) ends job
> [ job25]
> 2013/11/20 07:58:10.168 kid1| AsyncJob.cc(141) callEnd:
> HttpStateData::readReply(local=192.250.4.3:55985 remote=192.168.1.5:443
> flags=1, errno=104, flag=-1, data=0x10edade8, size=0, buf=0x10ee9ff0) ended
> 0x10edaea8
> 2013/11/20 07:58:10.168 kid1| AsyncCallQueue.cc(53)
> fireNext: leaving HttpStateData::readReply(local=192.250.4.3:55985
> remote=192.168.1.5:443 flags=1, errno=104, flag=-1, data=0x10edade8, size=0,
> buf=0x10ee9ff0)
> 2013/11/20 07:58:10.169 kid1| errorpage.cc(1120) Convert:
> errorConvert: %%E --> '(104) Connection reset by peer'
>
> I played around with various parameters in the squid.conf
> file to no avail:
>
> # Base ACL rules to allow connections on port 80 and 443
> acl all src all
> acl port80 port 80
> acl port443 port 443
> http_access allow port80
> http_access allow port443
> http_access deny all
> http_reply_access allow all
>
> # Forces no caching of failed requests
> negative_ttl 0
>
> # Timeout value for closing persistent idle connections
> pconn_timeout 1 seconds
>
> #request_body_max_size 20 MB
> #client_request_buffer_max_size 20 MB
>
> # Turns off some HTTP Headers we do not want exposed
> via off
> #forwarded_for off
> request_header_access From deny all
> request_header_access Server deny all
> request_header_access WWW-Authenticate deny all
> request_header_access Link deny all
> request_header_access Proxy-Connection deny all
> request_header_access X-Cache deny all
> request_header_access X-Cache-Lookup deny all
> request_header_access Via deny all
> request_header_access Keep-Alive deny all
>
> # SSL HTTP Listeners
> https_port 168.250.1.2:443 accel protocol=https vhost
> cert=/apps/squid/etc/ssl/sslcert.pem cafile=/apps/squid/etc/ssl/verisign.pem
> defaultsite=site.com
>
> # SSL Cache Peer
> cache_peer 192.168.1.5 parent 443 0 proxy-only
> originserver ssl name=pilot_ssl ssldomain=site.com sslflags=DONT_VERIFY_PEER
>
> # SSL ACL
> acl pilot_ssl_IP_acl myip
> 168.250.1.2
>
> # SSL Cache Peer Access
> cache_peer_access pilot_ssl allow pilot_ssl_IP_acl
> port443
>
> Squid was compiled in this manner:
>
> ./configure --prefix=/apps/squid3.3.9 --enable-icmp
> --enable-ssl --with-openssl=/apps/ssl --disable-internal-dns --disable-ipv6
> --with-large-files
> --enable-external-acl-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,DB,squid_radius_auth
>
> As I mentioned before, everything works just fine under
> Squid 2.6.22 hitting the exact same back-end web server. However it
> breaks on Squid 3.3.9 and Squid 3.3.10. Something appears to have changed
> in the manner in which the connection/buffer/something functions between Squid
> 3.3.x and the back-end parent server that causes the hiccup with 6MB or larger
> files. Attempted to change the timeout value on the back-end parent
> server (IIS Windows) to 240 seconds, however all that did was cause the timeout
> to happen at the 240 second mark rather than the default 120 second mark.
> It is as if something causes a stall between Squid and IIS as the file
> upload never actually make it to the back-end parent server. Again, this
> functions just fine under Squid 2.6.22 so something is amiss.
>
> Any assistance would be greatly appreciated to resolve or
> further troubleshoot this issue. Thanks!
>
Received on Fri Nov 22 2013 - 23:52:17 MST

This archive was generated by hypermail 2.2.0 : Sat Nov 23 2013 - 12:00:04 MST