Re: Async-IO

--MimeMultipartBoundary
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Oskar Pearson wrote:

> > Symptoms:
> > * aio_queue_request complaints on startup
> I get those too, but no side efects.

It is caused by a temporary overload while loading icons. [sort of
workaround]

> > * ~2 seconds to handle cache hits

This was caused by the 1 second timeout on select(), and only seen on
low loads. [fixed in two cooperating ways/hacks]

> > * pconn-banger failing when running more than one outstanding
> > request at a time (it defaults to 10).

This was a bug triggered when deferring reading of swap files. [fixed]

Another bug I quickly discovered while testing async-io was that the
main<->thread communication was badly designed (almost to the point of
abusing thread primitives) and could easily end up in a situation where
most or all of the I/O threads got into a 100% CPU spin for a couple of
seconds. This situation did occur if the main select() loop got
suspended for more than 3 seconds (i.e. if the system was paging).
[fixed]

> > I think we need to be very careful when recommending async-io on Linux.

> I wanted to get around that.. the benefits are REALLY worth it.

Agreed. Now that I have it working on my system, it works like a charm.

In view of the bugs I discovered yesterday I should have said "careful
when recommending async-io", as the problems was in Squid thread use and
not the Linux libraries.

> Can you send me the output of 'ldconfig -v'?

I was running RH4.2, with libc+libpthread from the FTP site pointed out
by the web page you recommended. Now I have upgraded my system to a
partial RH5.0 (RH5.0 development tools + latest RH5 libc update).

I had the same problems using both setups. I switched to RH5 libraries
(glibc), quickly discovered that it didn't help and took a deep dive
into the source.

I am preparing some patches this evening:
* aio fixes and optimizations
* a refreshed version of my body processing
* some other minor fixes
* updated test suite (tcp & pconn-banger + support in Squid).

And have a couple of thoughts on how to make a threaded Squid a bit
"snappier". More on this later.

/Henrik

--MimeMultipartBoundary--
Received on Tue Jul 29 2003 - 13:15:48 MDT