Re: /dev/epoll on squid

On Sun, 2002-11-24 at 04:27, Robert Collins wrote:
> On Sat, 2002-11-23 at 08:00, David Nicklay wrote:
> >
> > This is my first attempt to get /dev/epoll working on squid. I am
> > having problems getting it to work correctly, and I could use some
> > help. I get a response on the first time I do a request, but I don't
> > get anything on the second request.
>
> Do you mean on a single client fd, or a single server fd, or do you mean
> that the second accept() never occurs, or that it occurs and the request
> never gets processed.

The second accept() does occur. The problem seems to be in something
that resets everything before a specific fd number in the server gets
reused. If I do a 2nd and 3rd request in parallel, one of them will get
through. The one that gets through is the one which has a different fd
number than the first request. I am guessing that resetting on a close
would do it, but I am not sure that I can do that without hacking other
areas besides comm_select.c and structs.h. Incidentally, I didn't
include this in the previous email, but you also need to add two fields
to _fde in structs.h in order to save events. This seems to be the
cleanest place to add it to.

struct _fde {
...
  int revents; /* events we have received notifications of */
  int events; /* events we are watching for at the moment */
...
}

> > Here are some things I have found about how epoll works that make things
> > really difficult:
> >
> > - epoll apparently cannot tell you about events from before you began
> > monitoring. I have a work around in place using poll with a zero
> > timeout on a single fd when you first begin to monitor it. I make sure
> > to perform the poll after the event has been added to the interest set
> > in the epoll module.
>
> Yeah, this is kindof expected, but also annoying :}. How much of a race
> is there if the call sequence is:
>
> connect()
> epollnotifymeonthingy()
>
> i.e. we don't delay at all?
> or perhaps we need to do
> connect()
> epollnotifymeonthingy()
> select() <- once!
> if select flag as neededing action foo on next io loop.

The former is what ephttpd (from David Libenzi's site) does, and he is
the person who wrote /dev/epoll, so I am assuming it would work. The
latter is the approach I have taken, though I have used poll() rather
than select without knowing which is better for a single file
descriptor.

> > - epoll is a notification based poll and as such it 'loses' events which
> > you are not monitoring for. I have another workaround in place, where I
> > basically look for all events and only handle those that I care about at
> > any given moment.
>
> I.e. if you are not watching for remote shutdowns, you are not told
> about them? Thats cool. We should watch for most everything then. Can it
> discriminate between link errors, remote shutdowns, ready to read, and
> ready to write? If so, then we can start watching for everything, and
> selectively remove certain items as we finish parts of the request.
>
> Regarding your code sample, can you please provide a diff rather than a
> copy of the while thing? It's much easier to review diffs :].

I have attached one to this email, though be forewarned the patch is
bigger than the original file, as I basically started from scratch in
comm_select.c. What was there was too much tied to select() and/or
poll() for me to do much with it.

> Lastly, squid-3.0 is much better suited to epoll() modification, as
> there is a more event driven comms IO model, which should suit epoll()
> nicely.

Agreed, however, squid-3.0 is a moving target and I was hoping to get
something quick and dirty working in a stable version before I moved on
to that. I have generally been hearing a lot of good things about
squid-3.0 from Brian Akins, though so I may be trying it 'Real Soon
Now'. :-)

-- 
David Nicklay
Location: CNN Center - SE0811A
Office: 404-827-2698	Cell: 404-545-6218