I'm experimenting with epoll's edge-mode which apparently only signals
files which have become ready for read/write (as opposed to level-mode
which signals all ready files, regardless of whether there were
already ready, or just became ready)
First let's get a clear view of the system, you need an accurate mental model of how the system works. Your view of epoll(7) is not really accurate.
The difference between edge-triggered and level-triggered is the definition of what exactly makes an event. The former generates one event for each action that has been subscribed on the file descriptor; once you consume the event, it is gone - even if you didn't consume all the data that generated such an event. OTOH, the latter keeps generating the same event over and over until you consume all the data that generated the event.
Here's an example that puts these concepts in action, blatantly stolen from man 7 epoll:
The file descriptor that represents the read side of a pipe (rfd) is registered on the epoll instance.
A pipe writer writes 2 kB of data on the write side of the pipe.
A call to epoll_wait(2) is done that will return rfd as a ready file descriptor.
The pipe reader reads 1 kB of data from rfd.
A call to epoll_wait(2) is done.
If the rfd file descriptor has been added to the epoll interface using
the EPOLLET (edge-triggered) flag, the call to epoll_wait(2) done in
step 5 will probably hang despite the available data still present in
the file input buffer; meanwhile the remote peer might be expecting a
response based on the data it already sent. The reason for this is
that edge-triggered mode delivers events only when changes occur on
the monitored file descriptor. So, in step 5 the caller might end up
waiting for some data that is already present inside the input buffer.
In the above example, an event on rfd will be generated because of the
write done in 2 and the event is consumed in 3. Since the read
operation done in 4 does not consume the whole buffer data, the call
to epoll_wait(2) done in step 5 might block indefinitely.
In short, the fundamental difference is in the definition of "event": edge-triggered treats events as a single unit that you consume once; level-triggered defines the consumption of an event as being equivalent to consuming all of the data belonging to that event.
Now, with that out of the way, let's address your specific questions.
in edge-mode, am I informed of readiness events that happen while I'm
not epoll_waiting
Yes, you are. Internally, the kernel queues up the interesting events that happened on each file descriptor. They are returned on the next call to epoll_wait(2), so you can rest assured that you won't lose events. Well, maybe not exactly on the next call if there are other events pending and the events buffer passed to epoll_wait(2) can't accommodate them all, but the point is, eventually these events will be reported.
What about events on one-shot files that haven't been rearmed yet?
Again, you never lose events. If the file descriptor hasn't been rearmed yet, should any interesting event arise, it is simply queued in memory until the file descriptor is rearmed. Once it is rearmed, any pending events - including those that happened before the descriptor was rearmed - will be reported in the next call to epoll_wait(2) (again, maybe not exactly the next one, but they will be reported). In other words, EPOLLONESHOT does not disable event monitoring, it simply disables event notification temporarily.
Ok, so will the last epoll_wait always be notified of the readiness of
socket S? Event if S is #1 (i.e. it's not rearmed)?
Given what I said above, by now it should be pretty clear: yes, it will. You won't lose any event. epoll offers strong guarantees, it's awesome. It's also thread-safe and you can wait on the same epoll fd in different threads and update event subscription concurrently. epoll is very powerful, and it is well worth taking the time to learn it!
