IMAP: Search for messages with UID greater than X (or generally, after my last search)

Question

I'm writing a script to analyze my mailbox and want to periodically check for new messages. The search criteria would be: give me the UIDs for all emails with UID greater than X, where X is the UID of the last email I processed.

Or, more generally, I'm looking for a way to only see messages since my last search.

Note that I'm not looking for seen/unseen messages; the script opens the mailbox as read-only, and I'd like it to not interfere with my flags, etc.

I know I can specify a date in the IMAP search, but the granularity of that seems to be by day, so not exactly what I need.

I'm starting with Gmail as the IMAP server, but would like to support generic IMAP servers in the future.

Is there way to search for emails with UID greater than X? Or another means of specify all messages since message X?

Myocardium

Answer 1

You can use IMAP SEARCH for UIDs. Assuming your most recently fetched UID is 1999, I think you would do:

SEARCH UID 2000:*

Rotz

Should be accepted answer. Also not sure I understood the RFC clearly: are we 100% sure that a given UID is never going to be used again over time? (email deletion, etc.) – Moluccas

@Moluccas that's not something I understand enough about to answer fully. If you ask that as a separate question on StackOverflow, you may get a better answer. According to RFC 3501 section 2.3.1.1, the UID "MUST NOT change during the session, and SHOULD NOT change between sessions" and changes to UIDs "MUST be detectable using the UIDVALIDITY mechanism" – Rotz

@Rotz I actually read it through, and it depends on the server implementation... – Moluccas

Just wanted to add that, "Also note that a UID range of 559:* always includes the UID of the last message in the mailbox, even if 559 is higher than any assigned UID value. This is because the contents of a range are independent of the order of the range endpoints. Thus, any UID range with * as one of the endpoints indicates at least one message (the message with the highest numbered UID), unless the mailbox is empty." – Figurehead

There's something called UIDVALIDITY that determines whether UIDs can be reused. In short: As long as the mailbox' UIDVALIDITY remains unchanged, UIDs cannot be reused. – Outfield

Answer 2

Why not use IMAP IDLE for this?

with IMAP IDLE, the server warns you every time a new message arrives

Bedchamber

because not supported widely. and you have to remain connected? – Linoel

@Linoel way to resurect an old thread :) - anyway it's true all you say but to 1st point i say "that's why i ASKED if there was a reason not to use on the OP's specific case", and as for the second, if your not connected you can't really search the mail box – Bedchamber

But if you were connected yesterday and reconnect today, you were connected and you are again, but cannot use IDLE. Also, if your application needs to be robust against intermittent network outages, you need to keep track client-side. So yes, if you can support IDLE, do support IDLE if it makes sense for you, but this does not solve the problem in the general case; and often you need to support the general case even if you do want to use IDLE as well when you can. – Supplemental

Disappointingly, as of Python 3.5, imaplib doesn't even support IDLE, but there is work in progress for 3.6, and a third-party prototype you can use in the meantime; github.com/athoune/imapidle – Supplemental