IMAP - How to search for all messages in a conversation thread?
Asked Answered
T

1

7

I'm working on an IMAP client, and would like to be able to find a list of all messages that are referenced in a conversation thread.

I know that the "References" header includes a list of messages referenced in a conversation, so I tried searching it like so:

TAG1 UID SEARCH all HEADER References "<CAOZnC-Nr+Q+bS_Nn5XiZ6A-kw=ZRBYrNbdoRfgZ65OjGA4_BKg@mail.gmail.com>"

But it returns nothing. I've successfully searched for a single message using the "Message-ID" header, like so:

TAG2 UID SEARCH all HEADER Message-ID "<[email protected]>"

Is there anyway to do this using IMAP 4?

NOTE: I'm aware that searches only work on 1 mailbox at a time, but at least half of these messages are in the target folder for my searches, and they don't show up in my search results.

Theatheaceous answered 29/5, 2013 at 14:19 Comment(4)
Does your server support the THREADING extension?Laurelaureano
Yes, I believe I can assume that....Theatheaceous
I don't know any of the specifics, but the THREAD extension defined at tools.ietf.org/html/rfc5256 may help you out with relating messages together.Laurelaureano
Turns out there are some clients this needs to work with that don't support the THREAD extension, so I'll have to find another way of doing this if possible. Thanks for the suggestion though.Theatheaceous
Z
11

Your search query is wrong -- you should remove the ALL token from there -- what you're sending is not syntactically valid. A correct form is this one, for example:

1 UID SEARCH HEADER references "<CAOZnC-Nr+Q+bS_Nn5XiZ6A-kw=ZRBYrNbdoRfgZ65OjGA4_BKg@mail.gmail.com>"

That shall get you going.

However, please keep in mind that the References header of a nested message might not contain everything what a message earlier in the thread will contain -- the generic line length limits apply, and this means that the total size of the header is limited and some of the "middle" items might get removed.

There are methods to overcome this limitation, like the INTHREAD operator defined in an experimental extension which is supported by Dovecot (and no other IMAP servers, AFAIK). Using that, you can simply ask for all UIDs within a thread where a particular message is present, like this:

1 UID SEARCH INTHREAD REFS HEADER Message-Id "something"

I've actually tested this with Dovecot and it works (and has worked for years). Please be advised that Dovecot does not support the MESSAGEID search key, though, and that the syntax for INTHREAD REFS is different than what the draft standard says. The command above works, though.

If you need to work without any extensions, then it looks like you have no other chance but to either:

  • Fetch HEADER.FIELDS[Message-Id References In-Reply-To] at once and analyze the messages client-side,
  • Whenever your SEARCH HEADER References returns a new UID, fetch HEADER.FIELDS[References In-Reply-To], extract the "new" message-ids from there, add them at the end of your queue and proceed further.

Finally, you could also rely on GMail's X-GM-THRID if you target mainly Gmail users, but please advised that their implementation is rather limited, including a hard limit of at most 100 messages per thread, non-standard thread correlation etc etc.

Well, pick your poison.

Zarger answered 31/5, 2013 at 17:25 Comment(4)
thanks for the feedback.... I tried your variant on the search for references, and I still didn't get any other messages, even though I know they're in the same mailbox and contain my target Message ID in their references list... but lots of info here, which I'm going to look into some more...Theatheaceous
Looks like a bug in your IMAP server, then. Try this on e.g. Dovecot with a mailbox you can control (and verify via grep that the headers really match.Aubervilliers
the email server I was testing this on was the Gmail IMAP server.... I haven't had a chance to test it on anything else (contract work)... thanks for your help!Theatheaceous
FWIW, the original query was valid, just not optimal. Adding a top-level "all" term is just like adding 'true ||' in a c/c++/java if().Kanaka

© 2022 - 2024 — McMap. All rights reserved.