How to receive Email in Java EE application

Question

Obviously it's not so difficult to send out emails from a Java EE application via JavaMail. What I am interested in is the best pattern to receive emails (notification bounces, mostly)? I am not interested in IMAP/POP3-based approaches (polling the inbox) - my application shall react to inbound emails.

One approach I could think of would be

• Keep existing MTA (postfix on linux in my case) -> ops team already knows how to configure / operate it
• For every mail that arrives, spawn a Java app that receives the data and sends it off via JMS. I could do this via an entry in /etc/aliases like myuser: "|/path/to/javahelper" with javahelper calling the Java app, passing STDIN along.
• MDB (part of Java EE application) receives JMS message, parses it, detects bounce message and acts accordingly.

Another approach could be

• Open a listening network socket on port 25 on the Java EE application container.
• Associate a SessionBean with the socket. Bean is part of Java EE application and can parse/detect bounces/handle the messages directly.
• Keep existing MTA as inbound relay, do all its security/spam filtering, but forward emails to myuser (that pass the filter) to the Java EE application container, port 25.

The first approach I have done before (albeit in a different language/setup).

From a performance and (perceived) cleanliness point of view, I think the second approach is better, but it would require me to provide a proper SMTP transport implementation. Also, I don't know if it's at all possible to connect a network socket with a bean...

What is your recommendation? Do you have details about the second approach?

Futilitarian

Which approach did you finally chose? – Bigford

The project went on a backburner for quite a while. Right now I'm working on it again, but I have not yet implemented the receiving part. So far, my plan is to follow sleske's suggestion and scan an email inbox in regular intervals via IMAP. – Futilitarian

Get rid of the jms in op 1 and just send it via spawned curl/http to a rest endpoint and you can cut one piece (JMS/MDB) of conf/complexity out. – Townie

@Townie - thanks, but JMS is not the issue, the OP is about best pattern for receiving email. – Futilitarian

@hank - understood. Pointing out that the JMS is not necessary. It affirms pattern 1: use (active) external system to push mail via REST to JEE container. – Townie

FYI: I ended up following the second approach: Java AS implemented SMTP MTA by help of SubEtha SMTP. – Futilitarian

Answer 1

Perhaps SubEtha SMTP [edit: now relocated from Google Code to GitHub] would be of interest -- it is a Java library that allows your application to receive SMTP mail. I'm about to try it out.

Here's a somewhat related Stack Overflow question (and someone suggests SubEthaSMP):
What is the easiest way for a Java application to receive incoming email?

Cartercarteret

For me, this was the solution that worked best. – Futilitarian

Answer 2

I don't think the second approach is "cleaner". On the contrary, it requires you to implement a significant part of a standard MTA, so I would recommend against it.

I believe that polling a POP/IMAP server is actually the cleanest way to do this. Why did you decide against it? If the POP/IMAP server and your service are in the same LAN (or even on the same maching), a poll will be quite inexpensive. You can do it every 10-20s for minimum delay, that should not cause problems. While this may look a bit technically inelegant, you will use a standard interoperation protocol (POP3/IMAP), which gives you flexibility while avoiding to reimplement a mailserver.

The approach of spawning a Java app also seems viable, but I'd prefer the polling, because:

a) The interface you use (POP3/IMAP) is more standardized, while the interface you use to "plug in" to the mail server will be server-specific (on Unix, you could use e.g. procmail, but you still depend on specific software)

b) Launching a separate process per mail will probably have much more overhead than polling.

Incidentally: A third approach would be to somehow dump the incoming mails as files into an "incoming" directory (many mailservers can do this), then poll the directory. Polling a directory will be even less expensive than polling a server. Just beware of synchronization issues (reading half-written mail, several concurrent readers reading the same mail file...)

My experience:

I have implemented systems using both approaches (IMAP polling, and spawning a separate process). The polling was for a reasonably large Java app which processed data that people sent to a mailbox; I did not encounter any problems wrt polling. The spawning approach was for a small Perl script; I just did it since it was a simple program that only processed a few mail per day, and plugging into the mailserver was easier than doing IMAP in Perl.

Backbencher

@sleske: Thx for the detailed response! Unfortunately the MTA does not provide POP3/IMAP facilities (today)... I'll think about which part is easier. – Futilitarian

@Futilitarian were you able to solve it? – Doukhobor

Yes, we followed the 2nd approach and implemented a SMTP server on top of SubEtha SMTP. – Futilitarian

Answer 3

The "correct" way according to the Java EE architecture would be to have a JCA connector to do inbound/outbound connection with the SMTP server.

The JCA connector can do whatever you want, including threading and connection to external systems with sockets. Actually JMS is just a special kind of JCA connector that connects to JMS broker and delivers message to "regular" MDB. A JCA connector can then poll the SMTP server and delivers the message to a custom MDB.

The best document about JCA is Creating Resource Adapters with J2EE Connector Architecture 1.5, and it does actually use the example of email delivery. Luck you :) I suggest you have a look at it. The code can be found as part of the Java EE samples and uses JavaMail, but I don't know if it's production ready.

Related:

• Java EE architecture suggestions for a service/daemon?
• Java EE application which listen to a socket request.

Suggestive

That looks pretty complicated... Why not just implement a stateless EJB that periodically checks for new Emails (using the timer service)? Does the JCA approach bring me any additional advantages in terms of performance, reliability, etc.? – Bigford

@Bigford You could indeed use an EJB Timer or spawn a thread from a ServletContextListener. I've used both for background jobs, that works, but according to the JEE "philosophy" such infrastructure thingy should be handled by the app server itself or a JCA connector, and not be part of the application code. Also, infrastructure settings such as SMTP settings should not be considered the same as "business" settings with env-entry. But ultimately, that's a question of design purity, and what you want it to have the job done. So go ahead with what is simplest for you. – Suggestive

ewernli is 100% correct here (In fact this sounds like a really great potential OSS project). anything you do outside of the container managed feature set is not covered by the SLA of an JEE application server. This becomes specially critical if you start distributing across containers. – Townie

I've used the JCA Inbound Mail connector from the examples in production before. (Interactive Marketing, email messages in the millions per day) Made some minor changes thorugh my own code review, mostly around error handling and input validation. i.e. throw an exception if there isn't a DKIM header. It worked fine. Need to remember that JCA spec covers many types of integration and connection scenarios, thats why it seems so complicated. – Yocum

Answer 4

What about Apache James?

it implements all the stack, and let you react to incoming mail with a servlet-like approach; it's open source, fully apache licensed (so can be used in commercial products), and has already been tested for years.

Pinnatisect

Awesome idea, I'll look into it! Sounds like it could implement my event-based approach. – Futilitarian

Answer 5

Using an ESB - standalone or embeded.

"An ESB brings flow-related concepts such as transformation and routing to a Service-Oriented Architecture. An ESB can also provide an abstraction for endpoints. This promotes flexibility in the transport layer and enables loose coupling and easy connection between services."

For example MULE "Mule ESB is the most widely used open source ESB. A lightweight integration platform and service container, Mule ESB provides features for web services, message routing, mediation, transformation and transaction management, helping developers integrate their applications in hours, not weeks"

How to receive emails via mule http://books.dzone.com/articles/mule-messaging-chapter?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+zones%2Fsoa+(SOA+Zone)

Below is simply configuration which send JMS message as reaction on received message (it is all you need) - define inbound (imap/pop3/etc) - define outbound.

<imap:inbound-endpoint user="bob" password="password" host="localhost" port="65433" checkFrequency="3000"/> 
<jms:outbound-endpoint queue="my.destination" connector-ref="jmsQueueConnector"/>

Oriflamme

Your example with imap:inbound-endpoint will retrieve mails via IMAP, which is exactly what the OP explicitly rejected. – Backbencher

As a matter of fact, from the docs it seems Mule does not even support incoming SMTP, only outgoing. So Mule might not help here... – Backbencher

Though this isn't what the OP would look for, I find this to be a more proper solution for an enterprise as it decouples e-mail handling from the application, the application can expose a service endpoint for the messages. – Partheniaparthenocarpy