Email Forwarding like Craigslist - Rails

Question

I'm trying to do what craigslist's anonymous email does, but with Rails, also on the cheap. It is important for me to be able to add a header to the email, which is why basic email forwarding doesn't work.

One way I thought of was a SMTP server, whenever I read email via POP/IMAP, I then send an email to the true recipient of the email, with a proper FROM address and add in the header. This works, but a SMTP server is relatively costly.

The other way is to forward/redirect the email, but add in the header in between. I can't find any services or gems to do this though.

Please don't just say "Email Piping" because all that really means is feeding the email to your Rails program, what do you do after you have the email? How do you actually forward it.

Any ideas?

Portend

Answer 1

You'll need an email address using a domain with a MX server your sysadmin has control of. This could be a subdomain of your primary domain. Then what you do, is you configure the MTA software (Exim, Postfix... hopefully not qMail!) to pipe that email to Rails:

http://guides.rubyonrails.org/action_mailer_basics.html#receiving-emails

If the MTA is not installed on the same server as the rails application itself, you'll have to pipe the email to a little ad-hoc forwarder script that does something along the line of POSTing the email to your app, where you then manually pass that to your mailer.

In your mailer, you have access to all the headers, body, attachments etc. Provided you put some unique identifiers in the subject, or the Reply-To address, you can make the decision about which Mailer to instantiate to forward the mail onto its intended recipient.

We haven't done this yet, but we're going to be doing it for the same reasons. It may be a little over your head if you're not familiar with configuring an MTA however. Do you have a sysadmin you can land this task on?

At the code level, I'd be doing this:

1. User A (id = 1234) sends an email to User B (id = 5678)

2. Send the e-mail from any address you want, which you own, but set the Reply-To: to something like Reply-To: <[e-mail]>

   This is absolutely key to this working. It includes the ID of the sender, the ID of the recipient, and a checksum to prevent forgery. The checksum can be generated from a salt unique to each user, and is simply:

   checksum = Digest::MD5.hexdigest("#{sender.id}-#{recipient.id}-#{sender.mailer_salt}")

3. Now when you receive a reply via the MX you have configured for your "usermessages.your-domain.com" domain, the first thing you do is identify the sender and the recipient by parsing the To: field. You can easily identify who the sender and recipient are by split'ing out the parts. You can then generate a checksum and make sure it matches, to ensure somebody isn't trying to maliciously send mail as if it's from another user.

4. Once you have figured out the users involved, go ahead and send another e-mail, with one of these special Reply-To: headers (with the ID's reversed and the digest done using a different salt, obviously).

This is a very rudimentary, but perfectly functional example. You can put this digest anywhere you want, provided it will be preserved when the reply comes back (which makes the Reply-To: header a good fit. Some services use the subject line instead.

I would avoid making the salt something user-controlled, such as the user's password hash, since if the user changes that information (changes their password), the checksum will no longer validate.

Cheese

Thanks for the reply! I'm fine with the piping part. I suppose what I don't understand is the "forward the email onto is intended recipient" one. I'd like my from: address to be whoever send the email. However, from what I understand, changing the from address is very bad practice if you own your own mailing service and will often get you filtered out. Is this true? – Portend

That's right, you should never send an email "From" an address you don't own... it's likely to get stuck in spam filters. I thought you wanted to do this anonymously anyway? I'll update my answer with a more in-depth discussion of how I'd design this. – Cheese

See my revised answer for a technical discussion of what to do at the Rails level. Bear in mind I haven't done it; I just have been thinking about it since we will also being doing this. – Cheese

Also take note of Steve's answer, with regards to avoiding piping the mail directly to Rails when you have an already running environment up and ready to use via HTTP. – Cheese

Hmmm the reason I want to change the from address is because that is actually the way craigslist does it. It forwards them the email, making it look like I sent it. I assume the only way to do that reliably is through a service like sendgrid? – Portend

You can change the From:, provided you set the Sender: and the Return-Path: to an address you own, but I wouldn't advise it personally. Sendgrid won't be able to achieve anything different to you in terms of what it does with mail headers ;) – Cheese

Answer 2

If your app is going to scale, especailly across multiple servers then I wouldn't recomend the default Rails way of receiving email. Take a look at a blog post I wrote here about some options.

The basic premise is that you want to receive mail on a catch all domain. You can either forward/collect using imap/pop3 from a server like gmail or use a service like CloudMailin to take care of delivering the message to your app. You can give each user a unique to address or even just use the disposable part of the message such as [email protected].

Then it's just a case of using the mail gem to inspect the message and add any headers that you need to and send the message on again. Again you can use your own email server to do this delivery or rely on a service like Amazon's Simple Email Service if you want to improve your deliverability.

Squarerigger

Thanks for the reply. I hadn't seen Amazon's Simple Email Service. Will that give me control over the from: header, so that it looks like it's sent from the person who originally sent the email? – Portend

I agree with what you mention in your blog post... the way the Rails documentation suggest you do it (via a pipe, which is going to incur the phenomenal environment initialization overhead), is less than ideal. You definitely want to aim to POST this into your app instead. – Cheese

Good question, Thariq. I haven't used Amazon's SES (yet), but I think I read that it does let you send from an address of your choosing. (Setting the From: header is the easy part--it's the DKIM, etc. that you need to prevent clients from believing your message is phishing and the From: header forged). The catch is that you have to manually verify each sender... which is no problem if you control the sender address(es) but a problem if you want to let any of your app's users be the sender through Amazon SES. – Matadi