Emails are put into spam - wrong headers problem

Question

this is my first question on StackOverflow, but I think that we'll both come to the happy end. :) The problem is: I've got newsletter script written in PHP and when I send those emails to the various accounts they are put in the spam folder. Here is what I get in mail headers:

X-Spam-status: Yes, score=5.01 tagged_above=1 required=4.5 
tests=[HTML_FONT_SIZE_LARGE=0.001, HTML_MESSAGE=0.001, 
HTML_TAG_BALANCE_BODY=0.712, MIME_HTML_ONLY=1.105, MISSING_DATE=1.396, 
MISSING_MID=0.14, RCVD_IN_BRBL_LASTEXT=1.644,TO_EQ_FM_DOM_HTML_ONLY=0.001, 
T_FRT_CONTACT=0.01] autolearn=no

And here are my questions:

1. What is that and how to fix status RCVD_IN_BRBL_LASTEXT? I haven't found anything informative googling that phrase.

2. How to fix MISSING_DATE status? I've put in the code generating whole email:

   $mailHeaders[] = "Date: ".date('Y-m-d H:i:s', time());

but with no success. "Date" fields comes only as H:i (13:45, for example) - Thunderbird 3.1.3 FYI. Searching on SO didn't help me. I have also tried adding Delivery-Date status - still nothing.

Reputation

You should have a look at: codinghorror.com/blog/2010/04/…. There's a pretty detailed explanation how to go about sending mail. – Thusly

I am still having problems with MISSING_DATE and run out of ideas. – Reputation

Answer 1

RCVD_IN_BRBL_LASTEXT indicates that your email has been flagged by the Baracuda RBL, which is a service which tracks IP addresses that have been known to send spam.

Possibly your newsletter has been flagged up as spam in the past? or possibly its the ISP you're using to route your email which has been responsible for some spam. Either way, this particular point isn't an issue with your mail headers.

The missing date is the important one which will bring your points below the threshold.

The date format you need looks like this: Date: Wed, 15 SEP 2010 14:12:27 +0100

Most of that is self explanatory (the last bit is the time zone), and it looks like you know your way around the PHP date function, so hopefully that should sort you out. But I found this page helpful as a walk-through of a legitimate email header format.

Charles

Thanks for BRBL, I'll talk with my colleagues about that. As for date format, I see that it is RFC 2822 date format referred as PHP's date('r'); which also hasn't worked for me. I'll try it again now - thanks! – Reputation

The 'r' format is a relatively recent addition to the date() function; check that your PHP version supports it. – Charles

I am sending email with: Date: Thu, 16 Sep 2010 15:13:26 +0200 (CEST) but still is arriving as spam. I think that there may be issue with our company server, we will check it. Still thanks for your input. Accepted, sir! ;] – Reputation

Answer 2

The challenge with bulk email sending is that there's so many different factors that could throw you off and get you blocked as spam. Headers tell you what's going on, but in the grand scheme of things they're not one of the biggest challenges.

My company sends 50000+ emails per week, sometimes that many per day. Here's what we've learned:

1) If your server hasn't established "reputation" with email hosts, you're more likely to get flagged. There's no great way to establish it, though sites like Socket Labs simply throttle down new clients in the beginning and after 60-90 days release that throttle to allow more email to go through. As many emails as Socket Labs processes, it tells me it's a valid practice.

1a) Monitor the RBL list to ensure you're not on it. If you do get flagged (happens to just about everyone at some time or another) aggressively work to get yourself off ASAP. Contact the RBL in question and work with them to quickly right the situation.

2) The "big guys" including Gmail, Yahoo, AOL, and MSN are sensitive to being rapidly hit by the same host in succession. My company has chosen to overcome this by keeping track of who our email processes are sending to via a "log" If the next email has the same domain as the previous sent, we wait a period of time. If not, we fire at will. It prevents our system from sending more than 1 email per X seconds to the same host, and has meant our emails are getting through at a very high rate.

3) AOL mail is borderline worthless. I saw a stat once that someone had proven something like 20% of email sent to AOL just "disappears" I'm not sure if it's that high, but I know we have nothing but problems with getting AOL email through...it's the nature of the beast. The good news is that AOL is on its way out, so we shouldn't have to deal with it on this level too much longer.

4) The obvious step is to ensure that you're doing the best you can to stay CAN-Spam compliant. Include a real-time opt-out, company information in the footer, and don't try to deceive with your message.

5) Finally, don't send email to people who haven't requested it. It seems like a silly easy step, but it's abused SO much. You won't be flagged as spam if you send to people who want your email...it's that easy. If you get a bounce, process it out of your list immediately so that you're not trying to resend to a bad account.

Good luck.

Novelist

The company I'm working in sends newsletter to 30k+ subscribers weekly. Fortunately global mail hosters are quite lazy about spam score (7+), but not our company server (4.5). I am trying to solve problem with our server spam score threshold, because I am trying to ensure thateveryone gets mail in "inbox", and not in "spam". – Reputation

Hate to be negative, but the best you're going to be able to do is lessen the amount that goes to "spam" Spam Filtering is customizeable after all. All it takes is one overzealous IT person to lock down the spam blocker tool they use and voila! Spam. We've even had cases where people got legit emails and were too lazy to unsubscribe, so they hit the "spam" button and we were forced to go through a lengthy process to explain ourselves. ISP's err on the side of users, not senders. – Novelist

Answer 3

The X-Spam-Status header is being added by a Barracuda spam filter, and what you're seeing is diagnostic information it attaches to explain why it marked the message as spam.

Barracuda is a rules-based engine, and as you guessed, the score you're getting (5.01 in the example above, though you say you fixed the date so your score may be lower) is above the allowed threshold.

Since the highest-weighted signal is the RCVD_IN_BRBL_LASTEXT field, you're likely getting dinged primarily because your IP address is in their Real-time Block List (RBL). It's possible you can find the data source that hates your IP -- check http://www.spamhaus.org/lookup.lasso to see if it's Spamhaus that has tagged you -- and convince them to remove you, but more likely the best path is to use an IP that's already been groomed to have a good reputation. Two providers you may wish to check out are http://sendgrid.com and http://authsmtp.com, both of whom will allow you to proxy your traffic through their servers (assuming your content isn't likely to be voted as spam).

Hope that is helpful.

Rivero