Javamail gmail and OAuth2

Question

I'm trying to use the latest javamail 1.6.0 api in a web app (java 1.8/tomcat8) to send email on behalf of client users of the app. Some clients use gmail. I don't want to ask them to enable access to insecure apps in their google accounts as suggested by the javamail FAQ, and I'm willing to implement oauth2 if that's what it takes.

To do that, I did the following on google developer console:

• created an app
• created oauth2 credentials for the app (clientid, clientsecret)
• granted the app access to gmail api

Then, I implemented the oauth2 flow in my app using the google oauth client

The authorization redirect is constructed:

String url = 
        new GoogleAuthorizationCodeRequestUrl(clientId, 
            redirectUrl, 
            Collections.singleton(GmailScopes.GMAIL_SEND)
            ).setAccessType("offline").build();

That successfully redirects to the google site where I can authenticate and authorize my app to send mail on my behalf. It successfully redirects back to my app after I authorize, and the app processes the authorization code:

GoogleTokenResponse response =
                new GoogleAuthorizationCodeTokenRequest(
                        new NetHttpTransport(), 
                        new JacksonFactory(),
                      clientId, 
                      clientSecret,
                      code, 
                      redirectUrl)
                .execute();

(where code is the returned authorization code) This seems to work, and returns an access token and refresh token. (I can also go back to my google account settings and see that I have authorized the app to send email.

So now I want to try to use the access token to send mail through the javamail api using my gmail username (the one I logged in as to authorize the app), the access token, and the following settings:

host = "smtp.gmail.com";
port = 587;
props.put("mail.smtp.auth", "true");
props.put("mail.smtp.starttls.enable", "true");
props.put("mail.smtp.auth.mechanisms", "XOAUTH2");

The javamail code works fine for other smtp servers. I also turned on debug to trace the smtp flow

DEBUG: JavaMail version 1.6.0
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]
DEBUG SMTP: useEhlo true, useAuth true
DEBUG SMTP: trying to connect to host "smtp.gmail.com", port 587, isSSL false
220 smtp.gmail.com ESMTP c7sm3632131pfg.29 - gsmtp
DEBUG SMTP: connected to host "smtp.gmail.com", port: 587

EHLO 10.0.0.5
250-smtp.gmail.com at your service, [216.165.225.194]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
DEBUG SMTP: Found extension "SIZE", arg "35882577"
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "STARTTLS", arg ""
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "CHUNKING", arg ""
DEBUG SMTP: Found extension "SMTPUTF8", arg ""
STARTTLS
220 2.0.0 Ready to start TLS
EHLO 10.0.0.5
250-smtp.gmail.com at your service, [216.165.225.194]
250-SIZE 35882577
250-8BITMIME
250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
DEBUG SMTP: Found extension "SIZE", arg "35882577"
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "AUTH", arg "LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH"
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "CHUNKING", arg ""
DEBUG SMTP: Found extension "SMTPUTF8", arg ""
DEBUG SMTP: protocolConnect login, host=smtp.gmail.com, user=<[email protected]>, password=<non-null>
DEBUG SMTP: Attempt to authenticate using mechanisms: XOAUTH2
DEBUG SMTP: Using mechanism XOAUTH2
DEBUG SMTP: AUTH XOAUTH2 command trace suppressed
DEBUG SMTP: AUTH XOAUTH2 failed, THROW: 


 javax.mail.AuthenticationFailedException: OAUTH2 asked for more
...
DEBUG SMTP: AUTH XOAUTH2 failed
ERROR 2017-08-06 18:39:57,443  - send: 334 eyJzdGF0dXMiOiI0MDAiLCJzY2hlbWVzIjoiQmVhcmVyIiwic2NvcGUiOiJodHRwczovL21haWwuZ29vZ2xlLmNvbS8ifQ==

decoding that last line I see that the error is a 400 status, that I interpret to mean that the token is not valid.

I also verified that the token is good using the google rest api:

https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=<token>

I also tried using port 465 with ssl enabled, but got the same error.

what am I doing wrong here?

Karole

Did you try to regenerate the token? – Chiou

You're passing the access token, not the refresh token, to the JavaMail API as the password, right? And you're not encoding the token in any way before passing it, right? The token should be less than 80 characters and look something like "ya29.3QEMCT...". You might want to set the mail.debug.auth property to true so that you can see the details of the authentication exchange in the debug output, in case the server is providing more details of the error that are suppressed in the normal debug output. Can you configure Thunderbird to use OAUTH2 with your account? – Walley

Yes, tried using auth token as originally generated, and tried refreshing it. I do have mail.debug.auth enabled. Didn't see any more output from that. – Karole

Ah! My access token starts with "ya29.GlugBAd..." but is 132 char long, and it looks like different instances have slightly different lengths. I'm using the String that comes out of the GoogleResponse directly with no other encoding: String at=response.getAccessToken(); – Karole

...but I also ran that token through the googleapis tokeninfo method, as noted above, and it parsed the token to report that the scope is for gmail.send, it expires in the future, and it is an "offline" token. – Karole

I also just ran a test using thunderbird, and it was able to send mail through my gmail account. I noticed that it's oauth authorization included more scopes than I did in my code. I only requested the GMAIL.SEND scope. – Karole

Answer 1

This is an addition to the answer of user2000974. The documentation of Google about using OAuth to authenticate to an IMAP or SMTP server Gmail > IMAP > OAuth 2.0 Mechanism clearly states the following

This document defines the SASL XOAUTH2 mechanism for use with the IMAP AUTHENTICATE and SMTP AUTH commands. This mechanism allows the use of OAuth 2.0 Access Tokens to authenticate to a user's Gmail account.

The scope for IMAP and SMTP access is https://mail.google.com/.

I hope this will direct people that stumble upon this question in the future to the right documentation page.

Vowell

Thank you for the link! I scoured documentation for three days, but never thought to look under IMAP documentation for info about how to access SMTP. The page describing all google oauth2 scopes that I referenced earlier doesn't mention this.\ – Karole

Thanks worked for me, fine printed warnings can be easily ignored. Besides in one documentation it says you should use as restricted permissions as possible, in my case I only needed send permissions but for SMTP just send was not enough. – Hydrated

Answer 2

I tried playing with the requested scopes, and finally got it to work by requesting full access to the gmail account (scope = "https://mail.google.com/"). The limited documentation of the scopes suggests that the specific scope for sending mail SHOULD work, but apparently it doesn't. Requesting full access to the account works, but seems to defeat the purpose of having limited scopes. It's starting to sound like the SMTP server just isn't respecting the limited scope.

Karole

gmail.send allows sending emails via Gmail API only, for SMTP sending - you need to request full Gmail access - mail.google.com – Largescale

what is Gmail API? why it need different scope from JavaMail? – Backbone

Answer 3

Something important, beware from Gmail API and Gmail IMAP, because they're not the same when this problem occurs. Gmail API allows multiple and granular scopes to specific task, but when using Gmail IMAP it will require always the full scope.

Disparagement

Answer 4

I got "OAUTH2 asked for more" too, although the scopes were set correctly.

My issue was that the accesstoken was issued for user1, but when creating the the session for smtp, i've wrongly put an email for user2.

Took me some hours, maybe this helps someone else to save some time.

Means