Access gmail using imap with accountmanager token

Question

I'm trying to implement a IMAP gmail client using the token received from the Android's AccountManager instead of using username and password.

Google provides this example of IMAP with oauth2 http://code.google.com/p/google-mail-oauth2-tools/source/browse/#svn%2Ftrunk%2Fjava%2Fcom%2Fgoogle%2Fcode%2Fsamples%2Foauth2 http://code.google.com/p/google-mail-oauth2-tools/wiki/JavaSampleCode

public static IMAPStore connectToImap(String host,
                                    int port,
                                    String userEmail,
                                    String oauthToken,
                                    boolean debug) throws Exception {
Properties props = new Properties();
props.put("mail.imaps.sasl.enable", "true");
props.put("mail.imaps.sasl.mechanisms", "XOAUTH2");
props.put(OAuth2SaslClientFactory.OAUTH_TOKEN_PROP, oauthToken);
Session session = Session.getInstance(props);
session.setDebug(debug);

final URLName unusedUrlName = null;
IMAPSSLStore store = new IMAPSSLStore(session, unusedUrlName);
final String emptyPassword = "";
store.connect(host, port, userEmail, emptyPassword);
return store;

}

public static void main(String args[]) throws Exception {
if (args.length != 2) {
  System.err.println(
      "Usage: OAuth2Authenticator <email> <oauthToken>");
  return;
}
String email = args[0];
String oauthToken = args[1];

initialize();

IMAPStore imapStore = connectToImap("imap.gmail.com",
                                    993,
                                    email,
                                    oauthToken,
                                    true);
System.out.println("Successfully authenticated to IMAP.\n");

}

But, when I run the code above, I get an exception of "Empty username or password". Could someone tell me how to access gmail using imap with xoauth2? Thanks.

Update 2013/02/20, below comes debug log

 02-19 17:27:20.098  1905: 1905 I/System.out : setDebug: JavaMail version 1.4.1    
 02-19 17:27:20.098  1905: 1905 I/System.out : mail.imap.fetchsize: 16384    
 02-19 17:27:20.106  1905: 1905 I/System.out : enable SASL    
 02-19 17:27:20.106  1905: 1905 I/System.out : SASL mechanisms allowed: XOAUTH2    
 02-19 17:27:21.340  1905: 1905 I/System.out : * OK Gimap ready for requests from 36.224.98.49 z8if14713202igb.53
 02-19 17:27:21.348  1905: 1905 I/System.out : A0 CAPABILITY
 02-19 17:27:21.598  1905: 1905 I/System.out : * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH AUTH=XOAUTH2
 02-19 17:27:21.598  1905: 1905 I/System.out : A0 OK Thats all she wrote! z8if14713202igb.53
 02-19 17:27:21.614  1905: 1905 I/System.out : IMAP DEBUG: AUTH: XOAUTH    
 02-19 17:27:21.614  1905: 1905 I/System.out : IMAP DEBUG: AUTH: XOAUTH2    
 02-19 17:27:21.614  1905: 1905 I/System.out : DEBUG: protocolConnect login, host=imap.gmail.com, [email protected], password=<non-null>    
 02-19 17:27:21.622  1905: 1905 I/System.out : IMAP SASL DEBUG: Mechanisms: XOAUTH2    
 02-19 17:27:21.817  1905: 1905 I/System.out : IMAP SASL DEBUG: Failed to create SASL client: myjavax.security.sasl.SaslException: Cannot instantiate class com.research.oauth.OAuth2SaslClientFactory [Caused by java.lang.InstantiationException: can't instantiate class com.research.oauth.OAuth2SaslClientFactory]
  02-19 17:27:21.817  1905: 1905 I/System.out : A1 LOGIN [email protected] ""
 02-19 17:27:22.036  1905: 1905 I/System.out : A1 NO Empty username or password. z8if14713202igb.53
 02-19 17:27:22.044  1905: 1905 D/test       : javax.mail.AuthenticationFailedException: Empty username or password. z8if14713202igb.53

I use yor mail.jar and my app Failed to create SASL client: myjavax.security.sasl.SaslException: Cannot instantiate class com.research.oauth.OAuth2SaslClientFactory, then app use Empty password to login gmail. Please help me to figure out the problem, Thank you!

Tavy

Did you manage to get a token and with with scope? – Gerlac

Yes, I get the account token before run the code. – Tavy

What scope did you use and how does the token look like? Can you authenticate 'manually' with it? (by using telnet/openssl, etc. and typing the IMAP commands) – Gerlac

And the token looks like:ya29.AHeSvghvghvyjfgdthcjhvujyjgyfhjmvfxgvhjhbhkbgh........ – Tavy

I use the below code to get token. AccountManager mgr = AccountManager.get(this); Account[] accts = mgr.getAccountsByType("com.google"); Token = mgr.getAuthToken(accts[0], "oauth2:https:"//"mail.google.com/", null, this, new OnTokenAcquired(), null); – Tavy

Answer 1

Did you remeber to change your package name in OAuth2Provider? I forgot it when I was doing tests with that code.

public static final class OAuth2Provider extends Provider {
private static final long serialVersionUID = 1L;

public OAuth2Provider() {
  super("Google OAuth2 Provider", 1.0,
        "Provides the XOAUTH2 SASL Mechanism");
  put("SaslClientFactory.XOAUTH2",
      "com.example.testjavamail.OAuth2SaslClientFactory");
}

}

As I said in another answer, I only tested the connection, but it's working for me.

UPDATE

Here's the code I used, it's basically the example code, what really changed the porting of SASL support in Java Mail.

public class OAuth2Authenticator {
private static final Logger logger = Logger
        .getLogger(OAuth2Authenticator.class.getName());
private static Session mSession;

public static final class OAuth2Provider extends Provider {
    private static final long serialVersionUID = 1L;

    public OAuth2Provider() {
        super("Google OAuth2 Provider", 1.0,
                "Provides the XOAUTH2 SASL Mechanism");
        put("SaslClientFactory.XOAUTH2",
                "com.example.testjavamail.OAuth2SaslClientFactory");
    }
}

public static void initialize() {
    Security.addProvider(new OAuth2Provider());
}

public static IMAPStore connectToImap(String host, int port,
        String userEmail, String oauthToken, boolean debug)
        throws Exception {
    Properties props = new Properties();
    props.put("mail.imaps.sasl.enable", "true");
    props.put("mail.imaps.sasl.mechanisms", "XOAUTH2");
    props.put(OAuth2SaslClientFactory.OAUTH_TOKEN_PROP, oauthToken);
    Session session = Session.getInstance(props);
    session.setDebug(debug);

    final URLName unusedUrlName = null;
    IMAPSSLStore store = new IMAPSSLStore(session, unusedUrlName);
    final String emptyPassword = "";
    store.connect(host, port, userEmail, emptyPassword);
    return store;
}

public static SMTPTransport connectToSmtp(String host, int port,
        String userEmail, String oauthToken, boolean debug)
        throws Exception {
    Properties props = new Properties();
    props.put("mail.smtp.starttls.enable", "true");
    props.put("mail.smtp.starttls.required", "true");
    props.put("mail.smtp.sasl.enable", "true");
    props.put("mail.smtp.sasl.mechanisms", "XOAUTH2");
    props.put(OAuth2SaslClientFactory.OAUTH_TOKEN_PROP, oauthToken);
    mSession = Session.getInstance(props);
    mSession.setDebug(debug);

    final URLName unusedUrlName = null;
    SMTPTransport transport = new SMTPTransport(mSession, unusedUrlName);
    // If the password is non-null, SMTP tries to do AUTH LOGIN.
    final String emptyPassword = null;
    transport.connect(host, port, userEmail, emptyPassword);

    return transport;
}

public synchronized void testImap(String user, String oauthToken) {
    try {

        initialize();


        IMAPStore imapStore = connectToImap("imap.gmail.com", 993, user,
                oauthToken, true);

    } catch (Exception e) {
        Log.d("test", e.toString());
    }

}

public class ByteArrayDataSource implements DataSource {
    private byte[] data;
    private String type;

    public ByteArrayDataSource(byte[] data, String type) {
        super();
        this.data = data;
        this.type = type;
    }

    public ByteArrayDataSource(byte[] data) {
        super();
        this.data = data;
    }

    public void setType(String type) {
        this.type = type;
    }

    public String getContentType() {
        if (type == null)
            return "application/octet-stream";
        else
            return type;
    }

    public InputStream getInputStream() throws IOException {
        return new ByteArrayInputStream(data);
    }

    public String getName() {
        return "ByteArrayDataSource";
    }

    public OutputStream getOutputStream() throws IOException {
        throw new IOException("Not Supported");
    }
}

}

And here's the debug from Java Mail. Btw, post your debug log, it should help in undertanding what's going wrong

02-06 10:18:11.805: I/System.out(7434): DEBUG: setDebug: JavaMail version 1.4.1
02-06 10:18:11.905: I/System.out(7434): DEBUG: mail.imap.fetchsize: 16384
02-06 10:18:12.025: I/System.out(7434): DEBUG: enable SASL
02-06 10:18:12.040: I/System.out(7434): DEBUG: SASL mechanisms allowed: XOAUTH2
02-06 10:18:12.600: I/System.out(7434): * OK Gimap ready for requests from 2.233.xxx.xxx  2if1471965eej.3
02-06 10:18:12.605: I/System.out(7434): A0 CAPABILITY
02-06 10:18:12.635: I/System.out(7434): * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH AUTH=XOAUTH2
02-06 10:18:12.635: I/System.out(7434): A0 OK Thats all she wrote! 2if1471965eej.3
02-06 10:18:12.645: I/System.out(7434): IMAP DEBUG: AUTH: XOAUTH
02-06 10:18:12.645: I/System.out(7434): IMAP DEBUG: AUTH: XOAUTH2
02-06 10:18:12.645: I/System.out(7434): DEBUG: protocolConnect login, host=imap.gmail.com, [email protected], password=<non-null>
02-06 10:18:12.650: I/System.out(7434): IMAP SASL DEBUG: Mechanisms: XOAUTH2
02-06 10:18:12.695: I/System.out(7434): IMAP SASL DEBUG: SASL client XOAUTH2
02-06 10:18:12.695: I/System.out(7434): A1 AUTHENTICATE XOAUTH2
02-06 10:18:12.720: I/System.out(7434): + 
02-06 10:18:12.720: I/System.out(7434): IMAP SASL DEBUG: challenge:  :
02-06 10:18:12.730: I/System.out(7434): IMAP SASL DEBUG: callback length: 1
02-06 10:18:12.730: I/System.out(7434): IMAP SASL DEBUG: callback 0: myjavax.security.auth.callback.NameCallback@41760f78
02-06 10:18:12.730: I/System.out(7434): IMAP SASL DEBUG: response: [email protected]=Bearer ya29.... :
02-06 10:18:12.735: I/System.out(7434): dXNlcj1hbGVhbGVtYXp6b3R0aUBnbWFpbC5jb20BYXV0aD1CZWFyZXIgeWEyOS5BSEVTNlpRYklPeU8xU09sR01WSEo3X2tqVzlVdzNYY1RvODBtQ0hyWFVacjRsYlhIdwEB
02-06 10:18:12.870: I/System.out(7434): * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE
02-06 10:18:12.870: I/System.out(7434): A1 OK [email protected] My NAME authenticated (Success)
02-06 10:18:12.870: I/System.out(7434): A2 CAPABILITY
02-06 10:18:13.160: I/System.out(7434): * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE
02-06 10:18:13.160: I/System.out(7434): A2 OK Success

Thistledown

Yes, I changed my package name in OAuth2Provider. No matter I change it or not, I always get the exception of "Empty username or password" during the code "store.connect(host, port, userEmail, emptyPassword);". If you had tested the connection in another way, could you post the total code or mail me the project? Thanks. – Tavy

Thanks for reply, and I update the debug logs in my question content. My problem is app Failed to create SASL client: myjavax.security.sasl.SaslException, please help me to figure out the problem or may you email me the test project that you used, Thank you! – Tavy

btw, I used 4 permissions, did I need another permission? <uses-permission android:name="android.permission.ACCOUNT_MANAGER" /> <uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.GET_ACCOUNTS" /> <uses-permission android:name="android.permission.USE_CREDENTIALS" /> – Tavy

Here there's the test project I used, it's horrible to see but it works. Again, it only authenticated to imap. Inside there are the javamail jars. They are an earlier version of what I linked before and work for sure in this project. Maybe the other jar had some problem. dl.dropbox.com/u/3092259/TestJavaMail.zip – Thistledown

Thanks @alex! I was able to piece together a solution from the various responses you've had on the topic, and using your sample project. Really A+++ work! – Garganey

I searched for so long for this, and the link provided by Alex in the comments worked absolutely perfectly. I am forever grateful. – Safeguard

@unluddite! Can you share your working example, please? – Emptor

@Thistledown : i integrate your application but can't send the email to another email address:( please help me.... – Quizmaster

Following provided code examples, is there a way to change token value WITHOUT creating a new Store object? I do not want to create a new Store when token expires. Thanks! – Vacuole

Answer 2

What I did to resolve the same problem to recompile the source of java mail package, replace the calling of javax.security to asmack in the IMAPSaslAuthenticator file. the IMAP connection can go through,

11-29 15:51:46.921: D/11.28(2759):  --> the token is ya29.1.
11-29 15:51:46.937: I/System.out(2759): DEBUG: setDebug: JavaMail version ${mail.version}
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: mail.imap.fetchsize: 16384
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: mail.imap.appendbuffersize: -1
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: mail.imap.minidletime: 10
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: enable SASL
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: SASL mechanisms allowed: XOAUTH2
11-29 15:51:46.937: I/System.out(2759): DEBUG IMAPS: trying to connect to host "imap.gmail.com", port 993, isSSL true
11-29 15:51:47.085: D/dalvikvm(2759): GC_CONCURRENT freed 410K, 53% free 8847K/18503K, paused 3ms+6ms, total 38ms
11-29 15:51:47.085: I/System.out(2759): A4 LOGOUT
11-29 15:51:47.093: I/System.out(2759): DEBUG IMAPS: IMAPStore connection dead
11-29 15:51:47.093: I/System.out(2759): DEBUG IMAPS: IMAPStore cleanup, force true
11-29 15:51:47.093: I/System.out(2759): DEBUG IMAPS: IMAPStore cleanup done
11-29 15:51:47.093: D/AbsListView(2759): [unregisterDoubleTapMotionListener]
11-29 15:51:47.093: I/MotionRecognitionManager(2759):   .unregisterListener : / listener count = 0->0, listener=android.widget.AbsListView$4@41d9a1b0
11-29 15:51:47.093: I/System.out(2759): A4 LOGOUT
11-29 15:51:47.132: I/System.out(2759): * BYE LOGOUT Requested
11-29 15:51:47.140: I/System.out(2759): A4 OK 73 good day (Success)
11-29 15:51:47.140: I/System.out(2759): DEBUG IMAPS: IMAPStore connection dead
11-29 15:51:47.140: I/System.out(2759): DEBUG IMAPS: IMAPStore cleanup, force false
11-29 15:51:47.140: I/System.out(2759): DEBUG IMAPS: IMAPStore cleanup done
11-29 15:51:47.140: D/AbsListView(2759): [unregisterDoubleTapMotionListener]
11-29 15:51:47.140: I/MotionRecognitionManager(2759):   .unregisterListener : / listener count = 0->0, listener=android.widget.AbsListView$4@41de2ba0
11-29 15:51:47.171: I/System.out(2759): * OK Gimap ready for requests from 216.16.246.195 q6if13300240veb.40
11-29 15:51:47.171: I/System.out(2759): A0 CAPABILITY
11-29 15:51:47.218: I/System.out(2759): * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN
11-29 15:51:47.218: I/System.out(2759): A0 OK Thats all she wrote! q6if13300240veb.40
11-29 15:51:47.218: I/System.out(2759): DEBUG IMAPS: AUTH: XOAUTH
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: AUTH: XOAUTH2
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: AUTH: PLAIN
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: AUTH: PLAIN-CLIENTTOKEN
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: protocolConnect login, host=imap.gmail.com, [email protected], password=<non-null>
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: SASL authentication command trace suppressed
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: SASL Mechanisms:
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS:  XOAUTH2
11-29 15:51:47.226: I/System.out(2759): DEBUG IMAPS: 
11-29 15:51:47.233: I/System.out(2759): DEBUG IMAPS: SASL client XOAUTH2
11-29 15:51:47.272: I/System.out(2759): DEBUG IMAPS: SASL challenge:  :
11-29 15:51:47.272: I/System.out(2759): DEBUG IMAPS: SASL callback length: 1
11-29 15:51:47.272: I/System.out(2759): DEBUG IMAPS: SASL callback 0: org.apache.harmony.javax.security.auth.callback.NameCallback@41e082c8
11-29 15:51:47.272: I/System.out(2759): DEBUG IMAPS: SASL response: [email protected]=Bearer ya29.1...UhhyFDen9bUwcd5I :
11-29 15:51:48.101: I/System.out(2759): DEBUG IMAPS: SASL authentication succeeded
11-29 15:51:48.101: I/System.out(2759): A2 CAPABILITY
11-29 15:51:48.272: I/System.out(2759): * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH
11-29 15:51:48.272: I/System.out(2759): A2 OK Success
11-29 15:51:48.280: I/System.out(2759): A3 NAMESPACE
11-29 15:51:48.452: I/System.out(2759): * NAMESPACE (("" "/")) NIL NIL
11-29 15:51:48.452: I/System.out(2759): A3 OK Success

I downloaded the packages of

1. javamail-1.5.0-src
2. asmack-android-8-0.8.9.jar
3. activation.jar

I did not use the packages of

1. mail.jar
2. additionnal.jar

Ballou

Answer 3

This works for me if I run the testcode in main method, but deploying this on a glassfish server I got other problems. Following the code from http://code.google.com/p/google-mail-oauth2-tools/

This exception below is triggered then and I start to think this is related to a bug or other problems but cannot find out why. Is the Callback needed some ULR in Google API Console? Or why does it work locally but not when deployed at glassfish. java.lang.IllegalStateException: WEB9031: WebappClassLoader unable to load resource [com.google.code.samples.oauth2.OAuth2SaslClientFactory], because it has not yet been started, or was already stopped

Also looked if this bug was related to this: https://groups.google.com/forum/#!msg/google-appengine/OK3FuIPuA1I/dgov_VRffVgJ

Chambliss