How do you test an Android application across multiple Activities?

Question

We are building a complex Android application consisting of many screens and workflows spread across many Activities. Our workflows are similar to what you might see on a Bank's ATM machine, for example, there is an Activity to login in that transitions to a main menu Activity which can transition to other activities based on the user's choices.

Since we have so many workflows we need to create automated tests that span multiple activities so we can test a workflow from end to end. For example, using the ATM example, we would want to enter a valid PIN, verify that sends us to the main menu, choose withdraw cash, verify that we are on the withdraw cash screen, etc., etc., and eventually find ourselves back on the main menu or "logged" out.

We've toyed with the test APIs that come with Android (e.g. ActivityInstrumentationTestCase2) and also with Positron, but neither seem capable of testing beyond the bounds of a single Activity, and while we can find some utility in these tools for some unit testing, they won't meet our needs for testing scenarios that cut across multiple Activities.

We are open to an xUnit framework, scripting, GUI recorders/playbacks, etc. and would appreciate any advice.

Marcellusmarcelo

As of Android 4.1, there is now a new testing framework from Android which enables testing across activities and indeed the entire system: developer.android.com/tools/testing/testing_ui.html – Rotterdam

Robotium will suit this need too and in only a couple of lines. – Polled

Answer 1

I feel a bit awkward about answering my own bounty question, but here it is...

I've searched high and low on this and can't believe there is no answer published anywhere. I have come very close. I can definitely run tests that span activities now, but my implementation seems to have some timing issues where the tests don't always pass reliably. This is the only example that I know of that tests across multiple activities successfully. Hopefully my extraction and anonymizing of it did not introduce errors. This is a simplistic test where I type a username and password into a login activity, and then observe a proper welcome message is shown on a different "welcome" activity:

package com.mycompany;

import android.app.*;
import android.content.*;
import android.test.*;
import android.test.suitebuilder.annotation.*;
import android.util.*;
import android.view.*;
import android.widget.*;

import static org.hamcrest.core.Is.*;
import static org.hamcrest.core.IsNull.*;
import static org.hamcrest.core.IsInstanceOf.instanceOf;
import static org.junit.Assert.*;
import static com.mycompany.R.id.*;

public class LoginTests extends InstrumentationTestCase {

   @MediumTest
   public void testAValidUserCanLogIn() {

      Instrumentation instrumentation = getInstrumentation();

      // Register we are interested in the authentication activiry...
      Instrumentation.ActivityMonitor monitor = instrumentation.addMonitor(AuthenticateActivity.class.getName(), null, false);

      // Start the authentication activity as the first activity...
      Intent intent = new Intent(Intent.ACTION_MAIN);
      intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
      intent.setClassName(instrumentation.getTargetContext(), AuthenticateActivity.class.getName());
      instrumentation.startActivitySync(intent);

      // Wait for it to start...
      Activity currentActivity = getInstrumentation().waitForMonitorWithTimeout(monitor, 5);
      assertThat(currentActivity, is(notNullValue()));

      // Type into the username field...
      View currentView = currentActivity.findViewById(username_field);
      assertThat(currentView, is(notNullValue()));
      assertThat(currentView, instanceOf(EditText.class));
      TouchUtils.clickView(this, currentView);
      instrumentation.sendStringSync("MyUsername");

      // Type into the password field...
      currentView = currentActivity.findViewById(password_field);
      assertThat(currentView, is(notNullValue()));
      assertThat(currentView, instanceOf(EditText.class));
      TouchUtils.clickView(this, currentView);
      instrumentation.sendStringSync("MyPassword");

      // Register we are interested in the welcome activity...
      // this has to be done before we do something that will send us to that
      // activity...
      instrumentation.removeMonitor(monitor);
      monitor = instrumentation.addMonitor(WelcomeActivity.class.getName(), null, false);

      // Click the login button...
      currentView = currentActivity.findViewById(login_button;
      assertThat(currentView, is(notNullValue()));
      assertThat(currentView, instanceOf(Button.class));
      TouchUtils.clickView(this, currentView);

      // Wait for the welcome page to start...
      currentActivity = getInstrumentation().waitForMonitorWithTimeout(monitor, 5);
      assertThat(currentActivity, is(notNullValue()));

      // Make sure we are logged in...
      currentView = currentActivity.findViewById(welcome_message);
      assertThat(currentView, is(notNullValue()));
      assertThat(currentView, instanceOf(TextView.class));
      assertThat(((TextView)currentView).getText().toString(), is("Welcome, MyUsername!"));
   }
}

This code is obviously not very readable. I have actually extracted it into a simple library with an English-like API so I can just say things like this:

type("myUsername").intoThe(username_field);
click(login_button);

I've tested to a depth of about 4 activities and am satisfied that the approach works though as I said, there appears to be an occasional timing issue I have not completely figured out. I am still interested in hearing of any other ways of testing across activities.

Marcellusmarcelo

You might try adding the FlakyTest annotation to repeat the test automatically when timing issues cause it to fail. Not a solution, really, but a viable workaround in some situations. – Nolly

Thanks for writing this up! I was looking for something with the functionality of ActivityMonitors for my testing. I just couldn't find them. – Unhandled

As far as I know, nothing you did above cannot be done using ActivityInstrumentationTestCase2 – Albescent

any idea, on what condition, 'getInstrumentation().waitForIdleSync();' would get into infinite loop? In android 4.4.2_r2 running processor board, I'm facing this issue, while executing the CTS Test. – Yesseniayester

I think my son @pajato1 found and fixed your timing problem. His fix solved my problem. Here's what he said: "I just noticed in the javadoc that Instrumentation.startActivitySync() was blocking until the new Activity was ready and then returning it, so it seems like the Monitor wasn't necessary. Removing it proved this was correct. My theory is that Monitor was causing the Activity created by startActivitySync() to be restarted in some cases due to a race condition. I did spend some time reading the android source code, but nothing jumped out at me as the cause of the race condition." – Hufuf

Answer 2

Take a look at Robotium
'a open-source test framework created to make automatic black-box testing of Android applications significantly faster and easier than what is possible with Android instrumentation tests out-of-the-box.'

Homepage: http://www.robotium.org/
Source: http://github.com/jayway/robotium

Please note that the Robotium project is maintained by the company I work for

Mousebird

hi, Is there a recorder tool for this ? i checked many websites and found testdroid which records the scripts and runs it. Unfortunately it is not a freeware, do you know any freeware which do the recording process ? – Gripsack

@thndrkiss: I don't know of any such tool. If you place a questions on the Robotium forum you are likely to get a better answer. – Escolar

Robotium is a life saver. It will make your test extremely easy to write (you're basically talking to it in plain english: click on this, press back button, etc.) You can test anything but you are not required to know the tiny details. It has at least two major benefits: you can test apps you don't have the source code of, and it relies on the UI which make it very robust (you change your controllers / models much more than your views...) – Amentia

Answer 3

You could always use Robotium. It supports blackbox testing just like Selenium but for Android. You will find it at Robotium.org

Endora

Last time I checked Robotium could not be used across Activities. Has that been fixed now ? #3840534 – Lists

It has always worked across Activities as long as they belong to the same process. – Endora

Answer 4

I'm surprised no one has mentioned some of the leading automated functional testing tools. Compared with Robotium, these don't require writing Java code.

MonkeyTalk: an open-source tool backed by the company Gorilla Logic. Pros: provides recording as well as a higher-level scripting language easier for non-technical users, and is cross-platform (includes iOS). Given those benefits as requirements, we've found this to be the best solution. It also allows customization beyond what can be done in their scripting language using Javascript.

Calabash-Android: an open-source tool for Cucumber-style features. Pros: write features in the Gherkin language which is Business Readable, Domain Specific Language that lets you describe software’s behavior without detailing how that behavior is implemented. Similar but not exact support is available for iOS in cucumber-ios. Recording capabilities are not as good, as they produce a binary output.

A couple of other references:

• Here are some additional comparisons between Robotium, Monkeytalk and Calabash. It mentions TestDroid as another possibility.
• This blog mentions the above plus NativeDriver and Bot-bot.

Charcoal

Answer 5

I created a record-and-playback tool for Android and made it available on GitHub. It's easy to configure and use, requires no programming, runs against real devices (which do not have to be rooted) and automatically saves screenshots as it plays tests.

Scapular

This looks promising. For those who don't see the point: this seems to be a pretty good solution to test gestures (tapping, dragging and other things) – Amentia

Answer 6

First of all, use 'ActivityInstrumentationTestCase2', not 'InstrumentationTestCase', as your base class. I use Robotium and routinely test across multiple Activities. I found that I have to specify the login activity as the generic type (and class argument to the constructor).

The 'ActivityInstrumentationTestCase2' constructor ignores the package argument and does not require it. The constructor that takes the package is deprecated.

From the Javadocs: "ActivityInstrumentationTestCase2(String pkg, Class activityClass) This constructor is deprecated. use ActivityInstrumentationTestCase2(Class) instead"

Using the recommended base class allows the framework to handle certain boilerplate, like starting your activity. That's done by the call to 'getActivity()', if necessary.

Gilliangilliard

Answer 7

Found this useful with a couple of modifications. Firstly getInstrumentation().waitForIdleSync() will cure the flakiness SingleShot speaks of and also InstrumentationTestCase has a lauchActivity function that can replace the start activity lines.

Blackout

Answer 8

you can do it like this to avoid the flake waiting times out of sync :

final Button btnLogin = (Button) getActivity().findViewById(R.id.button);
Instrumentation instrumentation = getInstrumentation();

// Register we are interested in the authentication activity...
Instrumentation.ActivityMonitor aMonitor = 
        instrumentation.addMonitor(mynextActivity.class.getName(), null, false);

getInstrumentation().runOnMainSync(new Runnable() {
         public void run() {
             btnLogin.performClick();
         }
     });

getInstrumentation().waitForIdleSync();

//check if we got at least one hit on the new activity
assertTrue(getInstrumentation().checkMonitorHit(aMonitor, 1)); 

Heelpost

Answer 9

I'm working on pretty much the same thing, and I'll probably go with a variation on the accepted answer to this question, but I did come across Calculuon (gitHub) during my searches for a solution.

Unhandled

Answer 10

I haven't personally used it, but ApplicationTestCase looks like it might be what you're looking for.

Siderostat

Unfortunately there are no examples indicating that to be the case. – Marcellusmarcelo

Yea, looks like you're right...was tricked by the name. I can't figure this one out. Best approach I've got so far is to use positron's ActivityUnitTestCase to verify that the next activity is started, but that doesn't help you build coherent stories. Alternatively, InstrumentationTestCase.launchActivity might allow you to start an arbitrary number of activities, but I'm still trying to figure out the Instrumentation stuff. – Siderostat

Answer 11

There is another way to do the multiple activity using ActivityInstrumentation Class.. Its a normal automation scenario... First get the focus of what ever object you want and then send a key Simple as that sample code

button.requestFocus();
sendKeys(KeyEvent.KEYCODE_ENTER);

Only thing is understanding the every API calls will help us.

Centromere

Answer 12

Will accepted approach work with different Activities from different Applications, signed by different certificates? If not, Robotium the best way to test activities within same application.

Apanage

Answer 13

This answer is based on the accepted answer but modified to solve the timing issue which for me became consistent after adding about a half dozen tests. @pajato1 gets the credit for solving the timing issue, as cited in the accepted answer comments.

/**
 * Creates a test Activity for a given fully qualified test class name.
 *
 * @param fullyQualifiedClassName The fully qualified name of test activity class.
 *
 * @return The test activity object or null if it could not be located.
 */
protected AbstractTestActivity getTestActivity(final String fullyQualifiedClassName) {
    AbstractTestActivity result = null;

    // Register our interest in the given activity and start it.
    Log.d(TAG, String.format("Running test (%s) with main class: %s.", getName(), fullyQualifiedClassName));
    instrumentation = getInstrumentation();

    Intent intent = new Intent(Intent.ACTION_MAIN);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
    intent.setClassName(instrumentation.getTargetContext(), fullyQualifiedClassName);
    // Wait for the activity to finish starting
    Activity activity = instrumentation.startActivitySync(intent);

    // Perform basic sanity checks.
    assertTrue("The activity is null!  Aborting.", activity != null);
    String format = "The test activity is of the wrong type (%s).";
    assertTrue(String.format(format, activity.getClass().getName()), activity.getClass().getName().equals(fullyQualifiedClassName));
    result = (AbstractTestActivity) activity;

    return result;
}

Hufuf

Answer 14

Try the Monkey tool testing

Step 1:

open the android studio terminal(Tools-> open terminal)

Step 2:

In order to use monkey , open up a command prompt and just naviagte to the following directory.

 export PATH=$PATH:/home/adt-bundle-linux-x86-20140702/sdk/platform-tools

Step 3:

add this monkey command into terminal and press enter..

see the magic in your emulator.

adb shell monkey -p com.example.yourpackage -v 500

500- it is the frequency count or the number of events to be sent for testing.

you can change this count..

More reference,

http://www.tutorialspoint.com/android/android_testing.htm

http://androidtesting.blogspot.in/2012/04/android-testing-with-monkey-tool.html

Counterblast

Downvoters must tell the reason for downvoting... this is working code.. and official testing method too. If any mistake I am ready to correct my answer.. – Counterblast