Android: Robotium vs android test framework

Question

Everyone using Robotium for GUI testing.

Can you tell me what Android native test framework cannot do that Robotium can do? As I know Robotium can be used as black box testing, so I do not need to know about application resources. What else?

Powys

Answer 1

The difference between Robotium and the native tools is the fact that with Robotium is really simple to write a test. It's basically point and click, from the Solo instance object.

Here you can download the JAR files and an example project to test it by yourself.

UPDATE

As an example, I'm testing an Activity with some Edit Text, a spinner and a pop-up dialog that shows up when I click a spinner option. Note here that with other methods, filling the fields of the pop up dialog is a real pain.

Here is how to declare the test class and Robotium's initialization

import com.jayway.android.robotium.solo.Solo; //Using Robotium

//Robotium uses ActivityInstrumentationTestCase2.
//Note here the use of the template
public class AddNewQuestionTests extends
                ActivityInstrumentationTestCase2<AddNewQuestion> {

    public AddNewQuestionTests(Class<AddNewQuestion> name) {
        super(name);
    }

    public AddNewQuestionTests() {
        super(AddNewQuestion.class);
    }

    private Solo solo;

    protected void setUp() throws Exception {
        super.setUp();
//Initialize Solo with the instrumentation and the activity under test
        solo = new Solo(getInstrumentation(), getActivity());       
    }

And here is the test method:

    public void testHappyPathAddScaleQuestion() {

            // Type question title
            solo.clickOnEditText(0); //find the EditText, and click it

            solo.enterText((EditText) getActivity().findViewById(

//find the EditText, and put some string
                R.id.activity_add_new_question_editText_question_title),
                            "Question title scale ");
            // Type question description
            solo.clickOnEditText(1);
            solo.enterText((EditText) getActivity().findViewById(
                            R.id.activity_add_new_question_editText_question_description),
                            "Question description scale");
            // Type the question
            solo.clickOnEditText(2);
            solo.enterText((EditText) getActivity().findViewById(
                            R.id.activity_add_new_question_editText_question),
                            "Question scale");

            // Click the spinner and then the "Scale" question type
//Press an spinner option
            solo.pressSpinnerItem(0, 4);

//Wait for the popup dialog title to show up. When robotium reads it, continue working          solo.waitForText(getActivity().getResources().getString(R.string.activity_add_new_question_scale_selection_dialog_message)); 
            // Type minimum and maximum ranges
            solo.clickOnEditText(0);
            solo.searchText(getActivity().getResources().getString(R.string.activity_add_new_question_maximum_value_hint));
            solo.clickOnView(solo.getCurrentEditTexts().get(0));
            solo.enterText(0, "34");
            solo.clickOnView(solo.getCurrentEditTexts().get(0));
            solo.enterText(1, "55");

            // Click ok to close the dialog
            solo.clickOnButton(getActivity().getResources().getString(R.string.OK));

            // Click ok to get an ok message
            solo.clickOnButton(getActivity().getResources().getString(R.string.OK));

            //Wait for the ok toast message
            boolean flagOKDatabase=solo.waitForText(getActivity().getResources().getString(R.string.database_success_storing_data),1,120);
            assertEquals("Something wrong happened with the database", true, flagOKDatabase);
        }

Rockery

Answer 2

Robotium

pros:

• support hybrid applications with webViews.
• you can bound your test to unique IDs of tested application whenever with UIatomator to write complicated test cases will need great amount of work from you
• in UIatomator is used UiAutomatorTestCase Instrumentation which doesn't give you possibilities to call current activity and check that loaded appropriate one, you can't call connectivity- or audio- managers for wi-fi or sound tests. In Robotium you can easily call such gems and that is greatly improve efficiency of your tests
• Open source project, so you can modify tool to your needs

cons:

• No possibility to tests multi-process application where android:process tag used for different activities.

example code:

UIautomator

pros:

• Test case is independent from the process at which tested application works. So it can be used in places where are used additional libraries or activity is run in other process also useful if for test needed switching between several applications.

cons:

• Works only on Android version 4.1 or more!

• You can't use source IDs when you get ui-object instances. That mean if an application structure changed on one layout you need to refactor your test (this problem can be solved by using tags for each ui-element)

• You can't get current activity or Instrumentation. That mean you are limited in development of your tests and not used many of android's api methods for your tests.

• Hard to debug, you need to have script for buiding and starting your test fast and see output

hierarchy viewer:

I think both these tools are good ones, definitely possibility to switch between applications during test is awesome. You should select tool depending from your needs. I suggest Robotium for tests where you don't need to switch between applications because it have simple methods and opportunity to use Android API for writing flexible and smart tests in short code which can cover even testing of a web page inside webview and milti-process applications are very unusual.

Voodoo

Answer 3

There is no downside to using robotium over the instrumentation framework native to android. That is because when using robotium you can still do everything you could of done without it but you also have access to lots of helpful functions already.

There are other android automation frameworks out there though that are definitely worth looking at. If you have any code in a web view these are especially helpful as this is where robotium really lets itself down.

https://github.com/calabash/calabash-android

https://github.com/calabash-driver/calabash-driver

http://testdroid.com/

Procumbent