Protractor - run multiple tests in parallel on different browsers

Question

I can't find any information on how to set this up, but it seems like a pretty basic concept, so I'm sure there's an answer out there.

I know how to run protractor on different browsers by setting the browserName property of the capabilities object in the config. And that's working great. I can set it to 'chrome' or 'firefox' or whatever I need and it runs just as expected. But the only way to run a single test suite against multiple browsers (as far as I know) is to create separate config files, each with a different browserName and then run each browser with its own config. This works, but its really slow because tests are then running in sequence, rather than concurrently.

Is there any way to run it on multiple browsers in parallel?

Can it be done on SauceLabs? or even using a local Selenium-Grid?

We are just trying to streamline our testing process and this would be a huge help. Any suggestions or info would be greatly appreciated. Thanks in advance.

Opaline

Do u mean run parallel test – Leclerc

Answer 1

Update Feb 2014 - This answer is no longer valid. Use Paolo Moretti's answer below.

---

There may be a better way to do this but currently I am just executing these as concurrent Grunt tasks.

1) Add the grunt concurrent plugin

npm install grunt-concurrent --save-dev

2) Add a task for each browser under grunt.initConfig. We can add the browser as an arg to re-use our configuration file.

protractor: {
        options: {
            keepAlive: true,
            singleRun: false,
            configFile: "test/protractor.conf.js"
        },
        run_chrome: {
            options: {
                args: {
                    browser: "chrome"
                }
            }
        },
        run_firefox: {
            options: {
                args: {
                    browser: "firefox"
                }
            }
        }
    },

3) Register these as tasks;

grunt.registerTask('protractor-chrome', ['protractor:run_chrome']);
grunt.registerTask('protractor-firefox', ['protractor:run_firefox']);

4) Create your concurrent task under grunt.initConfig

concurrent: {
        protractor_test: ['protractor-chrome', 'protractor-firefox']
    },

5) Add the grunt task for concurrent

grunt.registerTask('protractor-e2e', ['concurrent:protractor_test']);

And executing that should give you concurrent protractor tests.

Pregnancy

This seems to be the best answer until Protractor adds direct support for Selenium Grid. See issue #111 How to pass more configuration to selenium grid? and Running Selenium tests in parallel. – Ticon

Answer 2

There's new option called multiCapabilities for that:

multiCapabilities: [{
  'browserName': 'chrome'
}, {
  'browserName': 'firefox'
}],

Here's a complete example.

Radiobroadcast

Thanks for the update. Do you know of any way to run separate spec files in each of those browsers? It looks like you can only run a single test suite in multiple browsers, but I'd really like to be able to run multiple specs in parallel... – Opaline

If you are using grunt-protractor-runner you can pass specs as a parameter. In that way you could create a task for each spec and then run all these in parallel using grunt-concurrent or something similar. – Pregnancy

I added above code and its not opening Firefox with or without FF driver. Does anyone know why? – Censorship

Answer 3

using multiCapabilities will run all the tests in each of the browsers. So the configuration below will run every test twice, once in Firefox and once in Chrome:

multiCapabilities: [{
  'browserName': 'chrome'
}, {
  'browserName': 'firefox'
}],

If you would rather have each test file just run once, but split up among multiple browsers, then use the splitTestsBetweenCapabilities option:

splitTestsBetweenCapabilities: true

This blog post goes into further detail about the splitTestsBetweenCapabilities

Cusack

That sounds great. What exactly does the splitTestsBetweenCapabilities option do? – Opaline

This option is removed right now; refer github.com/angular/protractor/blob/master/docs/… – Extended

Answer 4

I had the same problem and I've found that they are adding the feature for parallel execution as we speak! :D

Take a look at this: https://github.com/angular/protractor/pull/492

That change was merged to master, but a newer one (512) is still open. As soon as they merge it to master (should be today or tomorrow according to the discussion in the pull request) it should be in your hands :D.

Preceptive

Thanks for pointing that out! That'll be great once that's merged in. – Opaline

Answer 5

BrowserStack also can be used for this purposes. It has quite detailed start guide: https://www.browserstack.com/automate/node, but it is not free

Raffish

Answer 6

This is how I achieve this. Simply add the section below to the conf.js file:

capabilities: {
  browserName: 'chrome',
  shardTestFiles: true,
  maxInstances: 1
}

shardTestFiles = true causes each spec file to run in a new browser instance. maxInstances is the max number of browsers that can be open at the same time.

Iatrogenic

Did you have any problems with intermittent failures when doing this? – Custodian

I ran into a few issues that needed to be fixed in the code. I don't remember any specifics right now. – Iatrogenic