Share data between gatling scenarios

Question

I have a scenario that, with the help of a CSV file containing usernames and passwords, obtains session Ids and saves them using saveAs.

I want to be able to use those session IDs in a following scenario that performs a few actions which need session Ids. In addition, I also would like to correlate the session Ids with their usernames.

So essentially, I am trying to sequentialize the login operations (obtaining session IDs) from the rest of the operations. Is that possible in gatling? If so, how do I pass data between scenarios?

Georgiegeorgina

Answer 1

I realize this question is old, but I came across it while researching a similar problem for my own sake, and thought I would share the solution I reached in case others come across similar issues. My situation was not entirely similar, but the core essence of my problem was passing data between the two scenarios running in parallell, so I hope the answer may have some value for others in the future, even though it technically only answers half of the original question.

The following setUp shows the general idea of how the two scenarios ran together, where the second scenario started with a delay to ensure that data had been generated in scenario1:

setUp(
  scenario1.inject(constantUsersPerSecond(0.5) during (10 minutes)),
  scenario2.inject(nothingFor(3 minutes), constantUsersPerSecond(0.1) during (7 minutes))
).protocols(httpProtocol)

Simply merging the two scenarios would have been possible, but I kept the two Scenarios defined in two separate classes due to the size of them as both scenarios consisted of a long chain of exec-steps, and due to the fact that they would need to run in parallel with different injection profiles. The data necessary in scenario 2 was generated in scenario 1 and stored in its session.

In order to pass data from one scenario to the other I created an object that did absolutely nothing besides holding a single LinkedBlockingDeque item. I settled for this collection-type to hopefully be able to avoid any concurrency issues when running tests with high loads.

import java.util.concurrent.LinkedBlockingDeque

object DequeHolder {
  val DataDeque = new LinkedBlockingDeque[String]()
}

In scenario one, I then saved values to this deque at the end of each successful loop through the scenario:

val saveData = exec{ session =>
  DataDequeHolder.DataDeque.offerLast(session("data").as[String])
  session
}

val scenario1 = scenario("Scenario 1")
.exec(
  step1,
  step2,
  .....
  stepX,
  saveData
)

And finally in Scenario two I created a custom feeder that retrieved the data from the LinkBlockingDeque, and used this feeder as you would any other feeder:

class DataFeeder extends Feeder[String] {
  override def hasNext: Boolean = DataDequeHolder.DataDeque.size() > 0
  override def next(): Map[String, String] = Map("data" -> DataDequeHolder.DataDeque.takeFirst())
}

val scenario2 = scenario("Scenario 2")
.feed(new DataFeeder())
.exec(
  step1,
  step2,
  .....
  stepX,
)

This has so far proved to be a reliable way to pass the data across the two scenarios without running into concurrency issues. It is however worth noting that I have not run this with high loads, as my backend-system runs some very heavy operations, and is not intended to run with thousands of concurrent users. I do not know how this will function as well with systems under high loads.

Pleasant

Using a LinkedBlockingDeque will cause issues if you read faster than you write (it depends on your use case). If so, you can use a ConcurrentLinkedQueue and and have users loop and pause until they can fetch data. – Loosen

True, we are aware of that, though I expect that even if you loop and pause, if you're consistently reading faster than you can write, the users waiting will just keep accumulating, so it's not necessarily entirely reliable. Definitely depends a lot on the usecases indeed. That said, I will look into if a ConcurrentLinkedQueue is a better option for us as well anyway, thank you for the tip! – Maltz

The big difference is that you won't block the Gatling threads and don't risk ending up in a deadlock: users not being able to write in the queue because all the threads are blocked by users trying to read. – Loosen

Why not using a queue message system like RabbitMQ instead? – Joijoice

Answer 2

Based on Kim's answer I've created a Java class that can be used to collect attribute values from the session in one scenario and use the collected data as feeder in another scenario.

public class SessionAttributeCollectorAndFeeder<T> implements UnaryOperator<Session> {

    private final String attributeName;
    private final List<T> data = Collections.synchronizedList(new ArrayList<>());

    public SessionAttributeCollectorAndFeeder(String attributeName) {
        this.attributeName = attributeName;
    }

    /**
     * @see FeederBuilder#random()
     */
    public Supplier<Iterator<Map<String, Object>>> random() {
        return () -> new Iterator<>() {
            @Override
            public boolean hasNext() {
                return true;
            }

            @Override
            public Map<String, Object> next() {
                return Collections.singletonMap(attributeName,
                            data.get(ThreadLocalRandom.current().nextInt(data.size())));
            }
        };
    }

    /**
     * @see FeederBuilder#circular()
     */
    public Supplier<Iterator<Map<String, Object>>> circular() {
        ...
    }

    @Override
    public Session apply(Session session) {
        data.add(session.get(attributeName));
        return session;
    }

}

This class can be used like this:

    SessionAttributeCollectorAndFeeder<String> myCollectorAndFeeder = new SessionAttributeCollectorAndFeeder<>("myKey");

    ScenarioBuilder scenario1 = scenario("Scenario 1") 
                .exec(step1) 
                .exec(myCollectorAndFeeder);

    ScenarioBuilder scenario2 = scenario("Scenario 2") 
                .feed(myCollectorAndFeeder.random()) 
                .exec(step);

    setUp(scenario1.injectOpen(atOnceUsers(10)) 
                .andThen(scenario2.injectOpen(atOnceUsers(10))
                .protocols(...)
                .assertions(global().failedRequests().count().is(0L));

Misguide