Locking the test data when tests are running in parallel

Need for Data Locking#

When tests run in parallel - it may so happen that they access the same test data. This could cause unexpected behavior in tests and lead to failures.

For instance, two tests signing in with the same credentials - while one test added an item to the cart, another test could empty the cart - so the first test is going to fail for sure as it did not find the item added in the cart.

Ekam provides a mechanism to lock the test data once it has been supplied to the test. Once the test execution is complete the lock on test data would be released, so that another test could consume the test data.

Let us add 2 more secondary addresses to uat/address.json

{
  "default_address": [
    {
      "firstName": "Puja",
      "city": "Delhi",
      "pinCode": "22002"
    }
  ],
  "secondary_address": [
    {
      "firstName": "Puja",
      "city": "Chennai",
      "pinCode": "44004"
    },
    {
      "firstName": "Puja",
      "city": "Hyderabad",
      "pinCode": "55005"
    },
    {
      "firstName": "Puja",
      "city": "Bangalore",
      "pinCode": "66006"
    }
  ]
}

Now we have 3 secondary addresses - cities being Chennai, Hyderabad & Bangalore.

Add tests#

Let us add 3 tests that fetch secondary addresses

package ekam.example.web;

import com.google.inject.Inject;
import com.testvagrant.ekam.testBases.testng.WebTest;
import dataclients.address.Address;
import dataclients.address.AddressDataClient;
import org.testng.annotations.Test;

public class DataSetsClientTest extends WebTest {

    @Inject private AddressDataClient addressDataClient;

    @Test(groups = "web")
    public void shouldGetSecondaryAddress() {
        Address secondaryAddress = addressDataClient.getSecondaryAddress();
        System.out.println(secondaryAddress.getCity());
    }

    @Test(groups = "web")
    public void shouldGetSecondaryAddressAgain() {
        Address secondaryAddress = addressDataClient.getSecondaryAddress();
        System.out.println(secondaryAddress.getCity());
    }

    @Test(groups = "web")
    public void shouldGetSecondaryAddressOnceAgain() {
        Address secondaryAddress = addressDataClient.getSecondaryAddress();
        System.out.println(secondaryAddress.getCity());
    }
}

Let us make sure all 3 tests are running in parallel :

task runWebTests(type: Test) {
    filter {
        excludeTestsMatching "*.mobile.*"
        excludeTestsMatching "*.api.*"
        excludeTestsMatching "*.db.*"
    }
    outputs.upToDateWhen { false }
    useTestNG {
        parallel = "methods"
        threadCount Integer.parseInt(System.getProperty("sessions", "3"))
        includeGroups System.getProperty("tags", "web")
        testLogging.showStandardStreams = true
        useDefaultListeners true
        outputDirectory = file("$buildDir/" + System.getProperty('tags', 'NONE'))
    }
}

Execute tests with NO locking in place#

Let us execute tests

./gradlew clean runWebTests -Dconfig=uat

The output is :

    Bangalore
    Bangalore
    Bangalore

This shows all 3 tests picked the first test data object from the JSON. All tests are picking the same test data.

Execute tests with locking in place#

Let us introduce locking and see the behavior now :

How do we add lock once the test data is retrieved from JSON?

Pass third parameter to getValue() as true

package dataclients.address;

import com.testvagrant.ekam.commons.data.DataSetsClient;

public class AddressDataClient extends DataSetsClient {

    public Address getDefaultAddress() {
        return getAddress(DataKeys.DEFAULT_ADDRESS);
    }

    public Address getSecondaryAddress() {
        return getAddress(DataKeys.SECONDARY_ADDRESS);
    }

    private Address getAddress(String key) {
        return getValue(key, Address.class, true);
    }

    private static class DataKeys {
        public static final String DEFAULT_ADDRESS = "default_address";
        public static final String SECONDARY_ADDRESS = "secondary_address";
    }
}

Once the address is served to the test under execution, a lock is acquired until the test is complete. The lock would be released once the test’s execution is complete.

Now let’s execute the tests with lock.

./gradlew clean runWebTests -Dconfig=uat

The output is :

    Chennai
    Hyderabad
    Bangalore

This shows all 3 tests picked three different test data object from the JSON.