Spring integration tests: A kickstart to mistreat your software.

Nowadays there is a trend in the companies to avoid having testers in the teams and have developers test its own software. In my particular case it has been years since I have had a tester on my team. In general, testers have been moved outside the small teams and placed where they can test the system as a whole and not an api in particular.

Recently, a fellow colleague said a phrase a got stuck in my mind since then and made give another look to how I test the software I write:

The worst you treat your software during the development and testing phase, the better it will work in production.

It is just amazing how simple it sounds and yet how much sense it makes… well, the person who said the phrase used to be a university teacher so it should come as no surprise the he could articulate it that way.

This is an extremely long topic and it takes a lot of time to master. Do not expect a full crash course from this article but rather expect some key bullet points to see what features Spring provides for integration testing and how you can use them improve your own testing skills and start mistreating your software before going to production.

The project

In order to showcase the different features that you can use when you need to write integration tests with Spring, I have written a sample application that can be found here.

nacho270/spring-integration-tests-demo

I have used the same domain as in my previous articles about JOOQ and spring alternatives. With the addition of MongoDb and Kafka to demonstrate more features of the spring testing tool suite.

The most important flow to test is the one that creates the shipment which does the following:

• Looks up the products in the DB.
• Looks up the user in the DB and if it is not present, it will look for it in jsonplaceholder and then store it in the DB.
• Saves the shipment in the DB.
• Saves a log entry in MongoDb.
• Sends a “shipment created” news to Kafka.
• Eventually a Payments System will notify the outcome of the shipment payment and the application will update its DB.

Application components

SpringBootTest

The @SpringBootTest annotation is the starting point of a Spring integration test.

Among its configuration options you can find:

• properties: It is an array of strings where you add/override environment properties.

• webEnvironment: How you want to run your web dependencies. It can be either MOCK (default), NONE, DEFINED_PORT (where you can specify in which port you want your controllers to listen) or RANDOM_PORT (this is the most common to avoid clashing with other processes using a port).
• classes: In here you can also tell spring to include some component or configuration defined just for testing. By default everything component is loaded in the application unless you use a test slice (To be covered later).

To this, you can start adding other related annotations like:

• @ActiveProfiles: As the name states, you can define a list of profiles that will be used by your test. You can use this to load certain beans or configuration depending on the profile.

Notes on profiles

By convention, the configuration file is called application (can be .properties or .yaml) and Spring will load it automatically. But it will also load the application file for the context. For example: if you run with context test, Spring will load the application file and the application-test file. And if you run with profile integration, Spring will load the application file and the application-integration file.

If you have an application properties file in your test/resources folder, it will override the application file from you main/resources folder.

You can also run your tests with multiple profiles at the same and Spring will load all the related files, eg, if you run with contexts test and integration, Spring will load the application file, application-test file and the application-integration file.

• @ContextConfiguration: You can use this add hooks to your context with initializers. We will cover that later.
• @Import: This will allow you to load beans that would not be used normally such a specific test configuration that will override the default one. Or it can be used to import beans from another project that were not loaded in the context.
• @DirtiesContext: This annotation will tell Spring to reload the context before or after the test. Given Spring tries to create the context once and reuse it on every test, this will imply reloading all the dependencies which will cause you tests to take longer to run. It is useful if you have some classes that will hold data in memory for tests or if you want to quickly wipe your H2 database. You should design your application, your tests and configuration to avoid using this.

Configuration classes

It is super common that you will need different configuration for your tests than for your product. Sometimes it is not simply a matter of changing the profile to load different properties. You may need to create different implementations for some bean or even create new beans that you would only need for testing.

You can define them as nested static classes and they will be automatically loaded by Spring. However, if you want to reuse them in different tests, you can create them as standalone classes and use the classes attribute of @SpringBootTest in your test to load them.

What is the difference between @Configuration and@TestConfiguration ?

• @Configuration: Will replace the application’s configuration. Remember that@Configuration classes are also beans, so if you want to avoid problems when creating the context, name them the same. For example: If you have a @Configuration class in src/main/java/com/myapp/MyConfiguration and you want to fully replace it, create src/test/java/com/myapp/MyConfiguration annotated with @Configuration. If you encounter any issues, you can always fallback to annotating a bean as @Primary in order for it to take precedence, but it is not the most elegant solution.
• @TestConfiguration: Will be used a an addition to the application’s configuration. This is useful when you want to create beans only for testing. In my case, my application simply sends and a message to a queue and the forgets about it; But in the tests I want to make sure that works, so I had to create a Consumer bean that will listen to the topic where my applications sends the message so then I can assert that message has been sent.

Mocking

A mock is a simulated instance of class that will allow you to define a desired behaviour. There are many libraries such as mockito, powermock or easymock.

Spring comes bundled with mockito integration. So when you have a @SpringBootTest you can annotate you attributes with @MockBean and Spring will automatically inject them as mocks. Then you can interact with them as if you were using standard mockito, you can do when, verify, doThrow, etc.

Useful AssertJ custom assertions

Needless to say that the key factor in testing is asserting on the results obtained from the system. For this, there are many assertion libraries such as junit5-assertions, hamcrest or assertj.

In my case, the one I like the most is Assertj because it allows you to write fluent assertions that will make your test code more readable.

And one thing I like a lot is to write my own custom assertions so I can assert on my object as a whole rather than asserting its internal attributes individually. This makes it easy to encapsulate the assertion logic, reuse it and increase the readability of the tests.

Here is an example of you can write a custom assertions with Assertj:

Test Slices

Test Slices are a Spring Boot feature that will create a reduced application context for a specific slice of your app. Some of them are:

• @WebMvcTest: Covered in the next section.
• @DataJpaTest: Simply inject your repositories and then you can store and retrieve data. Be aware that @Modifying queries require transactions and you may need to flush if you want to retrieve the new value. For that you will need to inject TestEntityManager as seen below.

In the previous example I using @ParametrizedTest which can be super useful for testing multiple combination of parameters for the same use case. See this entry for a full explanation: https://www.baeldung.com/parameterized-tests-junit-5

• @DataMongoTest: Same as above, simply inject your Mongo repositories and you store and retrieve data.

You may find all the test slices in the spring source code.

WebMvcTest

Now that we have laid the foundation for the tests, we can start to put all these things together to test our application.

The easiest integration test we can make is for the rest api. Why? Because it will only care for simple things: request method, path, response and http response code.

For this test we will only start the web side of the app and forget about databases, Kafka, etc.

Spring provides a test slice for this: @WebMvcTest to which you need to specify the controller class you want to test. Under these tests, Spring will automatically create a MockMvc bean that will be in charge of interacting with the api and asserting on the results.

I reckon that in order to use this MockMvc in a readable way you need to make extensive use of static imports, otherwise it would be very hard to read.

There are 2 parts to this tests: the perform and the expects. For the perform part you will need a MockHttpServletRequestBuilder from MockMvcRequestBuilders that will allow you to create the request that you want. The perform will return a ResultActions instance that will allow you to do the assertions. In my case I have used jsonpath to assert on the response body of the result. You can find details of the jsonpath syntax here.

Ultimately, you can extract the body as a String, parse it and do the assertions yourself:

Testing the business logic

This is the point where we actually test the behaviour of the system and its interaction with the infrastructure.

Also, we can have some non-functional tests here (NFT). We can test if the transactions are working fine, if some retry mechanism works, if a circuit breaker works, etc. Of course we could not (and should not) do stress/load here for 2 reasons: One being that this tests will belong to your build pipeline and it should test the correctness of your system rather than performance, and the second being the lack of resources; a stress/load test only makes sense on a production like environment with real infrastructure.

The spectrum here is wide, it depends on how many integrations you have. For this example, my application needs to talk to an external Rest api, MongoDb, MySQL and Kafka.

Wiremock — mocking external api calls

This not part of Spring. Wiremock allows us to simulate external api call responses. The way it works is that it starts a local web server and then we can configure its behaviour upon receiving some request. The concept is the same as standard mocking: we configure a request matcher and instruct the Wiremock server to return the response that we want.

In order to configure it we need to play with the context life-cycle. Why? Because, as of today, Wiremock works Junit-4 @Rule and that is not available with Junit-5 which is what I use. If you want to see how to configure Wiremock with Junit-4, click here.

For Junit-5 we need to add a context initializer:

Do not forget to add the wireMockServer.stop() when the application is shutting down so you clear everything correctly and your shutdown is graceful.

Once the Wiremock server is configured and added to the context, the next step is to override the external endpoint configuration to make it go to the local Wiremock server. For that, it is good to have that configuration externalised in a property so that we can then override it with TestPropertyValues.

Once all of that is configured, you can inject the WiremockServer as a standard Spring bean an interact with it. In this example, we are instructing Wiremock to return a specific json payload with response code 200 when the application under test calls /users/1. You can even use regular expressions when defining which URL you want to mock. With that you can do things like failing if the user id is odd and return OK when it is even.

Embedded MongoDb

This comes bundled with SpringBoot. The main concept is that you will have an in-memory MongoDb database that your application will talk to.

Configuring this is super simple, you just need to add this to your pom file:

<dependency>
    <groupId>de.flapdoodle.embed</groupId>
    <artifactId>de.flapdoodle.embed.mongo</artifactId>
    <scope>test</scope>
</dependency>

That is it! To use it, simply inject a MongoTemplate in your test and you are good to go. Your application will seamlessly write to it and you can query it on your test:

Query query = new Query();
query.addCriteria(Criteria.where("identifier").is("1234"));
List<LogEntry> logEntries = mongoTemplate.find(query, LogEntry.class);

One thing to keep in mind is to add the following to your @SpringBootTest class to force it to wait until the embedded mongo stops once the test is finished:

@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)

Embedded DB — H2

This comes bundled with SpringBoot. Same as with embedded mongo, with H2 you will have an in-memory relational database that your application will talk to. This is standard tool in the market but it is a bit polemic. The problem is that there is not single RDBMS engine in the market and they behave differently and even have different SQL syntax to do the same things. For example, paging results is done differently in Oracle than in MySQL.

To make it even more complicated, H2 is yet another engine with its own behaviour. You can add some compatibility mode but it still needs extra configuration if you want to achieve an almost exact behaviour to your real engine. The other issue is with syntax, if you have native queries in your application, chances are you will run into trouble if you make use of more advanced features. You can avoid this by either using hibernate or JOOQ as shown in my previous article.

To configure it, you will have to add the following dependency in your pom:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>test</scope>
</dependency>

And this configuration in your tests:

spring:
  datasource:
    url: jdbc:h2:mem:test;DATABASE_TO_LOWER=TRUE
    username: root
    password:

You will notice the DATABASE_TO_LOWER=TRUE that was something I had to add in order to maintain compatibility between the real DB (MySQL) and H2.

Once you have that, your application will talk to H2 as if it were the real database.

One useful thing that Spring has is JdbcTestUtils. This will allow you to delete and count rows quickly in your tests without the need of writing specific methods in your production code, all you need to do is inject a JdbcTemplate in your test, which comes for free given you have configured a data-source:

Embedded Kafka + Awaitility

Embedded Kafka comes as part of Spring-Kafka. It is an in-memory Kafka broker.

To configure it, you will need this in your pom file:

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka-test</artifactId>
    <scope>test</scope>
    <exclusions> <!-- If you are using Junit5 -->
        <exclusion>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
        </exclusion>
    </exclusions>
</dependency>

And this in you application properties:

spring:
  kafka:
    bootstrap-servers: ${spring.embedded.kafka.brokers}

To use it in your test, you will need to annotate your @SpringBootTest class with @EmbeddedKafka(topics = {"topic_1", "topic_2"}). You will need to specify every topic that your tests will send messages to or receive messages from.

When it comes to asserting in theses tests is when things get tricky.

If your application sends a message you will need to create a Consumer for your test as I have shown before. But sending a message is an asynchronous operation and the message will eventually be received on the other side. For this there are 2 tools that will help: Spring’s KafkaTestUtils and Awaitility. The first allows us to consume records from a Kafka consumer. Awaitililty on the other hand is not related to Kafka at all and allows the test to wait until something happens. Combining these 2 together will result in something like this:

We wait for 2 seconds, then the we start checking every 500 milliseconds and up to 1 minute for something to happen. That something in my case is receiving a given number of messages matching a predicate from my Kafka consumer.

The same thing happens when you want to test how your application behaves when receiving a message. In your test you will send a message to Kafka and eventually your application will consume it and do something. We can also use Awaitility for that too:

In this case, we send a message to Kafka stating that some shipment has been paid and the we wait until we can find that shipment in our system with status PAID.

Feature testing

Feature tests, sometimes known as acceptance tests, are the place where you test your actual business flows purely based on your application’s public interfaces. You do not test one service in particular and all its methods, you test whatever is relevant for a business use case.

In my application, I wrote a test that creates a product and then creates a shipment on the back of that product. Afterwards, I not only make sure that the data is correct, I also make sure the entry in MongoDb is stored and the shipment news is published. I do all that through the rest api or Kafka which are the a public entry points of my application. Basically, theses tests should simulate external actors to your system, such as a user or another system producing messages to a queue that your system will consume.

BDD — Cucumber

The typical tool for building this kind of tests is BDD (Behaviour Driven Development) which simply consists on a set of structured natural language instructions that business person could read, and potentially write. Each instruction, also called step, is one line of text that is bound to a some java code in the background.

You can define the steps as you wish, add variables, make them generic, etc. In general, there are 4 keywords for these steps that can be used however you want, but the rule of thumb would be:

• Given: To establish preconditions
• When: Some stimulus happen to the system.
• Then: Check system response to that stimulus.
• And: Used to check on another condition.
• But: To define a condition that contradicts something.

You can add as much or as little responsibility to the steps as you wish. You can assert everything in the then for instance.

For BDD there are implementations such as Cucumber or JBehave. In my case I have chosen cucumber.

To configure Cucumber for Junit5 you need:

• These dependencies in your pom file:

<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>${cucumber.version}</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java8</artifactId>
    <version>${cucumber.version}</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit-platform-engine</artifactId>
    <version>${cucumber.version}</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-spring</artifactId>
    <version>${cucumber.version}</version>
    <scope>test</scope>
</dependency>

• Add a junit-platform.properties to your src/test/resources with:

cucumber.execution.parallel.enabled=false
cucumber.glue=cucumber
cucumber.plugin=pretty,timeline:target/timeline,html:target/cucumber-reports/cucumber.html,junit:target/cucumber-reports/cucumber.xml

• Create a cucumber package in the root source of your tests src/test/java with a cucumber runner. This comes from the glue property defined above. Cucumber will scan that package:

@Cucumber
public class CucumberTestsRunner {
}

• Create a @CucumberContextConfiguration somewhere inside that package which is where you will be configuring the spring context for the cucumber runner.
• Create a cucumber folder inside you src/test/resources which is where you will put you feature files (your tests).

You can put only one feature per feature file and multiple scenarios inside. You can also define a Background which is just a series of steps to be ran before each scenario.

For the scenarios, you have 2 types:

• Standard: This is the simplest one, it runs with values specified in the steps.

Feature: Shipment lifecycle
  
  Background:
   Given a product with name 'laptop' and a price of 1000.00
  Scenario: Can create a shipment
   When a shipment is created for user 1 with 2 items of the product
   Then the shipment payment status is PENDING
   And the shipment total is 2000.00
   And there is a log entry for the shipment
   And a shipment created news is sent

• Outline: This allows to run the same tests with different values, known as examples. In this case the test will run 2 times, replacing the the PAYMENT_OUTCOME variable with the values from the examples table.

Feature: Shipment lifecycle
  Background:
   Given a product with name 'laptop' and a price of 1000.00
  Scenario Outline: Can process payment outcome
   When a shipment is created for user 1 with 2 items of the product
   Then a shipment created news is sent
   Then a payment outcome of '<PAYMENT_OUTCOME>' is sent
   And the shipment payment status is <PAYMENT_OUTCOME>

    Examples:
      | PAYMENT_OUTCOME |
      | PAID            |
      | REJECTED        |

One key thing I mentioned before is that in the When steps you send a stimulus to the system but it is on the then/and steps where you verify. This means that the steps must share state. For that there is special cucumber annotation @ScenarioScope. Just create a bean annotated with it, inject it where you define your steps and then you will store the data in there. This is effectively creating a stateful bean with data shared across every step for that scenario. Remember everything has to be in the cucumber package. For example:

Last but not least, you need to define the steps. For this there are 2 ways:

• With annotations: This are simple methods inside the TestSteps. For example:

@Given("a product with name {string} and price {double} is created")
public void createProduct(String name, Double price) {
 ....
}

• With lambdas: For this, you will need to implement the interface io.cucumber.java8.En in your TestSteps class (There are interfaces for many other languages such as French or Spanish). Then in the constructor you can call the inherited methods like this:

public class TestSteps implements En {
  public TestSteps() {
     Before(() -> wireMockServer.resetAll());

     When("Something {int}", (Integer variable) -> {
         ....
     });
   }
}

In my case, I prefer the annotations.

You will notice that the steps have variables that you can specify with {string}, {int}, etc. You can also define you own types to make your code more readable. Those are called ParameterTypes:

As you can see, instead of saying the status is {string}, we can use our own parameter type and we will have type safety in the method as it will receive a instance of ShipmentPaymentStatus instead of a String that we would need to convert.

Lastly, in order to make the api calls I used RestAssured which is quite similar to the Spring MockMvc but it is not related to the spring context. Now, using this with Junit5 is a bit tricky, you will need the following dependencies:

<dependency>
    <groupId>io.rest-assured</groupId>
    <artifactId>rest-assured</artifactId>
    <version>${restassured.version}</version>
</dependency>
<dependency>
    <groupId>io.rest-assured</groupId>
    <artifactId>rest-assured-all</artifactId>
    <version>${restassured.version}</version>
</dependency>
<dependency>
    <groupId>io.rest-assured</groupId>
    <artifactId>json-path</artifactId>
    <version>${restassured.version}</version>
</dependency>

There is another thing that is tricky: In order to maintain portability of your application, your @SpringBootTest should run with webEnvironment = WebEnvironment.RANDOM_PORT. The problem with this is that port selected by Spring is not defined until the web server is actually running. So we need to intercept the point where that port is defined and use it configure RestAssured. Also if we want have type safety when we parse the responses from the api calls, we need to configure an ObjectMapper. Like this:

And to use it:

Tip

You can run a scenario individually if you want to test it in particular. The scenarios support tags and with them you can target the specific one you want to run. Once tagged, you will need to instruct the cucumber runner to run only those with that specific tag. The tags can be combined.

@current
Scenario: Can create a shipment
….

@ignored
Scenario: Can create a malformed shipment
….

And in the run configuration add this to the “VM options”:

-Dcucumber.filter.tags=”@current and not @ignored”

This applies to the latest version of cucumber. If you are on al onder version, it’s simpler. Just add this annotation to your cucumber configuration:

@CucumberOptions(tags = "@current and not @ignored")

In this example, it would be sufficient to just use @current. The cucumber runner will trigger those which have the tag. The addition of @ignored is just to demo how you can combine tags.

Failure scenarios

The combination of cucumber with mockito can be really powerful when testing failure scenarios. A failure scenario is when you simulate that some part of the system dependencies does not work as intended, eg, a DB failure. If you want to really mistreat your software, you should be adding a few these scenarios. You can test that your data is consistent given a failure in an external API call, in the DB, a few retries. You can even test a circuit breaker recovery scenario like something failing at the beginning and then recovering.

In my case, I wrote a simple failure scenario where the DB fails inside the Kafka listener that should trigger a retry and everything should work the second time.

Scenario: Should retry payment outcome upon failure
  Given saving the payment status fails but works the second time

One great thing you can do with mockito is to append multiple behaviours for the same method. In this case, I want the markAs method to throw an exception the first time is executed and to call the real method the second time. Here, doCallRealMethod works because the repository is actually a spy (partial mock) instead of a mock. If you have a mock you could do something like:

when(theMock.doSomething())
    .thenReturn(1)
    .thenReturn(2)
    .thenThrow(new RuntimeExecption("Some failure"));

Always keep in mind that the way to setup the mock behaviour is different based on the return type of the method, ie, if it returns void or not.

To achieve this behaviour in my project, I had to create a wrapper class around the shipmentRepository and use a spy. This is because you cannot spy/mock a Spring CRUD Repository because it is an interface that translates into a final proxy class on runtime and you cannot mock finals with Mockito. I used spy around that wrapper so I can trigger the real behaviour of the wrapped repository.

What about test containers?

TestContainers is a library that allows to run docker containers from with your tests so you can run your a real MySQL, MongoDb, Kafka in there.

Another options is that you could build your application’s docker image, have another project with a set of cucumber tests that spin up you application and all the necessary infrastructure using TestContainers or even docker-compose and run tests against it.

I did not a demo of this for this article because the main idea was to show the Spring features.

Conclusions

Many feel that it is harder to test the application than to write the application itself. And yes, it is hard to disagree on that.

Arguably, you need to have another set of configuration properties, beans, an extensive list of testing dependencies and try to make it all work together. And if you add to that the need of simulating failures and making sure the data is consistent… it can be pretty tough. Things get exponentially harder when you include asynchronous code such as queues or reactive programming. It is not uncommon to face flaky tests (some times they work, sometimes they do not) in those scenarios.

Ultimately, it is better to deal with all those issues while you are not in production. It is not pleasant to receive a 2 AM phone call when something fails or having to deal with inconsistencies in your data, specially when you work with money.

Having said that, I think the tool spectrum offered by Spring is quite broad and it gets the job done eventually. As I said before, testing requires tons of practice to get it right because there are tons of tools and options to design your tests. In this article I have covered Spring tests for rest apis and Kafka using Junit5 and embedded tools, but there’s also Junit4, TestNG, Spock, Selenium, TestContainers, Citrus, JBehave, XRray, etc. Not only that, then there is the side of deciding which failures you want to simulate and how simulate them, how to have reusable steps, reusable assertions… and of course everything has to be reliable and readable. The cherry on the top is whether you want to use TDD or not… limitless options as you can see.

As a final thought, I can say that us developers need to up our game when it comes to testing the software. It is not just a matter of adding coverage… it is a matter of adding value. Every failure or bug you detect in your tests, is one less headache in the future. It may not be the most fun thing to do, but you have to… just think of all the testing done to cars or medicines before being released to the market.

Testing your software extensively also increases your ownership, it forces you to learn a lot of things you did not know when something fails, to deeply understand everything that is happening whether it is a piece of code that you wrote or not. The more knowledge you have, the easier it will be for you to find a solution when something fails in production… because things will fail in production, no matter what. There are many different things that could fail there that are completely out of you control such as infrastructure failure, another system not responding, failing or returning wrong data, etc. At least you can be sure that your tests caters for the things that could go wrong in your application.

I hope you have enjoyed the article and that you have found it useful!

Happy testing :)