How Two simple techniques can make a big difference in your coding life

TLDR;

Test Driven Development (TDD) and Dependency Injection, or the Dependency Inversion Principle, go hand in hand. They seem to complement themselves. Getting proficient in their use is not hard: you just have to see an example and apply it. These techniques will make your code more modular and easier to test. On top of that, your dependencies will be visible, design smells will be clearer and your test pyramid will be an actual pyramid. In this article, we will see a simple example you can build upon in your daily work.

If you have been using Kafka, or learning about Kafka, you have most probably came across Kafka Connect. The idea of Kafka Connect is to simplify the process of getting data into and out of a Kafka Cluster. The idea is that Kafka Connect abstracts all the logic of resilience, scalability and reliability out of your way to let you focus on writing value-added services. Kafka Connect has a lot of plugins, some commercial and some open source. Its parallel with Kafka Streams makes it a good complement for your Kafka based applications. In this article, I will show you how to build your own Kafka Connect Docker image with your plugins, no matter if they are open source or your own, custom-developed plugins.

Have you ever heard of Test Driven Development? You probably have. You have also probably seen people advocate for it and against it. I believe it has come the time for us to own it as a technique that enables us to make better software. Just remember not to go too dogmatic on it, to introduce some key performance indicators (a.k.a. code coverage), and to turn it into a bottleneck and an impediment to your progress. After all, we have to use the right tool and technique for each job, so don’t go on the radical side.

Introduction

In Part II of this series, we saw how we could create a JUnit Test Extension to make testing with test containers easier. We just need to add the extension to the test to leverage both containers. In fact, we can even add more utility methods to the KafkTestCluster utility class for things we need. The problem is that each time we have a test that uses this extension, we pay more than 20 seconds for the start-up time of both Kafka and the Schema registry. How can we improve this solution?

Introduction

In Part Iof this series, we saw how we could leverage Test Containers to make integration tests that require a Kafka Cluster and a Schema Registry, although we used a single-node Kafka Cluster. In this part, we will refactor the code so that it can be easily used in multiple tests.

Creating a JUnit5 Extension

To reuse code, we will create a JUnit5 Extension. Usually we use these extensions, but creating one is quite simple and opens a new door for test code reuse. As a side note, what is being done here can also be done in JUnit4, but for JUnit4 you are probably better off developing a Test Rule.

Introduction

In this small series of articles, I will share some ideas on using JUnit and Test Containers to test a Kafka application. The use of test containers fits more into the integration test category than the unit test one, but I am not going to dive into that discussion. For me, tests help me develop software faster, be more focused and get faster feedback to know if I am going in the right direction or not. As a side note, yes, I believe and practice in Test Driven Development, but even if you write your tests after, I hope the ideas shared in this and the following articles are useful to you too.