Kota’s Method for Reliable Integration Tests
Fast, Predictable, and Repeatable: Why Our Integration Tests Are Now Faster to Write Than Testing Manually
At Kota, integration tests are a core part of how we ship reliable software. This post walks through the simple framework we use to catch issues early, move fast, and maintain confidence as our systems grow.
First things first, let’s clear up what we mean when we say “integration test.” For us, an integration test ensures that different components of our system interact with each other in the intended way. Some teams include UI in their integration tests (which of course can be valid), but we differentiate between tests that interact with the UI (covered in our previous blog post) and those that don’t. Here, we’re talking specifically about the ones that don’t touch the UI at all.
Our integration test suite uses real infrastructure (via Docker containers) to test the full application stack. This includes HTTP endpoints, database interactions, and authentication flows. Some might call this endpoint testing or API testing. Call it what you like; at least now we’re on the same page.
Our Principles for Integration Tests 📋
Let’s start by outlining some simple principles we follow for our integration tests. These aren’t strict “rules”, but rather practices we’ve arrived at after plenty of trial and error.
Test Isolation
Each test should be independent and able to run in parallel. This is important not only for keeping execution time low (you’re only as fast as your slowest test) but also because it better reflects how the application behaves in production. Requests can arrive at any time, in any order, and your system needs to be able to handle that.
Infrastructure Creation
Each test run should create the required infrastructure using Docker containers. The infrastructure should be created and destroyed for every run to avoid data corruption and guarantee a clean slate. This makes onboarding easier (new engineers can simply click “Run Tests”) and removes environment specific flakiness caused by inconsistent setups.
Direct Database Seeding
When a test depends on data existing in the database, it should insert that data directly rather than calling other endpoints to set it up. If we’re testing an endpoint such as GET /pension/{pensionId}, we don’t want the test to fail because another endpoint responsible for creating pensions unexpectedly broke or had delays. Seeding data directly isolates the behavior you want to test and significantly reduces flakiness.
No External Dependencies
External APIs should be mocked to ensure reliability. While there’s value in detecting breaking changes from third-party providers, we don’t want our pipelines blocked because of issues in someone else’s test infrastructure. Mocking keeps our tests consistent and under our control.
No Actual Messaging
If a test needs to assert behavior that normally results from a message consumer, we call that consumer directly rather than relying on a real message bus. While messages usually process quickly, they can occasionally be delayed. That delay can lead to slower, inconsistent test runs. In many cases, the consumer’s processing time isn’t relevant to what we’re actually testing.
For example, if an EmployeeAddressUpdated event is published, there might be many subscribers handling that event in separate processes. But for this test, we only care about one thing: when we call the endpoint to update the address, is the address updated?
So What Happens When a Test Runs? 🤷♂️
To follow the above rules, a few steps must occur before a test can actually run.
Docker Setup
The integration tests download Docker images for the system’s dependencies (PostgreSQL, Redis) and start those containers using Testcontainers. The first time someone runs the integration tests this can take ~60 seconds, but with cached images subsequent runs take only 5–10 seconds.
This ensures we are always working with a fresh, empty database free from any pre-existing data or corruption.Application Factory
We use WebApplicationFactory to create a test server (i.e. to start the application we will be sending requests to). This lets us apply behaviors and configurations that are only active in tests, not in production.public class CustomAppFactory(Dependencies dependencies) : WebApplicationFactory<Program> { protected override void ConfigureWebHost(IWebHostBuilder builder) { ConfigureSettings(builder); ConfigureTestAuthorization(builder); } private void ConfigureSettings(IWebHostBuilder builder) { builder.UseSetting(”IsIntegrationTests”, “true”); // set a fake connection string for service bus builder.UseSetting(”ConnectionStrings:ServiceBus”, “Endpoint=sb://fake-test-bus.servicebus.windows.net/”); // Set DB connection strings to our newly created image var postgres = dependencies.Postgres.GetConnectionString(); builder.UseSetting(”ConnectionStrings:DbConnection”, postgres + “;Include Error Detail=true”); } }Override Settings
By creating a CustomAppFactory, we can specify configurations that exist solely for our tests. In the example above, we override the connection strings for both the messaging service bus and the database. Our tests now run against the newly created database from step 1, and the messaging framework (MassTransit) no longer attempts to send messages to the real service bus, as it recognizes no such infrastructure exists.Override Authorization
JWT validation is disabled in the test environment to simplify setup and avoid requiring our authentication service. This means:Tests create JWT tokens in-memory using helper methods.
We have helper methods for each type of user, since each requires different claims in the token.
No external identity provider is needed.
No token signing or validation occurs.
Authorization is still enforced at the application logic level. This is important, as it allows us to test that authorization rules are correctly applied. For example, we verify that only an admin token can access an admin-only endpoint.
public static void ConfigureTestAuthorization(IWebHostBuilder builder)
{
builder.ConfigureTestServices(services =>
{
// Override the application AuthZ
services.PostConfigure<JwtBearerOptions>(
JwtBearerDefaults.AuthenticationScheme,
options =>
{
// Completely disable automatic configuration retrieval
options.ConfigurationManager = null;
options.Configuration = null;
options.Authority = null;
options.MetadataAddress = null;
// Disable all validation
options.RequireHttpsMetadata = false;
options.TokenValidationParameters.ValidateIssuer = false;
// ........
// signature validator accepts any JWT without validation
options.TokenValidationParameters.SignatureValidator =
(token, parameters) =>
{
var handler = new JwtSecurityTokenHandler();
return handler.ReadJwtToken(token);
};
// Use events allow us to debug and authorization issues
// that may happen while running the tests
options.Events = new JwtBearerEvents
{
OnAuthenticationFailed = context =>
{
var logger =
context.HttpContext
.RequestServices
.GetRequiredService<ILogger<TestAuth>>();
logger.LogError(context.Exception,
“== OnAuthenticationFailed ==”);
return Task.CompletedTask;
}
};
}
);
};
}
Database Migrations
Because these are fresh databases, we execute all database migrations on each schema to ensure the environment reflects the latest structure. This also required ensuring that all migrations work from a blank slate, which meant making a few retrospective edits to older migration scripts.private void ApplyMigrations(IServiceProvider services) { using var scope = services.CreateScope(); var logger = scope.ServiceProvider .GetRequiredService<ILogger<HarpAppFactory>>(); var dbContextType = typeof(ApplicationDbContext); logger.LogInformation( ”Running migrations for {DbContextType}...”, dbContextType.Name); using var dbContext = (DbContext)scope.ServiceProvider .GetRequiredService(dbContextType); dbContext.Database.Migrate(); logger.LogInformation(”Migrations applied successfully for {DbContextType}”, dbContextType.Name); }Warmup
At this point the hard part is done and the application is ready. We call a warmup endpoint to confirm that the application is running and ready to receive traffic.
Tests run
All tests are run in parallel. This keeps total runtime low and more closely mirrors real production load.Cleanup
After the test run completes, we dispose of all test infrastructure. The Docker containers are stopped and removed, and the environment is reset and ready for the next run where it all happens again.
Test Example 🔬
Below is an example of what an actual running test looks like in our system (yes, this is a real test).
[Test]
[CancelAfter(2 * 60 * 1000)]
public async Task Pension_UpdatePension_Returns200(CancellationToken ct)
{
// Arrange
// Create data in DB
await using var db = Bootstrapper.CreateDbContext();
var org = DatabaseHelper.CreateOrg(db);
var user = DatabaseHelper.CreateEmployee(db, org.id);
var pension = DatabaseHelper.CreatePension(db, user.id);
// Create http client + request
using var client = SharedBootstrapper.CreateHttpClient();
client.AddEmployeeAuthorization(user.Id);
var patchRequest =
new UpdatePensionRequest { EmployeeContribution = 0.1m };
// Act
// Send Http Request
var response = await client.SendAsync(
client.CreateRequest(
HttpMethod.Patch,
Uris.PatchPensionContributionUri,
patchRequest),
ct);
// Assert
response.EnsureSuccessStatusCode();
// Create fresh DB context, and assert that the Employee % is the
correct value based on our request
await using var assertDbContext = Bootstrapper.CreateDbContext();
var pension =
DatabaseHelper.GetPension(assertDbContext, pension.id);
assertPension.EmployeePercentage.Should().Be(0.1m);
}We start by arranging any data that the test will rely on. In this case, we create an organization, an employee, and a pension in the database using helper methods. This demonstrates the direct database seeding approach mentioned earlier in our principles.
Next, we arrange the HTTP client that will send the request. This includes adding the appropriate authorization for the type of user (in this case, an Employee) that will be calling the endpoint. You’ll also notice that we reference the actual request type, UpdatePensionRequest, and allow the HTTP client to handle serialization.
This lets our tests easily construct requests and make assertions against the responses. In this particular example, there’s no response body, just an affirmative status code.
Finally, we assert that the response was successful and that the expected changes were applied to the database. And that’s it, we’ve written a complete integration test.
While there is some initial effort required to build helper methods and test utilities, once we established the foundation it fundamentally changes how we develop. It’s now faster and easier to write an integration test than to manually debug and test the endpoint. I firmly believe that people follow the path of least resistance, and by making integration tests simple to write, we’ve naturally encouraged a steadily growing suite of automated, repeatable tests.
Wrapping up 🎁
We’ve built an integration testing setup that mirrors real infrastructure, isolates each test, eliminates external flakiness, all while staying fast enough to run on every machine, every time. We chose this approach because we wanted tests that developers actually want to write: predictable, debuggable, and reflective of real application behavior. Now that the foundation is in place we are implementing this across multiple teams and systems and it is paying off. Writing an integration test is often faster than manual debugging, and our growing suite of tests gives us confidence to move quickly. It’s become a core part of how we ship software at Kota, and it keeps getting better as we continue to invest in it.
Shoutout to the engineers here before my time for the great foundations they have laid within Kota in this area 👏 If anybody reading would like more information feel free to reach out ✌️