At Kota, integration tests are a core part of how we ship reliable software. This post walks through the simple framework we use to catch issues early, move fast, and maintain confidence as our systems grow.

First things first, let’s clear up what we mean when we say “integration test.” For us, an integration test ensures that different components of our system interact with each other in the intended way. Some teams include UI in their integration tests (which of course can be valid), but we differentiate between tests that interact with the UI (covered in our previous blog post) and those that don’t. Here, we’re talking specifically about the ones that don’t touch the UI at all.

Our integration test suite uses real infrastructure (via Docker containers) to test the full application stack. This includes HTTP endpoints, database interactions, and authentication flows. Some might call this endpoint testing or API testing. Call it what you like; at least now we’re on the same page.

Our Principles for Integration Tests 📋

Let’s start by outlining some simple principles we follow for our integration tests. These aren’t strict “rules”, but rather practices we’ve arrived at after plenty of trial and error.

Test Isolation

Each test should be independent and able to run in parallel. This is important not only for keeping execution time low (you’re only as fast as your slowest test) but also because it better reflects how the application behaves in production. Requests can arrive at any time, in any order, and your system needs to be able to handle that.

Infrastructure Creation

Each test run should create the required infrastructure using Docker containers. The infrastructure should be created and destroyed for every run to avoid data corruption and guarantee a clean slate. This makes onboarding easier (new engineers can simply click “Run Tests”) and removes environment specific flakiness caused by inconsistent setups.

Direct Database Seeding

When a test depends on data existing in the database, it should insert that data directly rather than calling other endpoints to set it up. If we’re testing an endpoint such as GET /pension/{pensionId}, we don’t want the test to fail because another endpoint responsible for creating pensions unexpectedly broke or had delays. Seeding data directly isolates the behavior you want to test and significantly reduces flakiness.

No External Dependencies

External APIs should be mocked to ensure reliability. While there’s value in detecting breaking changes from third-party providers, we don’t want our pipelines blocked because of issues in someone else’s test infrastructure. Mocking keeps our tests consistent and under our control.

No Actual Messaging

If a test needs to assert behavior that normally results from a message consumer, we call that consumer directly rather than relying on a real message bus. While messages usually process quickly, they can occasionally be delayed. That delay can lead to slower, inconsistent test runs. In many cases, the consumer’s processing time isn’t relevant to what we’re actually testing.

For example, if an EmployeeAddressUpdated event is published, there might be many subscribers handling that event in separate processes. But for this test, we only care about one thing: when we call the endpoint to update the address, is the address updated?

So What Happens When a Test Runs? 🤷‍♂️

To follow the above rules, a few steps must occur before a test can actually run.

1. Docker Setup
   The integration tests download Docker images for the system’s dependencies (PostgreSQL, Redis) and start those containers using Testcontainers. The first time someone runs the integration tests this can take ~60 seconds, but with cached images subsequent runs take only 5–10 seconds.
   This ensures we are always working with a fresh, empty database free from any pre-existing data or corruption.

2. Application Factory
   We use WebApplicationFactory to create a test server (i.e. to start the application we will be sending requests to). This lets us apply behaviors and configurations that are only active in tests, not in production.

   public class CustomAppFactory(Dependencies dependencies) : WebApplicationFactory<Program>
   {
       protected override void ConfigureWebHost(IWebHostBuilder builder)
       {
           ConfigureSettings(builder);
           ConfigureTestAuthorization(builder);
       }
   
       private void ConfigureSettings(IWebHostBuilder builder)
       {
          builder.UseSetting(”IsIntegrationTests”, “true”);
    
          // set a fake connection string for service bus
          builder.UseSetting(”ConnectionStrings:ServiceBus”, “Endpoint=sb://fake-test-bus.servicebus.windows.net/”);
          
         // Set DB connection strings to our newly created image
           var postgres = dependencies.Postgres.GetConnectionString();
           builder.UseSetting(”ConnectionStrings:DbConnection”, postgres + “;Include Error Detail=true”);
       }
   }

3. Override Settings
   By creating a CustomAppFactory, we can specify configurations that exist solely for our tests. In the example above, we override the connection strings for both the messaging service bus and the database. Our tests now run against the newly created database from step 1, and the messaging framework (MassTransit) no longer attempts to send messages to the real service bus, as it recognizes no such infrastructure exists.

4. Override Authorization
   JWT validation is disabled in the test environment to simplify setup and avoid requiring our authentication service. This means:

   • Tests create JWT tokens in-memory using helper methods.

   • We have helper methods for each type of user, since each requires different claims in the token.

   • No external identity provider is needed.

   • No token signing or validation occurs.

   • Authorization is still enforced at the application logic level. This is important, as it allows us to test that authorization rules are correctly applied. For example, we verify that only an admin token can access an admin-only endpoint.

public static void ConfigureTestAuthorization(IWebHostBuilder builder)
{
    builder.ConfigureTestServices(services =>
    {
       // Override the application AuthZ
        services.PostConfigure<JwtBearerOptions>(
            JwtBearerDefaults.AuthenticationScheme,
            options =>
            {
              // Completely disable automatic configuration retrieval
              options.ConfigurationManager = null;
              options.Configuration = null;
              options.Authority = null;
              options.MetadataAddress = null;

              // Disable all validation            
              options.RequireHttpsMetadata = false;                    
              options.TokenValidationParameters.ValidateIssuer = false;
              // ........

              // signature validator accepts any JWT without validation
             options.TokenValidationParameters.SignatureValidator =          
              (token, parameters) =>
              {
                var handler = new JwtSecurityTokenHandler();
                return handler.ReadJwtToken(token);
              };

             // Use events allow us to debug and authorization issues        
             // that may happen while running the tests                
             options.Events = new JwtBearerEvents                
             {                    

               OnAuthenticationFailed = context =>                    
               {                        
                 var logger = 
                  context.HttpContext
                         .RequestServices
                         .GetRequiredService<ILogger<TestAuth>>();                        

                  logger.LogError(context.Exception, 
                   “== OnAuthenticationFailed ==”);                        

                  return Task.CompletedTask;                    
                 }                
               };            
             }        
            );    
     };
}

5. Database Migrations
   Because these are fresh databases, we execute all database migrations on each schema to ensure the environment reflects the latest structure. This also required ensuring that all migrations work from a blank slate, which meant making a few retrospective edits to older migration scripts.

   private void ApplyMigrations(IServiceProvider services)
   {
       using var scope = services.CreateScope();
       var logger = scope.ServiceProvider
                   .GetRequiredService<ILogger<HarpAppFactory>>();
   
       var dbContextType = typeof(ApplicationDbContext);
       logger.LogInformation(
          ”Running migrations for {DbContextType}...”,
           dbContextType.Name);
               
        using var dbContext = 
          (DbContext)scope.ServiceProvider
          .GetRequiredService(dbContextType);
   
        dbContext.Database.Migrate();
   
        logger.LogInformation(”Migrations applied successfully for 
        {DbContextType}”, dbContextType.Name);
   }
   

6. Warmup

   At this point the hard part is done and the application is ready. We call a warmup endpoint to confirm that the application is running and ready to receive traffic.

7. Tests run
   All tests are run in parallel. This keeps total runtime low and more closely mirrors real production load.

8. Cleanup
   After the test run completes, we dispose of all test infrastructure. The Docker containers are stopped and removed, and the environment is reset and ready for the next run where it all happens again.

Test Example 🔬

Below is an example of what an actual running test looks like in our system (yes, this is a real test).

[Test]
[CancelAfter(2 * 60 * 1000)]
public async Task Pension_UpdatePension_Returns200(CancellationToken ct)
{
    // Arrange
    // Create data in DB
    await using var db = Bootstrapper.CreateDbContext();
    var org = DatabaseHelper.CreateOrg(db);
    var user = DatabaseHelper.CreateEmployee(db, org.id);
    var pension = DatabaseHelper.CreatePension(db, user.id);
    
    // Create http client + request
    using var client = SharedBootstrapper.CreateHttpClient();
    client.AddEmployeeAuthorization(user.Id);
    var patchRequest = 
      new UpdatePensionRequest { EmployeeContribution = 0.1m };
    
    // Act
    // Send Http Request
    var response = await client.SendAsync(
        client.CreateRequest(
            HttpMethod.Patch,
            Uris.PatchPensionContributionUri,
            patchRequest),
        ct);

    // Assert
    response.EnsureSuccessStatusCode();

    // Create fresh DB context, and assert that the Employee % is the   
       correct value based on our request
    await using var assertDbContext = Bootstrapper.CreateDbContext();
    var pension = 
        DatabaseHelper.GetPension(assertDbContext, pension.id);
    assertPension.EmployeePercentage.Should().Be(0.1m);
}

We start by arranging any data that the test will rely on. In this case, we create an organization, an employee, and a pension in the database using helper methods. This demonstrates the direct database seeding approach mentioned earlier in our principles.

Next, we arrange the HTTP client that will send the request. This includes adding the appropriate authorization for the type of user (in this case, an Employee) that will be calling the endpoint. You’ll also notice that we reference the actual request type, UpdatePensionRequest, and allow the HTTP client to handle serialization.
This lets our tests easily construct requests and make assertions against the responses. In this particular example, there’s no response body, just an affirmative status code.

Finally, we assert that the response was successful and that the expected changes were applied to the database. And that’s it, we’ve written a complete integration test.

While there is some initial effort required to build helper methods and test utilities, once we established the foundation it fundamentally changes how we develop. It’s now faster and easier to write an integration test than to manually debug and test the endpoint. I firmly believe that people follow the path of least resistance, and by making integration tests simple to write, we’ve naturally encouraged a steadily growing suite of automated, repeatable tests.

Wrapping up 🎁

We’ve built an integration testing setup that mirrors real infrastructure, isolates each test, eliminates external flakiness, all while staying fast enough to run on every machine, every time. We chose this approach because we wanted tests that developers actually want to write: predictable, debuggable, and reflective of real application behavior. Now that the foundation is in place we are implementing this across multiple teams and systems and it is paying off. Writing an integration test is often faster than manual debugging, and our growing suite of tests gives us confidence to move quickly. It’s become a core part of how we ship software at Kota, and it keeps getting better as we continue to invest in it.

Shoutout to the engineers here before my time for the great foundations they have laid within Kota in this area 👏 If anybody reading would like more information feel free to reach out ✌️

At Kota, we’re always looking for ways to improve how we work - not just for our engineering team, but ultimately for our customers too. One of the biggest wins for us recently has been creating end-to-end (E2E) tests with Playwright, integrated directly into our GitHub Actions workflow.

Why? Because our embed product is complex, every provider has slightly different rules and flows and we spend lot’s of time manually checking them. Now we have built an automated safety net that catches issues early, ultimately saving hours of manual testing and makes everyone a little bit more confident when hitting “merge”.

Why we went down the E2E Road..

Let’s be honest: at this point, having automated tests in your CI isn’t groundbreaking - it’s become a fundamental part of engineering. But when you’re dealing with a product as complex as a multi-provider health insurance platform, “basic coverage” just wasn’t cutting it.

We needed something more comprehensive, something that gave us confidence in all of our user journey’s, from setting up an employer to onboarding employees, across multiple insurers.

That’s where our Playwright suite comes in.

What we built..

We designed our tests around real-world flows, not just isolated cases. A few things we are happy about:

• Provider-specific coverage: every insurer we support has dedicated tests

• Realistic test data: our scenarios look real

• Success + Failure checks: we don’t just test the happy path

• Reusable utilities: actions, selectors, navigation helpers and API utilities keep the suite clean and easy to extend.

So basically we can spin up employers and employees on the fly, run them through real flows and get reliable results.

How we integrated into our CI..

Sure, running tests in CI is pretty standard, but we didn’t just want to tick the box. The challenge was making them fast and reliable so they actually help us, not slow everyone down.

How we structured it

• Local backend runs → spin up the backend in Docker, so tests always hit the latest code

• Build first, then test → E2Es only run after a successful build, so we don’t waste time if things are already broken

• Parallel checkouts → frontend and backend clone side-by-side, no waiting around

How we made it faster

• Build cache hits save 5–8 minutes per PR

• Parallel jobs make validation about 3× faster

• Browser caching shaves off another 2–3 minutes

• Smart cleanup keeps us under GitHub’s storage limits

What it adds up to

• Build < 1 minute

• Validation (lint, unit tests, typecheck) ~2 minutes

• E2Es ~10 minutes (cloning backend, installing Playwright, running tests)

Because validation and E2Es run in parallel, the total wall-clock time from PR to merge is ~ 12 minutes. Engineers can open a PR, get the full suite run, and merge to main with confidence — all in under a quarter of an hour.

And for the smaller stuff, we’ve added a bypass label. If a PR is just a dependency bump or a trivial change, engineers can skip E2Es entirely, saving time without sacrificing safety where it really matters.

Why does it matter?

The payoff is simple: engineers can open a PR, get feedback fast, and merge with confidence. We don’t have to spend hours manually checking flows, and we catch issues earlier, before they sneak into production.

More importantly, it frees us up to focus on the fun stuff: building features that actually make health insurance easier for employers and employees.