Component tests in Node.js and Jest

Do you feel comfortable releasing your service at 3pm on a Friday? We do.

In this article, we will take a look at how to run component tests in a Node.js service along with unit tests in place and become comfortable to release on a Friday afternoon. Component tests provide a lot of confidence, when it comes to releasing changed or new functionality into production.

Testing - What is it about?

One statement upfront, and I am sure you follow this when reading such a blog post: Test your code!

Why should you test? From my perspective, you and your team should feel comfortable to release your code at any time. You should not fear to break the system and prevent your users enjoying your application or service. Definitely, you will not cover 100% upfront, but you should do as good as possible. In addition to tests, you should also have intensive logging, monitoring, and alerting in production to increase your confidence to release.

There are different types of tests. This article will only give you an idea, how to have component tests next to your unit tests within a Node.js service code base, and run them with Jest. It will not provide a deep dive into test theory at all.

One excellent source about testing, we attempt to follow at LeanIX, is the info deck on microservice testing by Toby Clemson. Check it out for all the details around test types and how to use them in a microservice environment.

Unit tests are close to your code and always should test a single function or small scope of your code. Due to the nature of unit tests, they are really fast, and you can have a lot of them.

Component tests can be seen as a test of the whole service. In general, you test an entire path through your service which can include numerous modules and external services, in our case a database. Therefore, component tests are expensive, as they might take a lot more time. This is the reason you should only have a few, testing the most important paths within your service.

Code example and dependencies

The code shown in this article is publicly available on GitHub. You are welcome to fork the repository and play around or using it as a starting point for your own component tests.

The code example works with a PostgreSQL databases and the lean Knex.js query builder library for the connection and communication to the database. We use Jest to write and run the component tests.

To get started, let's take a closer look at the file structure in our repository.

File structure

We introduced a specific structure in our code base to have a clear separation between unit and component tests.

Every unit test gets the unit as additional tag in the name, e.g., foo-bar.unit.test.ts or foo-bar.unit.test.ts. In contrast, the component tests are named as foo-bar.component.test.ts.

In addition to the differentiation in naming, we have separate places to put the files at. Unit tests are placed next to the tested files and the component tests in a general test folder src/tests.

Take a look at the example, how the repository structure look like.

- src
  - domain
    - foo.ts
    - foo.unit.test.ts
  - tests
    - bar.component.test.ts

Component Tests

First, let's briefly talk about component tests. The idea of component tests is, to test the full path through a service. E.g., from incoming HTTP requests in a controller through some processing logic into the database.
This component tests can only be achieved with an external database running. Testcontainers for node is a useful library to spin up our database in a docker container during the test setup.

Here is an example of a component test file (as mentioned in the beginning, all code snippets are available in the GitHub repository):

// src/tests/bar.component.test.ts
port knex, { Knex } from 'knex';
import path from 'path';
import { PostgreSqlContainer, StartedPostgreSqlContainer } from 'testcontainers';
import { main } from './foo-main';

describe('Component test', () => {
  let container: StartedPostgreSqlContainer;
  let pg: Knex;

  beforeAll(async () => {
    // Use pre-defined container class for postgres table
    container = await new PostgreSqlContainer()
                          .withDatabase('postgres')
                          .withPassword('secret')
                          .withUser('postgres')
                          .start();

    // Set up proper Knex client from created test container
    pg = knex({
      client: 'pg',
      connection: {
        user: container.getUsername(),
        host: container.getHost(),
        database: container.getDatabase(),
        port: container.getPort(),
        password: container.getPassword(),
        ssl: false
      },
      pool: { min: 3, max: 10 },
      migrations: { directory: path.join(__dirname, './database/migrations') },
      debug: false
    });

    // Create extension for auto generated UUID
    await pg.raw('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"');

    // Log version to check database is working
    const version = await pg.raw('select version()');
    console.log('VERSION: ', version.rows[0].version);

    // Run the Knex migration to have proper database schema in place
    await pg.migrate.latest();
  });

  afterAll(async () => {
    await container.stop();
    await pg.destroy();
  });
  
  describe('Test Foo handler', () => {
    it('Creates a list of foos', async () => {
      const elements = ['Foo1', 'Foo2', 'Foo3', 'FooBar'];
      const creationResult = await main(pg, elements);

      const foos = await pg.table('foo').select();

      console.log(foos);

      expect(foos).toMatchObject(creationResult);
    });
  });
});

Let's cut this file down into smaller pieces to understand each part separately.

Setup

You need a container running a Postgres database and a Postgres client, which should be accessible as a single instance throughout the test file.

  let container: StartedPostgreSqlContainer;
  let pg: Knex;

Before running any describe or it block, we set up the Docker container with the Postgres database and the specific client within the beforeAll() function provided by Jest.

In addition to the container and connection setup, we use this step to install the additional UUID extension, which is only to showcase, how you can handle custom extensions from your production environment within the test.

The last important step within the beforeAll() function is to run the migration. You can use the example repository as starting point for migrations, but this article will not explain migrations with Knex.js. All migrations are files placed in the src/database/migrations folder whose path we define during the setup of the client.

beforeAll(async () => {
    // Use pre-defined container class for postgres table
    container = await new PostgreSqlContainer()
                          .withDatabase('postgres')
                          .withPassword('secret')
                          .withUser('postgres')
                          .start();

    // Set up proper Knex client from created test container
    pg = knex({
      client: 'pg',
      connection: {
        user: container.getUsername(),
        host: container.getHost(),
        database: container.getDatabase(),
        port: container.getPort(),
        password: container.getPassword(),
        ssl: false
      },
      pool: { min: 3, max: 10 },
      migrations: { directory: path.join(__dirname, './database/migrations') },
      debug: false
    });

    // Create extension for auto generated UUID
    await pg.raw('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"');

    // Log version to check database is working
    const version = await pg.raw('select version()');
    console.log('VERSION: ', version.rows[0].version);

    // Run the Knex migration to have proper database schema in place
    await pg.migrate.latest();
  });

Currently, we have a single file running all component tests, to re-use the same database. This part might be somehow extracted for re-use in different component test files, if you have ideas how to achieve this without loosing benefits of Jest running in parallel, I am happy about feedback in the GitHub repository!

Teardown

After all test run through or not, we need to stop the external container and destroy the client.

afterAll(async () => {
  await container.stop();
  await pg.destroy();
});

Tests

describe('Test Foo handler', () => {
    it('Creates a list of foos', async () => {
      const elements = ['Foo1', 'Foo2', 'Foo3', 'FooBar'];
      const creationResult = await main(pg, elements);

      const foos = await pg.table('foo').select();

      console.log(foos);

      expect(foos).toMatchObject(creationResult);
    });
  });

In our describe we have a single test which runs the main() function to create Foo elements in the database and receives the result. In a real scenario, we would mimic an incoming HTTP request or other function call. Afterwards, we query the elements directly from the database and check that the returned result is the same as our main() function returns.

This is only a simple example, in a real service it might include much more computation layers and side effects, which can be tested during the component test.

How do we start the tests, especially without running the unit tests? Let's take a look at pattern matching in Jest and how we profit from our previous explained naming pattern.

Run tests

Luckily, Jest supports file pattern matching out of the box. Within the package.json we define multiple scripts (see code snippet below) which can be called via npm e.g., npm run test:unit or npm run test:component. These scripts call Jest according to their name, with a file pattern matching for unit or component.

"scripts": {
  "test:component": "jest component",
  "test:unit": "jest unit",
  "test:all": "jest"
}

Jest automatically runs all *.test.ts files with a unit or component part in the name. This allows us to run different types of tests independent or all at once by not adding any pattern matcher, as we do in the test:all script. That's the trick, how to run specific files according to their test type.

Conclusion

An easy trick by adding specific naming patterns to our test files and the out-of-the-box pattern matching functionality of Jest allows us to separate our tests according to their type.

Having unit and component tests in place leads to a more robust building pipeline and development of the service. The combination of unit and component tests give us a high confidence during refactoring, adding new functionality, that everything is working as expected. You define a contract and can reflect this, in your component test, as long as you test on API level.

The introduction of more complex and expensive component tests makes sense as soon as you have a database included, and some complex functionality. It is necessary to have only a handful of component tests in place, as they are much more expensive in comparison to fast running unit tests.

Published by Nikolas Rist

Visit author page