MANIK MEHTA

Setting up separate Testing Instances of Redis and RabbitMQ for Node Application: Multi Networking in Docker

Manik Mehta's photo
Manik Mehta
·

6 min read

You might be using Docker to set up containerized open-source projects, and let me tell you, there is one more advantage (but not the last!) of containerizing your project: making it ready for testing. In today's blog, we will be containerizing my own project, which you can find on GitHub Link. The Dockerfile and docker-compose file can be obtained from this repository. Let's first understand our need.

The Need

While working on my project, like any other developer, I initially neglected writing tests. However, someone advised me that if I wanted to learn to write good code, I should make my code testable and write tests for it. This would elevate my learning to the next level, ultimately resulting in code that closely resembles production-level quality. Motivated by this advice, I began writing tests for my server.

Things were progressing smoothly until a new challenge emerged: creating a separate testing instance for Redis and RabbitMQ. Redis offers an option to start more than one instance on the same machine, but luckily:), RabbitMQ does not. I tackled the Redis-related issue within a day by adding some scripts to the package.json file. Take a look at the changes I made to the file:

 "start-testing-redis": "redis-server --port 6300",
 "prepare-server": "wait-on tcp:localhost:6300 && vitest",
 "test": "npm run start-testing-redis & npm run prepare-server"

When npm test is run then this script run a Redis server on port 6300 (you can select any other port except the default port 6379), then wait-on for it's starting (this is a package given by node) and then run the tests.

I proceeded to make the necessary changes in the code to connect to this Redis server when testing. That's how I established the testing instance for Redis. However, the question remained: What about RabbitMQ?

In my quest to find a method for creating a testing instance of RabbitMQ, I encountered various suggestions, with most advising to pull a Docker image of RabbitMQ and use that image for testing. Despite this, I decided to take it a step further by fully containerizing my server. This way, any developer interested in setting up this project locally wouldn't have to worry about these problems.

Let's conclude this section by summarizing our needs:

A single Docker Image should be able to run server and tests simultaneously with separated instances of all the services for running server and tests!

My First Approach

I was unaware of the concept of Multi Networks in Docker, as it was not emphasized in most tutorials (I don't even know whether this is actually a topic or not!). Initially, my approach involved running a single network with multiple containers of the same service. However, this approach led to the same problem. Let's delve into the issue:

There are two ports in Docker: Host and Container port. These two ports are not related very closely. Host port is the port which by will be bind to your Machine by Docker and Container port is that port on which your service will be running inside Container but remember that in a Single Network you can't use the same Container port for two different services! (You can visualize the network in Docker same a Virtual Machine)

You might be wondering: Why is it even a problem? Well, it's a bit complicated to change the default ports of Redis and RabbitMQ (although it's easy to change that of Redis) because it involves modifying the .conf file of both services every time the container is started. Believe it or not, I spent 15 days attempting to start the Testing Server on a different port, and I was still failing. Then, I discovered the concept of Communicating between Multi-Networks in Docker, and surprisingly, it solved my problem within a day!

Actual Approach: Multi Networking

Let me quote the solution of above problem:

We can't bind two servers to same port in a Single Network! What about using more than one network?

See the given docker-compose.yaml file

version: "3"
services:
  web:
    build: 
      context: .
      dockerfile: Dockerfile
    user: "node:node"
    ports:
      - "3000:3000"
    depends_on:
      - redis
      - rabbitmq
      - redis-testing
      - rabbitmq-testing
    networks:
      - mainnetwork
      - testingnetwork  
  redis:
    image: redis:alpine
    container_name: client
    ports:
      - "6390:6379"
    command: ["redis-server", "--bind", "redis", "--port", "6379"]
    networks:
      - mainnetwork
  rabbitmq:
    image: rabbitmq
    container_name: rabbit
    ports:
      - "5674:5762"
      - "15674:15672"
    networks:
      - mainnetwork
  redis-testing:
    image: redis:alpine
    command: ["redis-server", "--bind", "0.0.0.0", "--port", "6379"]
    ports:
      - "6392:6379"
    container_name: client-testing
    networks:
      - testingnetwork
  rabbitmq-testing:
    image: rabbitmq
    ports:
      - "5673:5672"
      - "15673:15672"
    container_name: rabbit-testing  
    networks:
      - testingnetwork 
networks:
  mainnetwork:
    driver: bridge
  testingnetwork:
    driver: bridge

Although this file is self-explanatory but still let me explain you the overview of this file:

  • The DockerFile (which is also located in the project) should have two networks: mainnetwork and testingnetwork

  • The services which this file is using are kind of then declared and initialized later.

  • The Redis is running on default Container Port and host port can be any free port.

  • The command which Redis service is using actually binds the server with it's default port (not using this command will give errors!)

  • Similarly Redis testing, RabbitMQ main and testing services are instantiated on their respective Container and Host Ports.

  • The 0.0.0.0 address used in redis-testing signifies usage of Redis Server from any network.

But how will your application then connect to these containers? And how will your application identify whether the app is running in Docker or on Actual (Virtual also) machine because both requires little bit different method to connect to these services. I solved the second problem by asking it from developer in the .env file! and then later on integrating it with my node app!

 // in .env file
// RUNNING_ON_DOCKER=false
// in node app 
export const IfRunningOnDocker=process.env.RUNNING_ON_DOCKER!

Connecting to Node App: Significance of Container Names

For RabbitMQ testing channel:

import amqp from 'amqplib/callback_api'

export const createTestingChannel=(callback: (chnl: amqp.Channel)=>void)=>{
// Not explaining the function here!
// Just focus on the first parameter of this function, the connection string
// To connect to the Container running on 5672 port on host we use this:
// amqp://container-name:hostport     
amqp.connect('amqp://rabbit-testing:5672', async function(error: any, connection: amqp.Connection){
        if(error){
            console.log(error);
        }   
        try{
        connection.createChannel((err, channel)=>{
            callback(channel);
        })
       }
       catch(e: any){
        console.log(e.toString());
       }

    })
}

Similarly with Redis Testing client:

import {createClient} from 'redis';
const typeClient = createClient()
export type RedisClientType = typeof typeClient 
// import the IfRunningOnDocker
const connectionUri:string=ifRunningOnDocker=='true'?'redis://client-testing:6379':'redis://localhost:6300'
export const createRedisInstance=async():Promise<RedisClientType|null>=>{
  try{
        const testingClient=createClient({
        url:connectionUri
    })
        await testingClient.connect()
        console.log('Redis running')
        return testingClient
  }catch(e:any){
    console.log('Error in redis server'+e.toString())
    return null
  }
}
export const closeRedisServer=async(redis: RedisClientType): Promise<void>=>{
    await redis.flushDb()
    await redis.quit()
  }

This way we can run Multiple Testing and Main instances for our server.

I hope you enjoyed this blog! Do leave a feedback in the form of comment!

DockerDevopskunalkushwahaWeMakeDevsRedisrabbitmqTypeScriptNode.jsJavaScriptDocker composedocker-network