RabbitMQ is an open-source message broker. It is lightweight and easy to deploy on-premises and in the cloud. It runs on many operating systems and cloud environments and provides a wide range of developer tools for the most popular languages.
Let's talk a bit about work queues. Generally, when you code on a local machine, it does the task for you. But what if the task could be granularized and distributed across machines?
Let's talk about an example of a website that does the image/video manipulation and exports the output. It will require quite a bit of resource. One machine is overloaded with multiple tasks for handling multiple jobs, leading to greater resource consumption and high task completion time due to resource blocking or other failures.
In such a scenario, RabbitMQ comes to the rescue. It allows us to create multiple servers and reroute the task to them.
Work Queues with RabbitMQ
Let's install the library packages with npm
We can now create index.ts, consumer.ts, producer.ts and constants.ts files in an src directory according to this article's project configuration. You can proceed anyways.
Let's write up our constants.ts file with the constants we will use through this project. It will contain a port for express where our project will run, a rabbit host and a queue name.
Now let's write up our producer.ts file that utilizes the amqplib package to connect and create a channel. It will also have a process to send the data to the queue.
Now let's write up our consumer.ts file that utilizes the amqplib package to connect and listen to the channel. It will just print the message received for now.
Finally, our index.ts file has dummy data to send, an express server with a route that will call up the producer function to relay that data to the consumers.
Let's compile the typescript
tsc -p tsconfig.json from the project's directory. According to the config from my earlier blog, it will create a dist directory. Now we proceed to run our main file with
node dist/index.js. And run our consumer code in another two terminals, each with
The default index route of our express server calls up our producer's function to send a message to the queue. Which in turn distributes that task/message to the available workers.
We have an incremental index value in our dummy data from index.ts that allows us to track the number and distribution of tasks. Let's call up our index route http://localhost:3000 in the browser five times and see the results.
We can see that the task was evenly distributed among the workers in a round-robin fashion. You can carry out the test with more or fewer consumers and observer the task distribution.
Hold on! This is not the only thing RabbitMQ does. It allows us to further ease the queuing process by providing us with delivery acknowledgement, delivery acknowledgement timeout, dispatching techniques, message durability, fair dispatch, etc., for queues.
You can find more details about all of these processes on their site. Furthermore, they support publishing/subscribing, routing, topics, RPC, etc. You can get started on those from their getting-started page.
Hope you learned something. Subscribe for the upcoming dose of tech blogs. Happy coding!