Experience with choosing messaging services for projects

Hi Tech Hunters,

I had another scenario or situation from my professional life. It may seem silly, but I used to think about it and do some research on the topic.

I’m guessing you got the point from the title. Today I am discussing the popular messaging services Kafka and RabbitMQ.

When I worked for an aerospace project, we used RabbitMQ for inter-service communication. After three years, I left that company to work on the Bank Project, which used Kafka as its messaging service in the current bank project.

Between the bank and aerospace projects, I attended interviews for a job change.

Interviewers asked questions about Kafka, and I said I didn’t have hands-on experience. even though they asked for differences or major key features between Kafka and RabbitMQ.

I used to give stupid answers all the time. Kafka is a sophisticated messaging service that can handle billions of messages.

After a few days of going over Kafka and implementation, etc., I finally learned how to implement Kafka, but something is still missing.Every time I felt like something was missing, it was a major point.

We understand that we must choose wisely between them or other alternatives based on project requirements.

After going through the blogs and pages, I understood that. There were many notable features, but this is the one I recall most vividly.

In Kafka, messages can be subscribed to by “multi consumers,” which means many consumer types, not many instances of the same one.

In RabbitMQ, messages can be routed to numerous queues depending on the exchange type (such as fanout or topic) and the queue bindings. In each queue, only one consumer of that queue can process the message, but if the message goes to multiple queues, it can be processed by multiple consumers.

When we compared project scenarios, we discovered

1) In the Aerospace Project, it’s an intranet application, and only 600 members will use the application. So waiting for requests won’t create issues with work flow or target dates. As a result, rabbit MQ was used.

2) In banking, the auction model requires some validations and approvals.Requests will go to multiple microservices. So they chose Kafka.

Conclusion :

Actually, RabbitMQ is enough for simple use cases, with low traffic. You have certain benefits, like a priority queue and flexible routing options. But for massive data and high throughput, use Kafka without debate.

If you need a commit log or multiple consumers for the same messages, then go to Kafka because RabbitMQ can’t assist you with it.

Feel free to comment or provide suggestions.