I spent more time than I would like to admit trying to solve a problem I thought would be standard in the Docker world: passing a secret to Docker build in a CI  environment (GitHub Actions, in my case). This post is not about mounting a file with environment secrets,

We have just released a new major version of BullMQ. And as all major versions it includes some new features but also some breaking changes that we would like to highlight in this post. If you are using Typescript (as we dearly recommend), you will get compiler errors if you

As the communication between microservices increases and becomes more complex, we often have to deal with limitations on how fast we can call internal or external APIs. When the services are distributed and scaled horizontally, we find that limiting the speed while preserving high availability and robustness can become quite

BullMQ has reached version 2.0. Which according to the "SemVer" standard means that we have introduced some breaking changes. It is always an extra effort to deal with breaking changes, so we try to minimize them as much as we can. There are normally 2 types of

Divide and conquer is a well-known programming technique for attacking complex problems in an easier way. In a distributed environment it can be used to accelerate long processes by dividing them into small chunks of work and then joining the results together. BullMQ provides a powerful feature called "flows

A common pattern that arises when working with queues in general and BullMQ in particular, is the one of waiting for a job to complete. The reasoning goes, that if the job is doing something useful we need to wait for it to be complete so that we can get

Writing code using Typescript and BullMQ is quite straightforward most of the time since BullMQ itself is written in Typescript. However there is a special case that has kept people scratching their heads unnecessarily and that is when you want to use sandboxed workers, i.e. a worker that is

When the first version of Bull was released in 2012, it simply aimed at solving a use case we had for Castmill [https://castmill.com], namely we needed queues for processing simple video and image transcoding jobs. Those where the young days of NodeJS but there was a promising package