bullmq

Queue for messages and jobs based on Redis

weekly downloads2.9M

versionv5.69.3

licenseMIT

categorybackend

bullbullmqqueuesjobsredis

Overview

BullMQ is a Node.js library that implements a distributed job queue system using Redis as its data store. It enables you to offload time-consuming operations from your application's main thread by queuing them for asynchronous processing by separate worker processes. This architecture prevents blocking operations from degrading user-facing response times and allows you to scale job processing independently from your web servers.

The package provides sophisticated job scheduling capabilities including delayed execution, recurring jobs via cron patterns, and complex parent-child job dependencies with unlimited nesting. Jobs are persisted to Redis with atomic operations, ensuring durability across server restarts and preventing data loss. BullMQ handles failure scenarios through automatic retry mechanisms with configurable backoff strategies and maintains dead letter queues for jobs that permanently fail.

With over 2.8 million weekly downloads, BullMQ has become a production-standard solution for companies running video transcoding pipelines, AI workloads, and payment processing systems. It supports multi-language interoperability, allowing Python, PHP, or Elixir services to produce jobs that Node.js workers consume, or vice versa. The library provides granular control over concurrency, priorities, rate limiting, and job deduplication, making it suitable for both simple background task queues and complex distributed workflow orchestration.

BullMQ is the spiritual successor to the original Bull library, rewritten from scratch with improved architecture, better TypeScript support, and enhanced features while maintaining the familiar API patterns that developers expect from a job queue system.

Quick Start

typescript

import { Queue, Worker } from 'bullmq';

const connection = {
  host: 'localhost',
  port: 6379
};

const emailQueue = new Queue('email-notifications', { connection });

const worker = new Worker(
  'email-notifications',
  async (job) => {
    const { to, subject, body } = job.data;
    console.log(`Processing email to ${to}`);
    
    await sendEmail({ to, subject, body });
    
    return { sentAt: new Date(), messageId: `msg-${job.id}` };
  },
  {
    connection,
    concurrency: 5,
    limiter: {
      max: 10,
      duration: 1000
    }
  }
);

worker.on('completed', (job) => {
  console.log(`Job ${job.id} completed with result:`, job.returnvalue);
});

worker.on('failed', (job, err) => {
  console.error(`Job ${job.id} failed:`, err.message);
});

await emailQueue.add(
  'welcome-email',
  {
    to: 'user@example.com',
    subject: 'Welcome!',
    body: 'Thanks for signing up'
  },
  {
    attempts: 3,
    backoff: {
      type: 'exponential',
      delay: 2000
    },
    removeOnComplete: 100,
    removeOnFail: 500
  }
);

await emailQueue.add(
  'weekly-digest',
  { templateId: 'digest' },
  {
    repeat: {
      pattern: '0 9 * * MON'
    }
  }
);

async function sendEmail({ to, subject, body }) {
  await new Promise(resolve => setTimeout(resolve, 100));
}

Use Cases

Email and notification delivery: Queue outbound emails, SMS messages, and push notifications to process them asynchronously without blocking API responses. Apply rate limits to avoid triggering spam filters or exceeding third-party service quotas, and automatically retry failed deliveries with exponential backoff.

Media processing pipelines: Handle video transcoding, image resizing, and thumbnail generation by breaking large files into jobs. Use parent-child dependencies to create multi-stage pipelines where a video upload job spawns separate encoding jobs for different resolutions, each processed in parallel by available workers.

Scheduled maintenance tasks: Run database cleanups, report generation, and data synchronization on recurring schedules using cron patterns. Jobs persist across deployments and server restarts, ensuring scheduled tasks execute reliably even during maintenance windows.

API rate limiting and integration: Queue requests to third-party APIs that enforce rate limits (like payment processors, social media APIs, or data enrichment services). BullMQ's built-in rate limiting ensures you never exceed per-second or per-minute quotas while maximizing throughput.

Webhook processing and event handling: Handle incoming webhooks from external services by immediately queuing them for processing. This pattern ensures you respond quickly to webhook requests (avoiding timeouts) while performing validation, business logic, and database operations asynchronously with proper error handling and retry logic.

Pros & Cons

Pros

+Battle-tested reliability with atomic Redis operations ensuring no job loss, automatic retries with backoff, and persistent job state across crashes
+Advanced scheduling features including cron-based recurring jobs, millisecond-precision delays, parent-child dependencies, and failure propagation strategies
+Comprehensive monitoring and observability through built-in events, metrics, and integration with bull-board dashboard for real-time queue inspection
+Production-grade performance with configurable concurrency, priority queues, rate limiting per queue or job group, and job deduplication for debounce patterns
+Multi-language ecosystem allowing producers and consumers written in different languages to interoperate through shared Redis backend

Cons

−Requires Redis infrastructure which adds operational complexity, costs, and another potential failure point in your system architecture
−Learning curve for complex features like job dependencies, flow builders, and advanced retry strategies can be steep for teams new to distributed job queues
−Performance depends heavily on Redis configuration and network latency between application servers and Redis instances, requiring tuning for high-throughput scenarios
−Memory usage grows with job retention policies and large job data payloads stored in Redis, requiring careful capacity planning and monitoring
−Breaking changes between major versions (Bull to BullMQ migration) require code updates and careful migration planning for existing production systems

Comparisons

bullmq vs p-queue

backend

Install

bash

npm install bullmq

bash

pnpm add bullmq

bash

bun add bullmq

Overview

Quick Start

typescript

import { Queue, Worker } from 'bullmq';

const connection = {
  host: 'localhost',
  port: 6379
};

const emailQueue = new Queue('email-notifications', { connection });

const worker = new Worker(
  'email-notifications',
  async (job) => {
    const { to, subject, body } = job.data;
    console.log(`Processing email to ${to}`);
    
    await sendEmail({ to, subject, body });
    
    return { sentAt: new Date(), messageId: `msg-${job.id}` };
  },
  {
    connection,
    concurrency: 5,
    limiter: {
      max: 10,
      duration: 1000
    }
  }
);

worker.on('completed', (job) => {
  console.log(`Job ${job.id} completed with result:`, job.returnvalue);
});

worker.on('failed', (job, err) => {
  console.error(`Job ${job.id} failed:`, err.message);
});

await emailQueue.add(
  'welcome-email',
  {
    to: 'user@example.com',
    subject: 'Welcome!',
    body: 'Thanks for signing up'
  },
  {
    attempts: 3,
    backoff: {
      type: 'exponential',
      delay: 2000
    },
    removeOnComplete: 100,
    removeOnFail: 500
  }
);

await emailQueue.add(
  'weekly-digest',
  { templateId: 'digest' },
  {
    repeat: {
      pattern: '0 9 * * MON'
    }
  }
);

async function sendEmail({ to, subject, body }) {
  await new Promise(resolve => setTimeout(resolve, 100));
}

Use Cases

Pros & Cons

Pros

+Battle-tested reliability with atomic Redis operations ensuring no job loss, automatic retries with backoff, and persistent job state across crashes
+Advanced scheduling features including cron-based recurring jobs, millisecond-precision delays, parent-child dependencies, and failure propagation strategies
+Comprehensive monitoring and observability through built-in events, metrics, and integration with bull-board dashboard for real-time queue inspection
+Production-grade performance with configurable concurrency, priority queues, rate limiting per queue or job group, and job deduplication for debounce patterns
+Multi-language ecosystem allowing producers and consumers written in different languages to interoperate through shared Redis backend

Cons

−Requires Redis infrastructure which adds operational complexity, costs, and another potential failure point in your system architecture
−Learning curve for complex features like job dependencies, flow builders, and advanced retry strategies can be steep for teams new to distributed job queues
−Performance depends heavily on Redis configuration and network latency between application servers and Redis instances, requiring tuning for high-throughput scenarios
−Memory usage grows with job retention policies and large job data payloads stored in Redis, requiring careful capacity planning and monitoring
−Breaking changes between major versions (Bull to BullMQ migration) require code updates and careful migration planning for existing production systems