Redis Memory Management for BullMQ: maxmemory, Eviction Policies & Production Best Practices

It's Sunday evening and your pager goes off. Redis is returning OOM errors. Your queue workers have stalled. Jobs that were supposed to process hours ago are simply gone. This scenario — an all-too-common production incident for teams running BullMQ — stems from one root cause: poor Redis memory management.

Redis is the backbone of BullMQ, storing every job state, worker heartbeat, and rate-limit bucket. When Redis runs out of memory, your queue doesn't just slow down — it breaks. This guide covers everything you need to know about Redis memory management for BullMQ: configuring maxmemory, selecting the right eviction policy, optimizing job storage, and monitoring memory before it becomes a crisis.

---

Why Memory Management Matters for Queue Systems

Redis is not "just a cache" for queues

Many teams treat Redis as a cache that can gracefully lose data. That assumption is dangerous when Redis powers a job queue. Unlike a cache where a few lost keys mean a slightly stale page, a queue that loses keys loses real work.

BullMQ relies on Redis as a durable data store for job state. It maintains sorted sets for waiting, active, completed, failed, and delayed jobs, plus hashes for individual job data, lists for worker heartbeat tracking, and more. All of these must persist for queue integrity.

When Redis evicts a single key that BullMQ did not expect to lose, the consequences cascade:

• Delayed job keys evicted: Jobs vanish silently — they never execute, and no error is thrown.
• Worker heartbeat keys evicted: BullMQ's stalled-job detection uses keys like bull:queue:worker:<id>:heartbeat. When those keys disappear, BullMQ assumes the worker died and triggers re-processing, potentially causing duplicate execution.
• Rate-limiter keys evicted: Rate limits are bypassed, allowing bursts that downstream systems can't handle.

Consequences of poor memory management

Let's break down what actually happens when Redis memory management fails in a BullMQ environment:

• Lost jobs: Evicted job data means work never executes and never appears in failure logs. These are silent data loss events — the hardest kind to debug.
• Stalled jobs: BullMQ's stall detection relies on worker heartbeat keys with periodic updates. If Redis evicts these keys under memory pressure, BullMQ marks workers as stalled and attempts to re-process their active jobs. This can lead to duplicate processing or, in edge cases, permanent failures if the re-processing logic isn't idempotent.
• OOM write errors: When maxmemory is reached with the noeviction policy (the only correct choice for BullMQ), Redis refuses all write commands. Your queue.add() calls throw errors, workers can't update job status, and the entire queue system grinds to a halt.
• Performance degradation: Even before hitting the limit, high memory pressure causes Redis to swap, increasing latency for all queue operations — especially BRPOPLPUSH-style moves and rate-limit checks that BullMQ relies on for throughput.

Key metrics to watch: evicted_keys (from INFO stats), keyspace_hits/keyspace_misses, and the used_memory / maxmemory ratio.

---

Redis maxmemory — What It Is and How to Set It

Understanding maxmemory

maxmemory is the Redis configuration directive that defines the upper memory limit for data. On 64-bit systems, the default is 0 — meaning unlimited. This is dangerous for production. Without a limit, Redis will consume all available system RAM, triggering the kernel's OOM killer, which can kill Redis (and possibly other processes on the same machine).

Best practice: Set maxmemory to approximately 70–80% of your available system RAM. The remaining headroom accommodates:

• Replication buffers (for replicating data to replicas)
• AOF rewrite buffers (during background persistence operations)
• OS page cache
• Memory fragmentation

How to configure maxmemory

You can set maxmemory in your redis.conf file or at runtime:

# In redis.conf (persistent across restarts)
maxmemory 2gb
maxmemory-policy noeviction

# At runtime (applies immediately, lost on restart)
redis-cli CONFIG SET maxmemory 2gb
redis-cli CONFIG SET maxmemory-policy noeviction

# Persist runtime changes
redis-cli CONFIG REWRITE

What happens when maxmemory is reached

This is a critical detail: Redis checks memory before executing write commands, not proactively in the background. When memory is at or above maxmemory, Redis evaluates the eviction policy. With noeviction, it returns an error immediately for any write command.

One important nuance: a single large pipeline or MULTI/EXEC transaction can push Redis past the maxmemory limit before eviction (or OOM rejection) kicks in for individual commands. Redis checks at the start of each command, so if your pipeline contains 1,000 writes and each one pushes memory a little higher, the last few may fail. This is why leaving headroom matters.

See it in action with redis-cli

# Check current maxmemory and usage
redis-cli INFO memory | grep -E "maxmemory|used_memory"

# Sample output:
# used_memory: 1073741824
# used_memory_human: 1.00G
# maxmemory: 2147483648
# maxmemory_human: 2.00G
# maxmemory_policy: noeviction

---

Redis Eviction Policies — A Complete Breakdown

Redis ships with eight eviction policies, but for BullMQ queues, only one matters. Let's understand all of them and why each one (except noeviction) is unsafe for queue workloads.

Policy categories

The policies fall into three categories:

• noeviction — Return an error on write commands when the memory limit is reached. No data is automatically removed.
• allkeys-* — Evict keys from the entire keyspace, regardless of whether they have a TTL set.
• volatile-* — Evict only keys that have an explicit TTL/expiration set. Keys without TTLs are safe from eviction — but they can still cause writes to fail.

All available policies

Policy	Scope	Eviction Target	Best For
noeviction	—	No eviction	BullMQ queues (required)
allkeys-lru	All keys	Least recently used	General-purpose cache
allkeys-lfu	All keys	Least frequently used	Hot/cold access patterns
allkeys-random	All keys	Random	Equal-access workloads
volatile-lru	Expiring keys	Least recently used	Mixed cache + persistent data
volatile-lfu	Expiring keys	Least frequently used	Mixed with access patterns
volatile-random	Expiring keys	Random	Mixed with equal access
volatile-ttl	Expiring keys	Shortest remaining TTL	When TTLs are good eviction hints

How LRU approximation works

Redis does not use a true LRU algorithm (which would require tracking every key access in a linked list, consuming significant memory). Instead, it uses an approximated LRU: it samples a configurable number of keys (maxmemory-samples, default 5) and evicts the oldest among the sampled set.

For higher eviction accuracy, you can increase maxmemory-samples to 10, which adds roughly 1% CPU overhead but significantly improves the quality of eviction decisions.

LFU — when access frequency matters

Redis also supports LFU (Least Frequently Used) eviction via allkeys-lfu and volatile-lfu. Instead of tracking when a key was last accessed, LFU tracks how often it's accessed using a probabilistic Morris counter that saturates logarithmically. This is useful for workloads with clear hot/cold access patterns, but again — not for BullMQ queues.

LRM (Redis 8.6+) — least recently modified

Redis 8.6 introduced LRM (Least Recently Modified), which only updates eviction timestamps on writes, not reads. This is beneficial for read-heavy cache workloads — frequently read but never modified data won't "look young" and prevent eviction of stale, written data. As of this writing, LRM is available in Redis 8.6+ and Valkey-compatible forks.

---

The One Rule for BullMQ — Always Use noeviction

Why noeviction is mandatory

This is the single most important configuration rule for BullMQ: you must set maxmemory-policy to noeviction. The BullMQ production guide states it directly: "This is the only setting that guarantees the correct behavior of the queues."

Why is this so strict? Because BullMQ stores the vast majority of its keys without TTLs. Job data in waiting, active, delayed, completed, and failed sets is meant to persist until explicitly removed by removeOnComplete/removeOnFail or manual cleanup. If an LRU eviction policy decides that a waiting job key is "least recently used" and evicts it, that job simply disappears — no error, no recovery.

Furthermore, BullMQ's internal keys are equally vulnerable:

• Worker heartbeats: bull:queue:worker:<id>:heartbeat — evict these and BullMQ falsely detects stalled workers.
• Lock keys: bull:queue:job:<id>:lock — losing these can cause duplicate processing.
• Rate-limiter buckets: bull:queue:limiter — losing them bypasses rate limits.
• Delay stream markers: Used for scheduling delayed jobs — eviction silently drops delayed work.

What happens if you use an eviction policy with BullMQ

Let's examine three real scenarios:

• Scenario A (`allkeys-lru']): A completed job's result is evicted because the queue adds new jobs. The result is "least recently used" from Redis's perspective, but it's still valuable data that your application expects to find.
• **Scenario B (volatile-ttl']):** Since BullMQ doesn't set TTLs on most job keys, volatile-ttleffectively behaves likenoeviction` — until you add some keys with TTLs (say, a custom cache alongside your queue). Then Redis evicts only the TTL'd keys until none remain with TTLs, at which point it starts returning OOM errors. You get a false sense of security until it's too late.
• Scenario C (`allkeys-random']): Complete chaos. Any key can vanish at any time. Heartbeat keys, job data, lock keys — all equally likely to be evicted. This is the fastest path to a corrupted queue state.

Code example — verifying your Redis config programmatically

import IORedis from 'ioredis';

async function checkRedisMemoryConfig(connection: IORedis) {
  const info = await connection.info('memory');
  const maxmemory = info.match(/maxmemory:(\d+)/)?.[1];
  const policy = info.match(/maxmemory_policy:(\w+)/)?.[1];

  console.log(`maxmemory: ${Number(maxmemory) / 1024 / 1024} MB`);
  console.log(`eviction policy: ${policy}`);

  if (policy !== 'noeviction') {
    console.warn(
      '⚠️  WARNING: maxmemory-policy is not "noeviction".',
      'BullMQ requires noeviction to prevent job data loss.'
    );
  }

  return { maxmemory: Number(maxmemory), policy };
}

// Use it during application startup
const connection = new IORedis({ host: 'localhost', port: 6379 });
checkRedisMemoryConfig(connection).then((config) => {
  if (config.policy === 'noeviction') {
    console.log('✅ Redis memory config is safe for BullMQ');
  }
});

---

Memory Optimization Strategies for BullMQ Queues

Setting noeviction prevents data loss, but it also means you must actively manage memory usage. Here are proven strategies to keep your Redis memory under control.

1. Job auto-removal (removeOnComplete / removeOnFail)

By default, BullMQ keeps completed and failed jobs forever. In a busy production system, this leads to unbounded memory growth — the most common cause of Redis OOM errors in queue systems.

Always configure removeOnComplete and removeOnFail, either per-job or at the queue level:

import { Queue, Worker } from 'bullmq';

// Option A: Configure at the queue level (BullMQ v5+)
const emailQueue = new Queue('email', {
  connection: { host: 'localhost', port: 6379 },
  defaultJobOptions: {
    removeOnComplete: { age: 3600, count: 1000 },   // Keep 1 hour, max 1000
    removeOnFail: { age: 86400, count: 5000 },       // Keep 24 hours, max 5000
  },
});

// Option B: Configure per job (overrides queue defaults)
await emailQueue.add(
  'send-newsletter',
  { to: 'user@example.com', template: 'weekly-digest' },
  {
    removeOnComplete: { age: 7200, count: 500 },    // Keep 2 hours, max 500
    removeOnFail: { age: 48 * 3600, count: 2000 },  // Keep 48 hours, max 2000
  }
);

// Option C: Retroactively clean existing data
const cleanedCount = await emailQueue.clean(3600, 1000, 'completed');
console.log(`Cleaned ${cleanedCount} old completed jobs`);

Best practice: Keep a small number of completed jobs for visibility (100–1000), keep more failed jobs for debugging (5000+). The age parameter adds a time-based safety net.

2. Minimize job payload size

Redis memory is expensive — 1 MB of job payload × 1 million jobs = 1 TB of RAM. Store large data externally and pass only a reference or ID in the job payload:

// BAD: Storing large data in the job payload
await queue.add('process-report', {
  reportId: 'rpt_123',
  reportData: { /* 10 MB of generated report data */ },
});

// GOOD: Storing a reference to external storage
await queue.add('process-report', {
  reportId: 'rpt_123',
  s3Key: 'reports/2026/06/rpt_123.json',
  s3Bucket: 'my-app-reports',
});

A real-world case from the BullMQ issue tracker (#2734) reported 10 GB of memory consumed by 4.5 million delayed jobs. The fix was reducing payload size and tightening retention policies.

3. Monitor queue depth and growth trends

A growing "delayed" or "waiting" queue may indicate a bottleneck — not just a memory problem. Set up alerts on queue depth crossing thresholds to catch issues before they escalate into memory emergencies.

4. Use proper data types and TTLs where appropriate

BullMQ stores job data as Redis Hashes — compact but not free. If you store custom data alongside queues (e.g., caching worker configurations), consider setting TTLs to auto-expire stale entries. Be careful not to set TTLs on BullMQ's own keys — the library manages their lifecycle internally.

5. Deduplicate and debounce high-frequency jobs

During traffic spikes, the same event can trigger thousands of identical jobs. BullMQ Pro offers built-in deduplication, and BullMQ open source supports debouncing. These features prevent queue bloat during bursts and protect Redis memory from redundant job data.

---

Monitoring Redis Memory Usage

Proactive monitoring is your safety net. Here's how to track Redis memory usage effectively.

Using INFO memory

# Key metrics to watch
redis-cli INFO memory | grep -E \
  "used_memory|used_memory_rss|used_memory_peak|maxmemory|mem_fragmentation_ratio"

Metric	What It Tells You
used_memory	Total bytes allocated by Redis for data
used_memory_rss	Actual physical memory used (OS-reported)
used_memory_peak	Highest memory usage since Redis started
mem_fragmentation_ratio	used_memory_rss / used_memory (>1.5 indicates fragmentation)
maxmemory	Configured limit
evicted_keys	Total evicted keys (should be 0 with noeviction)

Using MEMORY USAGE to inspect specific keys

When you need to find which BullMQ keys are consuming the most memory, use the MEMORY USAGE command:

# Check how much memory a particular BullMQ key consumes
redis-cli MEMORY USAGE "bull:email:231"            # Specific job
redis-cli MEMORY USAGE "bull:email:waiting"        # Waiting list
redis-cli MEMORY USAGE "bull:email:completed"      # Completed set

Code example — finding the biggest BullMQ keys programmatically

import IORedis from 'ioredis';

async function findLargeKeys(
  connection: IORedis,
  pattern: string,
  threshold: number = 1024 * 1024  // 1 MB
) {
  const stream = connection.scanStream({ match: pattern, count: 1000 });
  const largeKeys: { key: string; bytes: number }[] = [];

  for await (const keys of stream) {
    for (const key of keys) {
      const usage = await connection.memory('USAGE', key);
      if (usage && usage > threshold) {
        largeKeys.push({ key, bytes: usage });
      }
    }
  }

  return largeKeys.sort((a, b) => b.bytes - a.bytes);
}

// Usage
const connection = new IORedis({ host: 'localhost', port: 6379 });

findLargeKeys(connection, 'bull:*').then((keys) => {
  keys.forEach(({ key, bytes }) => {
    console.log(`${key}: ${(bytes / 1024 / 1024).toFixed(2)} MB`);
  });
});

Setting up alerts (the 80% rule)

Alert when used_memory / maxmemory > 0.8. Why 80%? The remaining 20% provides headroom for:

• Replication buffers during sync operations
• AOF rewrite temporary memory
• Burst traffic and pipeline commands
• Memory fragmentation overhead

Tools you can use: RedisInsight, Datadog, Prometheus + redis_exporter, or QueueHub's built-in monitoring.

How QueueHub (Queue Hub) helps

QueueHub provides purpose-built monitoring for BullMQ that gives you visibility into Redis memory:

• Live memory usage dashboard: Real-time view of used_memory, maxmemory, and usage percentage per Redis backend.
• Queue-level memory insights: Per-queue job counts (waiting, active, delayed, completed, failed) to identify which queue is consuming memory.
• Job detail inspection: View job payload size directly to debug oversized jobs.
• Multi-backend comparison: Compare memory across environments (staging, prod, per-region) when running multiple Redis instances.
• Agent tunneling for private Redis: Get memory metrics even when Redis is behind a firewall or VPC.
• Team alerts (planned/roadmap): Slack and email notifications when queue memory crosses thresholds.

---

Best Practices Checklist — Redis Memory for Production Queues

1. Set maxmemory to ~70–80% of system RAM.
2. Set maxmemory-policy noeviction — this is non-negotiable for BullMQ.
3. Configure removeOnComplete and removeOnFail on every job or queue default.
4. Minimize job payloads — store large data externally; pass references only.
5. Monitor evicted_keys — with noeviction, this must always be 0.
6. Watch mem_fragmentation_ratio — if > 1.5, consider restarting Redis or enabling activedefrag yes.
7. Set up 80% memory alerts — catch problems before writes are rejected.
8. Use MEMORY USAGE to find oversized keys when troubleshooting.
9. Plan for peak queue depth — delayed jobs can pile up silently during outages.
10. Use a monitoring dashboard (QueueHub, RedisInsight, or Prometheus/Grafana) for visual observability.

---

Common Pitfalls and FAQs

"I set noeviction but Redis is still returning OOM errors"

Check that maxmemory is high enough for your peak queue depth plus overhead from replication buffers and AOF. Use INFO memory and check mem_not_counted_for_evict to estimate buffer size. You may need to raise maxmemory or reduce memory consumption with the strategies above.

"My completed set is consuming all the memory"

You likely forgot to configure removeOnComplete. Configure it immediately using defaultJobOptions on the queue constructor. Use queue.clean() to retroactively purge old completed jobs.

"Can I use volatile-lru if I set TTLs on my BullMQ keys?"

No. BullMQ does not set TTLs on most of its internal keys. volatile-lru will behave like noeviction until Redis runs out of keys with TTLs to evict, then it starts returning errors. But by that point, your queue data is already at risk from whatever TTL'd keys were evicted. Stick with noeviction.

"What about Redis on Kubernetes / ElastiCache / Valkey?"

• Kubernetes: Ensure the Redis pod's memory limit (in the container spec) aligns with (or exceeds) your maxmemory setting. If the pod is OOM-killed before Redis hits maxmemory, your queue loses all in-memory data on restart.
• Amazon ElastiCache: The default maxmemory-policy is volatile-lru. You must change this via a parameter group to noeviction.
• Valkey: Valkey uses the same configuration syntax as Redis. Set maxmemory-policy noeviction — it works identically.

---

Conclusion

Redis memory management is not optional for queue systems — it's foundational. A properly configured Redis instance with noeviction, adequate maxmemory, and active monitoring is the difference between a reliable job queue and a production incident waiting to happen.

The three pillars of Redis memory management for BullMQ are:

1. Configuration: maxmemory-policy noeviction + appropriate maxmemory limit.
2. Hygiene: removeOnComplete/removeOnFail, minimal payloads, and debouncing high-frequency jobs.
3. Visibility: Proactive monitoring of memory usage, evicted keys, and queue depth.

Don't wait for a Sunday evening pager alert to audit your Redis configuration. Check your maxmemory-policy today, set up job retention, and implement the monitoring practices covered in this guide.

Ready to take control of your BullMQ queue observability? QueueHub gives you the real-time visibility you need to stay ahead of memory issues across all your Redis backends. Start with a free account at queuehub.tech and see your queues like never before.