# Job Queues: Streams vs Lists — Redis Pub/Sub & Streams

Source: https://www.skillbyai.com/en/redis-streams/d-queue

> Build reliable job queues with Streams or with BLMOVE-based lists.

## Two ways to queue work

Before Streams, the classic Redis job queue was a **list**: producers `LPUSH`, workers `BRPOP`. The problem: once a worker pops a job and crashes, the job is gone. The **reliable queue** pattern fixes this with **`BLMOVE`** (formerly `BRPOPLPUSH`): atomically move the job from the main list to a per-worker **processing list**, remove it with `LREM` after success, and have a reaper move jobs back from processing lists of dead workers. It works, but you build the bookkeeping yourself. **Streams with a consumer group** provide that bookkeeping natively: the PEL tracks in-progress jobs, delivery counts and owners, `XAUTOCLAIM` recovers abandoned jobs, several groups can consume the same jobs for different purposes, and history is available for debugging. Libraries such as **BullMQ** (Node.js) build full job systems with retries, delays and priorities on Redis. For new designs, prefer Streams or an established library over hand-rolled list queues.

## List queue and stream queue

A list hands each job out and forgets it; a stream remembers who is working on what.

![Left: a vertical stack with an item moving into a side box labelled by an icon of a worker. Right: a horizontal strip with an attached small ledger table showing owners.](assets/figures/redis-streams/section-6-map.svg) — Figure 6.1 — Reliable list queue versus stream consumer group.

## Reliable list queue with BLMOVE

The job is never only in the worker's memory; it sits in a processing list until removed.

```python
def worker(worker_id):
    processing = f"jobs:processing:{worker_id}"
    while True:
        job = r.blmove("jobs", processing, timeout=5, src="RIGHT", dest="LEFT")
        if job is None:
            continue
        try:
            do_work(json.loads(job))
            r.lrem(processing, 1, job)          # done: remove from processing list
        except Exception:
            r.lmove(processing, "jobs:failed", "LEFT", "LEFT")

# a separate reaper moves jobs from processing lists of dead workers back to "jobs"
# Streams + consumer groups provide this bookkeeping (PEL, XAUTOCLAIM) built in
```

## Do not reinvent a job system

Retries with backoff, delayed jobs, priorities, rate limits and dashboards take months to build well. If you need them, use a mature library on Redis (BullMQ, Sidekiq, RQ, Celery with Redis) or a dedicated broker.

**Quiz:** What problem does BLMOVE solve compared with a plain BRPOP job queue?

- [ ] It makes jobs smaller
- [ ] It removes the need for workers
- [x] It keeps the job in a processing list so it is not lost if the worker crashes
- [ ] It sorts jobs by priority

*Answer:* It keeps the job in a processing list so it is not lost if the worker crashes. Moving atomically to a processing list preserves the job until the worker confirms completion.