pg-boss in Production: Reliable Background Jobs for Node.js

Some backend work does not belong in an HTTP request.

Imports, exports, data synchronization, report generation, image processing, notification batches, and other long-running tasks can make an API slow or fragile when they execute in the request path. The user should not have to keep a connection open while the server performs several minutes of work.

In one high-volume production Node.js backend, we used pg-boss to move that work into PostgreSQL-backed queues. It was remarkably reliable. It also taught us an important lesson: a dependable queue does not remove the need for dependable job code.

The queue could retry an interrupted job. Our code had to make sure that retrying it did not charge, send, create, or update something twice.

Why pg-boss was a good fit

pg-boss is a background job system for Node.js that uses PostgreSQL as its storage and coordination layer. If an application already depends on PostgreSQL, that is a compelling operational advantage.

We did not need to add another major piece of infrastructure just to process jobs. The queue lived in a database our team already knew how to monitor, query, back up, and operate. Job state was durable, and it was possible to inspect what was queued, active, completed, or failed using familiar database tools.

pg-boss uses PostgreSQL's SKIP LOCKED behavior so multiple workers can safely fetch work without claiming the same available job at the same time. It also provides practical queue features such as:

• Delayed and scheduled jobs
• Automatic retries and retry backoff
• Configurable worker concurrency
• Dead-letter queues
• Priorities
• Rate limiting and debouncing patterns
• Transactional job creation

That last capability is especially useful. When application data and the queue share PostgreSQL, a business change and the job it requires can be committed together. This avoids a common failure mode where the application saves a record but crashes before publishing the corresponding job.

PM2 kept background work away from web traffic

We ran the Node.js backend with PM2. The web application and background processing could run in separate managed processes, so expensive work did not have to compete with incoming API requests in the same event loop.

That separation mattered. A request could validate input, persist the necessary state, enqueue a job, and respond quickly. A worker process could then perform the slower work independently.

The shape was simple:

1. The API received a request.
2. It validated and saved the request data.
3. It queued a pg-boss job with a stable business identifier.
4. A separate worker claimed and processed the job.
5. The worker recorded the result, failure, or next retry.

This is a useful boundary even when the workload is not enormous. It protects response times, makes concurrency easier to control, and gives operations a place to inspect failed work. For integrations, it also complements business-outcome monitoring: the API being online is not enough if the queue stops draining.

The difficult part was deployment, not normal processing

Under normal conditions, the system was dependable. The more interesting failures happened during frequent application updates.

A deployment could restart a PM2 process while a long job was still running. The worker might have already completed part of the job but not yet reported completion to pg-boss. Later, the job could be retried. If a new scheduled run started around the same time, two logically related jobs could overlap.

Consider a scheduled synchronization that processes thousands of records:

• The first run updates 70 percent of the records.
• A deployment interrupts the worker.
• pg-boss eventually makes the unfinished job eligible for retry.
• The next scheduled run begins before that retry finishes.
• Both jobs encounter some of the same records.

The queue is working as intended. It did not lose the interrupted work. But the application now has to handle repeated and overlapping attempts safely.

Exactly-once delivery is not exactly-once business behavior

pg-boss describes its job delivery as exactly once because PostgreSQL locking prevents multiple workers from concurrently claiming the same available job record. That is valuable, but it is not a guarantee that every external side effect happens exactly once.

There is always a vulnerable boundary:

1. A worker calls an external API or changes business data.
2. That action succeeds.
3. The process exits before recording job completion.
4. The queue retries the job.

From pg-boss's perspective, the first attempt never finished. From the external system's perspective, the action may already have happened.

This is why production job handlers should be designed for at-least-once execution, even when the queue has strong delivery guarantees.

Make every job idempotent

An idempotent job can run more than once without producing a different final result. In practice, that usually requires stable identifiers and explicit state.

Instead of telling a job to “create another shipment,” tell it to “ensure shipment X exists for order Y.” Instead of “send a payment,” use the same idempotency key for every attempt. Instead of “increment the synced count,” calculate or upsert the desired state.

Useful patterns include:

• Put a business identifier in every job payload, not just a generated queue ID.
• Add database uniqueness constraints for effects that must occur once.
• Prefer upserts and state transitions over blind inserts.
• Pass idempotency keys to external services that support them.
• Check the destination system before recreating a record after an uncertain failure.
• Store checkpoints for large jobs so a retry can resume or safely revisit completed items.
• Record attempts and outcomes with enough context to reconcile ambiguous results.

A database marker by itself is not always sufficient. If code writes the marker before calling an external service, it can suppress a needed retry after the call fails. If it writes the marker after the call, the process can exit between the successful call and the marker. The strongest solution is usually an idempotency key honored by the destination. When that is unavailable, use a reconciliation step that can ask the destination what actually happened.

This is a broader systems integration principle: retries are only safe when both sides agree on the identity of the operation.

Give scheduled runs their own identity

Recurring jobs need protection at two levels.

First, identify the scheduled run itself. A daily job might use a key such as inventory-sync:2026-08-11. An hourly job might include the hour and timezone. That prevents the scheduler, a manual replay, and a retry from accidentally creating multiple logical copies of the same run.

Second, make each item processed by the run idempotent. A run-level lock reduces overlap, but it does not protect against a process dying halfway through. Item-level safety is what makes the recovery boring.

It also helps to decide explicitly what should happen when one interval takes longer than the next:

• Skip the new run while the previous one is active.
• Queue the new run and let it wait.
• Merge the intervals into one catch-up run.
• Allow both to run because their records are safely partitioned.

There is no universal answer. The wrong answer is allowing overlap accidentally.

Treat shutdown as part of job processing

PM2 supports graceful shutdown, but the application has to participate. Its documentation recommends handling the shutdown signal and cleaning up active resources before the process exits.

For a worker, graceful shutdown usually means:

• Stop accepting new jobs.
• Allow active jobs to finish within a defined window.
• Close pg-boss and database connections cleanly.
• Configure PM2's kill timeout to match the shutdown strategy.
• Assume the process can still be killed before cleanup completes.

That final assumption is important. Graceful shutdown reduces interrupted jobs; it cannot eliminate them. A server can crash, a container can be terminated, a deployment can exceed its timeout, or the machine can disappear. Idempotency remains the real recovery mechanism.

Job expiration settings also need to reflect reality. If a valid job can run for an hour but the system treats it as abandoned after fifteen minutes, another worker may begin a retry while the first attempt is still active. Timeouts should be based on measured job duration, with alerts for work that exceeds the expected range.

Operational visibility made the queue trustworthy

A production queue needs more than logs saying that a worker started.

We needed to know:

• Queue depth and age of the oldest job
• Active, completed, failed, and retrying counts
• Attempts per job
• Execution duration
• Stalled scheduled runs
• Dead-letter queue activity
• The business record associated with each job

The most useful alert is often not “a job failed.” A transient failure followed by a successful retry may be normal. More actionable signals include a queue that is no longer draining, a job that keeps retrying, or a scheduled run that never produced its expected business result.

Would I use pg-boss again?

Yes, especially for a Node.js application that already uses PostgreSQL and does not need a separate distributed messaging platform.

pg-boss gave us durable jobs, controlled concurrency, scheduling, and retries without forcing the team to operate another data system. Paired with separate PM2 worker processes, it let the backend do substantial asynchronous work without holding up normal web traffic.

But the library was only one part of the reliability story. The production-grade system came from combining the queue with:

• Idempotent handlers
• Stable business keys
• Careful retry rules
• Overlap protection for scheduled work
• Graceful shutdown
• Useful monitoring and reconciliation

The queue made failures recoverable. Resilient job design made recovery safe.