About

DataQueue is an open source lightweight job queue for Node.js/TypeScript projects, backed by PostgreSQL or Redis. It lets you easily schedule, process, and manage background jobs. It's ideal for serverless environments like Vercel, AWS Lambda, and more.

Features

• Simple API for adding and processing jobs
• Strong typing for job types and payloads, preventing you from adding jobs with the wrong payload and ensuring handlers receive the correct type
• Works in serverless environments
• Supports job priorities, scheduling, canceling, and retries
• Reclaims stuck jobs: No job will remain in the processing state indefinitely
• Cleans up old jobs: Keeps only jobs from the last xxx days
• Choose your backend: Use PostgreSQL or Redis -- same API, same features, your choice

Who is this for?

This package is for you if all of the following apply:

Backend Options

PostgreSQL

If you already use PostgreSQL, it makes sense to use it for job queues, thanks to SKIP LOCKED.

The update process in DataQueue uses FOR UPDATE SKIP LOCKED to avoid race conditions and improve performance. If two jobs are scheduled at the same time, one will skip any jobs that are already being processed and work on other available jobs instead. This lets multiple workers handle different jobs at once without waiting or causing conflicts, making PostgreSQL a great choice for job queues and similar tasks.

The PostgreSQL backend requires running database migrations before use.

Redis

If you already have a Redis instance in your stack, you can use it as the backend instead. The Redis backend uses Lua scripts for atomic operations and sorted sets for priority-based job claiming, providing the same guarantees as the PostgreSQL backend.

The Redis backend requires no migrations -- it automatically creates the necessary keys when jobs are added.