Short answer: for a Node.js SaaS that must fan out a shipment update, put each file processing, email sending, and webhook sync attempt behind a background job queue. Use cron to discover periodic work, such as a retry sweep. A clock can find work; it cannot remember which subscriber was delivered, which one timed out, or which side effect may have happened just before a crash.
Here is the small choice matrix I would start with:
| Work | Default | Delivery reason |
|---|---|---|
| Process a file after an upload | Queue job | The job has one owner, attempt history, and a retry deadline |
| Send an email for a shipment update | Queue job | One slow recipient stays isolated from the fan-out |
| Sync a webhook subscriber | Queue job | Each endpoint needs its own backoff and idempotency key |
| Find work due at 02:00 | Cron, then enqueue | The schedule discovers records; a worker performs the effects |
| Remove expired temporary files | Cron | A later run can safely discover the same cleanup again |
The easiest setup is usually a durable jobs table, one worker, and one scheduler. You do not need to start with a large messaging system to get the important guarantee. You do need durable state.
How should a beginner use a background job queue for file processing in a Node.js SaaS?
Start by naming the guarantee in plain English. For the shipment event shp_4821, the SaaS might need to attempt an email for subscriber sub_19, process a label PDF, and synchronize a webhook endpoint. Those are three jobs, even if one button created them. Each has a different destination, timeout, payload, and definition of success. Treating the fan-out as one function call hides the exact information an operator needs when a customer says, “the email arrived, but the partner system did not update.”
The useful promise is usually at-least-once processing with idempotent effects. Exactly-once delivery sounds tidy, but an external email or webhook can accept a request immediately before the worker loses power. The worker then cannot know whether repeating the request is safe. I would rather document that boundary than hide it behind a green status badge.
Keep these values with the job:
-
kind,payload, and the shipment event ID -
attempts,nextAttemptAt, and a retry deadline -
status,leaseUntil, and the last error category - a stable idempotency key for the logical side effect
The key should identify the effect, not merely the worker run. For example, shipment:shp_4821:subscriber:sub_19:webhook:v3 is more useful than a random UUID generated during a retry. A database uniqueness constraint can stop two workers from creating the same logical delivery record. The receiver still needs to honor a request key, or your system must accept that a crash can produce a duplicate.
Three words matter here: claim, effect, record. Claim a job for a limited lease. Perform one bounded effect. Record success only after the effect returns. If the process exits between the last two steps, the lease expires and the job becomes available again. That is the failure path your design has to explain.
RabbitMQ's consumer acknowledgement documentation describes the same handoff concern: an acknowledgement tells the broker that a consumer has handled a delivery, while an unacknowledged delivery can be made available again. The acknowledgement is useful state. It is not proof that an unrelated third-party side effect happened exactly once.
What delivery data belongs in the seven-day ledger?
Cron answers “when should I look?” It does not naturally answer “which of 18,000 subscribers is on attempt four, and which one is due in six hours?” A queue or durable job table can make that per-delivery state explicit.
This distinction matters for delayed messages up to 7 days. A retry due next week should have a stored deadline and a stored idempotency key. It should not depend on a Node.js process staying alive or on a cron script remembering local variables from last Tuesday.
Backoff should be bounded. A practical policy can use exponential delay with jitter, cap the delay at seven days, and move a permanently rejected job to a dead state. Classify errors before applying that policy. A timeout or transient network failure may be retryable. A malformed payload, invalid recipient, or authorization failure usually needs repair instead of twelve identical requests.
My test case includes a 409 from a deduplication constraint. I treat that as “the logical effect already exists,” not as a reason to send the email again. I'm not sure every provider uses that exact status or meaning, so the mapping belongs in the adapter for that provider, not in a generic worker.
The long delay also changes operations. Retain enough metadata to investigate a missed delivery without retaining secrets forever. Store a file reference instead of copying a large file into every job. Redact webhook credentials and email addresses in logs. A seven-day retry policy without a way to inspect or expire its payload is just a seven-day data-retention problem.
Can a cron workflow discover email and webhook work without owning it?
Make the scheduler a producer. It finds due work in bounded batches, inserts jobs with a uniqueness rule, and exits. Make the worker the owner of delivery. This division keeps a missed scheduled run recoverable and keeps a long webhook retry from turning into one enormous cron invocation.
The following TypeScript is deliberately an interface boundary. The database and queue implementations can change without changing the delivery rules.
type JobKind = "file.process" | "email.send" | "webhook.sync";
type DeliveryJob = {
id: string;
kind: JobKind;
shipmentEventId: string;
idempotencyKey: string;
attempts: number;
nextAttemptAt: number;
expiresAt: number;
};
type JobStore = {
claim(now: number, leaseMs: number): Promise<DeliveryJob | null>;
markDone(id: string): Promise<void>;
scheduleRetry(id: string, at: number, reason: string): Promise<void>;
markDead(id: string, reason: string): Promise<void>;
};
type DeliveryEffect = {
run(job: DeliveryJob): Promise<"done" | "retryable" | "permanent">;
};
async function processOne(store: JobStore, effect: DeliveryEffect): Promise<void> {
const job = await store.claim(Date.now(), 60_000);
if (!job) return;
try {
const result = await effect.run(job);
if (result === "done") {
await store.markDone(job.id);
return;
}
if (result === "permanent" || Date.now() >= job.expiresAt) {
await store.markDead(job.id, "delivery deadline reached");
return;
}
const exponentialMs = Math.min(24 * 60 * 60 * 1000, 2 ** job.attempts * 1_000);
const jitterMs = Math.floor(Math.random() * 1_000);
await store.scheduleRetry(job.id, Date.now() + exponentialMs + jitterMs, "transient failure");
} catch (error) {
await store.scheduleRetry(job.id, Date.now() + 60_000, String(error));
}
}
The adapter behind effect.run checks the idempotency key at the effect boundary. That matters because a lease can expire while a file processor is still finishing, allowing another worker to see the same logical job. A check only in the scheduler is too late.
Run the scheduler like this: select rows whose nextAttemptAt is due, insert a bounded batch, and let the uniqueness rule reject duplicates. GitHub Actions documents scheduled workflow triggers as a time-based mechanism; the same principle applies to any cron host. The scheduled process should not carry the delivery state in memory, and it should not report success until the discovered work is durable.
How can I benchmark the handoff before adding a broker?
Cron is a good fit for periodic, bounded work where lateness is acceptable and the next run can repair an omission. Expiring temporary files, reconciling a small table, and finding retryable rows are reasonable examples. A five-minute sweep can enqueue due delivery jobs without pretending that every subscriber is one five-minute task.
Measure it.
The catch is that cron alone is not suitable when subscribers have independent deadlines, rate limits, attempt histories, or dead-letter handling. It also becomes awkward when file processing takes longer than the schedule interval or when one failed email blocks later recipients in the same loop. Use a queue for that execution layer.
A queue can be the wrong first move, too. If the team cannot operate a broker and the workload is modest, a database-backed queue with polling may be easier to reason about. If the task has no meaningful recovery path and a missed run is harmless, adding durable per-item delivery state is configuration bloat. Keep the least complex mechanism that records the guarantee you actually need.
Do not choose by monthly price alone. Measure time to first call, queue age, p95 delivery latency, retry rate, dead-job count, and operator steps during a replay. I care about those numbers because a cheap queue that needs a custom script for every failure is not cheap in engineering time. There is a large difference between a job that is late for 30 seconds and one that is still pending after its subscriber's seven-day window; both can look like “the worker is running” in a shallow dashboard, while only the latter is a delivery incident. Your mileage may vary with traffic and latency targets.
What does the queue-versus-cron comparison say about low-stakes cleanup?
The failure drill belongs in this benchmark, not in a launch checklist nobody runs. Start two workers against the same job. Kill one after the outbound request returns but before markDone. Pause one webhook endpoint. Feed the file processor a large input. Advance the clock beyond the seven-day deadline.
The expected result is not “nothing runs twice.” The expected result is that repeated logical work is harmless or visible, the job can be traced by its idempotency key, and a permanent failure stops retrying. A green email response does not prove that the database update happened afterward.
Log the job ID, shipment event ID, attempt number, and outcome category. Do not log credentials or full webhook payloads by default. Alert on old pending jobs and an unusual retry rate, not only on a dead worker process. Those signals tell you whether delivery is drifting before a customer reports a missing update.
The decision rule is short: cron discovers time-based work; a background job queue owns independent delivery attempts. For a Node.js SaaS sending shipment updates to many subscribers, that boundary makes retries, idempotency, and delayed messages explicit without promising a guarantee the outside system cannot provide.
References
- RabbitMQ consumer acknowledgements: https://www.rabbitmq.com/docs/confirms
- GitHub Actions scheduled workflow triggers: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows
Top comments (0)