← Graph

Passing Large Lists Of IDs To Jobs

question 2 connections

Audience question about passing a whole list of a few hundred object IDs to one job versus scheduling many single-ID jobs. Paweł recommended using Sidekiq batches: schedule many small jobs and use the batch callback (typically another background job) when they're all done. Batches are a Sidekiq Pro/Enterprise feature but open-source replacements exist.

answer_summary
Prefer Sidekiq batches of single-ID jobs with a completion callback, using open-source replacements if not on Pro/Enterprise.
question Passing Large Lists Of IDs To Jobs
about
Sidekiq Batches concept
Paweł's answer recommends Sidekiq batches with per-ID jobs.
question Passing Large Lists Of IDs To Jobs
asked_at
Question asked during the Q&A of this talk.

Provenance