← Graph

Sidekiq Batches

concept 4 connections

Sidekiq Pro feature that groups many workers into a single batch and fires a callback when all jobs in the batch have finished. Used in the wroclove.rb 2022 lightning talk to parallelize per-row processing of a large file, with each worker writing its result to Redis under a sequence key so a final callback worker can assemble the output file in order.

category
pattern
about
Sidekiq Batches concept
Core subject of the talk.
about
Sidekiq Batches concept
Small-job decomposition pairs well with Sidekiq batches for progress tracking and callbacks.
about
Sidekiq Batches concept
Paweł's answer recommends Sidekiq batches with per-ID jobs.
recommends
Sidekiq Batches concept
In Q&A, recommends Sidekiq batches for processing large lists of IDs.

Provenance

Created in
wroclove.rb 2022 Lightning Talks 2026-04-17 18:12
Read by
13 extractions