Sidekiq strategy to support an advanced queue control – limiting, pausing, blocking, querying
Go to file
Dean Perry 9bf29b2484
Use OpenSSL::Digest where possible
2022-09-28 09:03:39 +01:00
.github/workflows test 6.5 and remove 5.0, 5.1 and 5.2. need to fix tests 2022-06-09 18:06:29 +00:00
bench Update bench/compare to work with current version 2014-08-18 10:46:45 +04:00
demo Fix demo app 2016-12-09 18:11:56 +09:00
gemfiles 4.3.2 2022-09-01 10:40:27 +01:00
lib Use OpenSSL::Digest where possible 2022-09-28 09:03:39 +01:00
spec Merge pull request #101 from 907th/fix_redis_reloading_bug 2022-03-29 20:32:19 +01:00
.gitignore Prepare for Sidekiq v4.0.0 2015-10-13 17:01:35 +08:00
.rspec Release support for early adopters of sidekiq 4.0 2015-10-15 02:38:25 +03:00
Appraisals test 6.5 and remove 5.0, 5.1 and 5.2. need to fix tests 2022-06-09 18:06:29 +00:00 4.3.2 2022-09-01 10:40:27 +01:00
Gemfile Update sidekiq 2016-01-04 16:46:10 +09:00
LICENSE update license [skip ci] 2022-03-26 10:26:20 +00:00 v4.1.0 and updated readme 2022-03-29 20:49:57 +01:00
Rakefile Correctly support redis namespace 2013-06-18 15:23:12 +04:00
sidekiq-limit_fetch.gemspec 4.3.2 2022-09-01 10:40:27 +01:00


This project has been taken over by @deanpcmad

Sidekiq strategy to support a granular queue control limiting, pausing, blocking, querying.

CI Gem Version


Add this line to your application's Gemfile:

gem 'sidekiq-limit_fetch'

Then bundle install.


Important note: At this moment, sidekiq-limit_fetch is incompatible with

  • sidekiq pro's reliable_fetch
  • sidekiq-rate-limiter
  • any other plugin that rewrites fetch strategy of sidekiq.


If you are using this with Rails, you don't need to require it as it's done automatically.

To use this Gem in other Ruby projects, just add require 'sidekiq-limit_fetch'.


Specify limits which you want to place on queues inside sidekiq.yml:

  queue_name1: 5
  queue_name2: 10

Or set it dynamically in your code:

Sidekiq::Queue['queue_name1'].limit = 5
Sidekiq::Queue['queue_name2'].limit = 10

In these examples, tasks for the queue_name1 will be run by at most 5 workers at the same time and the queue_name2 will have no more than 10 workers simultaneously.

Ability to set limits dynamically allows you to resize worker distribution among queues any time you want.

Limits per process

If you use multiple sidekiq processes then you can specify limits per process:

  queue_name: 2

Or set it in your code:

Sidekiq::Queue['queue_name'].process_limit = 2

Busy workers by queue

You can see how many workers currently handling a queue:

Sidekiq::Queue['name'].busy # number of busy workers


You can also pause your queues temporarily. Upon continuing their limits will be preserved.

Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
Sidekiq::Queue['name'].paused? # => true
Sidekiq::Queue['name'].unpause # allows workers to use the queue
Sidekiq::Queue['name'].pause_for_ms(1000) # will pause for a second

Blocking queue mode

If you use strict queue ordering (it will be used if you don't specify queue weights) then you can set blocking status for queues. It means if a blocking queue task is executing then no new task from lesser priority queues will be ran. Eg,

  - a
  - b
  - c
  - b

In this case when a task for b queue is ran no new task from c queue will be started.

You can also enable and disable blocking mode for queues on the fly:

Sidekiq::Queue['name'].blocking? # => true

Advanced blocking queues

You can also block on array of queues. It means when any of them is running only queues higher and queues from their blocking group can run. It will be easier to understand with an example:

  - a
  - b
  - c
  - d
  - [b, c]

In this case tasks from d will be blocked when a task from queue b or c is executed.

You can dynamically set exceptions for queue blocking:

Sidekiq::Queue['queue1'].block_except 'queue2'

Dynamic queues

You can support dynamic queues (that are not listed in sidekiq.yml but that have tasks pushed to them (usually with Sidekiq::Client.push)).

To use this mode you need to specify a following line in sidekiq.yml:

:dynamic: true

Dynamic queues will be ran at the lowest priority.


If you use flushdb, restart the sidekiq process to re-populate the dynamic configuration.