deanpcmad--sidekiq-limit_fetch/README.md

110 lines
2.7 KiB
Markdown
Raw Normal View History

2013-01-11 10:56:41 +00:00
## Description
2013-01-17 15:21:10 +00:00
Sidekiq strategy to restrict number of workers
2013-01-11 10:56:41 +00:00
which are able to run specified queues simultaneously.
2013-02-05 11:37:22 +00:00
[![Build Status](https://secure.travis-ci.org/brainopia/sidekiq-limit_fetch.png)](http://travis-ci.org/brainopia/sidekiq-limit_fetch)
[![Gem Version](https://badge.fury.io/rb/sidekiq-limit_fetch.png)](http://badge.fury.io/rb/sidekiq-limit_fetch)
[![Dependency Status](https://gemnasium.com/brainopia/sidekiq-limit_fetch.png)](https://gemnasium.com/brainopia/sidekiq-limit_fetch)
[![Code Climate](https://codeclimate.com/github/brainopia/sidekiq-limit_fetch.png)](https://codeclimate.com/github/brainopia/sidekiq-limit_fetch)
2013-01-24 13:54:26 +00:00
2013-01-11 10:56:41 +00:00
## Installation
Add this line to your application's Gemfile:
gem 'sidekiq-limit_fetch'
## Usage
2013-01-25 07:42:21 +00:00
### Limits
2013-01-11 10:56:41 +00:00
Specify limits which you want to place on queues inside sidekiq.yml:
```yaml
:limits:
queue_name1: 5
queue_name2: 10
2013-01-11 10:56:41 +00:00
```
2013-01-17 15:18:56 +00:00
Or set it dynamically in your code:
```ruby
2013-01-24 12:03:28 +00:00
Sidekiq::Queue['queue_name1'].limit = 5
2013-01-17 15:18:56 +00:00
Sidekiq::Queue['queue_name2'].limit = 10
```
In these examples, tasks for the ```queue_name1``` will be run by at most 5
workers at the same time and the ```queue_name2``` will have no more than 10
2013-01-11 11:02:39 +00:00
workers simultaneously.
2013-01-11 13:07:37 +00:00
2013-01-17 15:18:56 +00:00
Ability to set limits dynamically allows you to resize worker
distribution among queues any time you want.
2013-01-25 07:42:21 +00:00
### Pauses
2013-01-17 15:49:45 +00:00
You can also pause your queues temporarely. Upon continuing their limits
will be preserved.
```ruby
Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
2013-01-25 07:18:07 +00:00
Sidekiq::Queue['name'].paused? # => true
Sidekiq::Queue['name'].unpause # allows workers to use the queue
2013-01-17 15:49:45 +00:00
```
2013-01-25 07:18:07 +00:00
You can see how many workers currently handling a queue:
```ruby
Sidekiq::Queue['name'].busy # number of busy workers
```
2013-01-25 07:42:21 +00:00
### Multiple processes
Limits are applied per process. In case you have several worker
processes and want to have global locks between them, you'll need to
2013-01-24 12:03:28 +00:00
enable global mode by setting global option, eg:
```yaml
:global: true
2013-01-24 12:03:28 +00:00
```
2013-01-17 15:18:56 +00:00
2013-01-25 07:38:32 +00:00
or
```ruby
Sidekiq.options[:global] = true
```
2013-01-25 07:42:21 +00:00
### Blocking queue mode
2013-01-25 07:18:07 +00:00
If you use strict queue ordering (it will be used if you don't specify queue weights)
then you can set blocking status for queues. It means if a blocking
queue task is executing then no new task from lesser priority queues will
be ran. Eg,
```yaml
:queues:
- a
- b
- c
2013-01-25 07:38:32 +00:00
:blocking:
2013-01-25 07:18:07 +00:00
- b
```
In this case when a task for `b` queue is ran no new task from `c` queue
will be started.
You can also enable and disable blocking mode for queues on the fly:
```ruby
Sidekiq::Queue['name'].block
Sidekiq::Queue['name'].blocking? # => true
Sidekiq::Queue['name'].unblock
```
2013-01-25 07:42:21 +00:00
### Thanks
2013-01-11 13:07:37 +00:00
Sponsored by [Evil Martians].
[Evil Martians]: http://evilmartians.com/