mirror of
https://github.com/deanpcmad/sidekiq-limit_fetch.git
synced 2022-11-09 13:54:36 -05:00
v4.1.0 and updated readme
This commit is contained in:
Dean Perry
parent
afb6c259bb
commit
5ff70b18fc
3 changed files with 49 additions and 41 deletions
|
|
@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
### Changed
|
||||
|
||||
- [#101](deanpcmad/sidekiq-limit_fetch#101) Fix stuck queues bug on Redis restart from [@907th](https://github.com/907th).
|
||||
- #101 - Fix stuck queues bug on Redis restart from [@907th](https://github.com/907th).
|
||||
|
||||
## [4.0.0] - 2022-03-26
|
||||
|
||||
|
|
@ -17,6 +17,6 @@ This project was taken over by [@deanpcmad](https://github.com/deanpcmad)
|
|||
|
||||
### Changed
|
||||
|
||||
- [#120](deanpcmad/sidekiq-limit_fetch#120) - Migrate CI to GitHub Actions from [@petergoldstein](https://github.com/petergoldstein).
|
||||
- [#124](deanpcmad/sidekiq-limit_fetch#124) - Fixed redis v4.6.0 pipelines deprecation warning from [@iurev](https://github.com/iurev).
|
||||
- [#83](deanpcmad/sidekiq-limit_fetch#83) - Processing dynamic queues from [@alexey-yanchenko](https://github.com/alexey-yanchenko).
|
||||
- #120 - Migrate CI to GitHub Actions from [@petergoldstein](https://github.com/petergoldstein).
|
||||
- #124 - Fixed redis v4.6.0 pipelines deprecation warning from [@iurev](https://github.com/iurev).
|
||||
- #83 - Processing dynamic queues from [@alexey-yanchenko](https://github.com/alexey-yanchenko).
|
||||
76
README.md
76
README.md
|
|
@ -1,6 +1,6 @@
|
|||
## Description
|
||||
|
||||
*This project is currently being taken over by [@deanpcmad](https://github.com/deanpcmad) and will be updated soon*
|
||||
*This project has been taken over by [@deanpcmad](https://github.com/deanpcmad)*
|
||||
|
||||
Sidekiq strategy to support a granular queue control – limiting, pausing, blocking, querying.
|
||||
|
||||
|
|
@ -11,7 +11,11 @@ Sidekiq strategy to support a granular queue control – limiting, pausing, bloc
|
|||
|
||||
Add this line to your application's Gemfile:
|
||||
|
||||
gem 'sidekiq-limit_fetch'
|
||||
```
|
||||
gem 'sidekiq-limit_fetch'
|
||||
```
|
||||
|
||||
Then `bundle install`.
|
||||
|
||||
### Limitations
|
||||
|
||||
|
|
@ -31,19 +35,19 @@ To use this Gem in other Ruby projects, just add `require 'sidekiq-limit_fetch'`
|
|||
Specify limits which you want to place on queues inside sidekiq.yml:
|
||||
|
||||
```yaml
|
||||
:limits:
|
||||
queue_name1: 5
|
||||
queue_name2: 10
|
||||
:limits:
|
||||
queue_name1: 5
|
||||
queue_name2: 10
|
||||
```
|
||||
|
||||
Or set it dynamically in your code:
|
||||
```ruby
|
||||
Sidekiq::Queue['queue_name1'].limit = 5
|
||||
Sidekiq::Queue['queue_name2'].limit = 10
|
||||
Sidekiq::Queue['queue_name1'].limit = 5
|
||||
Sidekiq::Queue['queue_name2'].limit = 10
|
||||
```
|
||||
|
||||
In these examples, tasks for the ```queue_name1``` will be run by at most 5
|
||||
workers at the same time and the ```queue_name2``` will have no more than 10
|
||||
In these examples, tasks for the `queue_name1` will be run by at most 5
|
||||
workers at the same time and the `queue_name2` will have no more than 10
|
||||
workers simultaneously.
|
||||
|
||||
Ability to set limits dynamically allows you to resize worker
|
||||
|
|
@ -54,14 +58,14 @@ distribution among queues any time you want.
|
|||
If you use multiple sidekiq processes then you can specify limits per process:
|
||||
|
||||
```yaml
|
||||
:process_limits:
|
||||
queue_name: 2
|
||||
:process_limits:
|
||||
queue_name: 2
|
||||
```
|
||||
|
||||
Or set it in your code:
|
||||
|
||||
```ruby
|
||||
Sidekiq::Queue['queue_name'].process_limit = 2
|
||||
Sidekiq::Queue['queue_name'].process_limit = 2
|
||||
```
|
||||
|
||||
### Busy workers by queue
|
||||
|
|
@ -69,7 +73,7 @@ Or set it in your code:
|
|||
You can see how many workers currently handling a queue:
|
||||
|
||||
```ruby
|
||||
Sidekiq::Queue['name'].busy # number of busy workers
|
||||
Sidekiq::Queue['name'].busy # number of busy workers
|
||||
```
|
||||
|
||||
### Pauses
|
||||
|
|
@ -78,10 +82,10 @@ You can also pause your queues temporarily. Upon continuing their limits
|
|||
will be preserved.
|
||||
|
||||
```ruby
|
||||
Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
|
||||
Sidekiq::Queue['name'].paused? # => true
|
||||
Sidekiq::Queue['name'].unpause # allows workers to use the queue
|
||||
Sidekiq::Queue['name'].pause_for_ms(1000) # will pause for a second
|
||||
Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
|
||||
Sidekiq::Queue['name'].paused? # => true
|
||||
Sidekiq::Queue['name'].unpause # allows workers to use the queue
|
||||
Sidekiq::Queue['name'].pause_for_ms(1000) # will pause for a second
|
||||
```
|
||||
|
||||
### Blocking queue mode
|
||||
|
|
@ -92,12 +96,12 @@ queue task is executing then no new task from lesser priority queues will
|
|||
be ran. Eg,
|
||||
|
||||
```yaml
|
||||
:queues:
|
||||
- a
|
||||
- b
|
||||
- c
|
||||
:blocking:
|
||||
- b
|
||||
:queues:
|
||||
- a
|
||||
- b
|
||||
- c
|
||||
:blocking:
|
||||
- b
|
||||
```
|
||||
|
||||
In this case when a task for `b` queue is ran no new task from `c` queue
|
||||
|
|
@ -106,9 +110,9 @@ will be started.
|
|||
You can also enable and disable blocking mode for queues on the fly:
|
||||
|
||||
```ruby
|
||||
Sidekiq::Queue['name'].block
|
||||
Sidekiq::Queue['name'].blocking? # => true
|
||||
Sidekiq::Queue['name'].unblock
|
||||
Sidekiq::Queue['name'].block
|
||||
Sidekiq::Queue['name'].blocking? # => true
|
||||
Sidekiq::Queue['name'].unblock
|
||||
```
|
||||
|
||||
### Advanced blocking queues
|
||||
|
|
@ -118,13 +122,13 @@ running only queues higher and queues from their blocking group can
|
|||
run. It will be easier to understand with an example:
|
||||
|
||||
```yaml
|
||||
:queues:
|
||||
- a
|
||||
- b
|
||||
- c
|
||||
- d
|
||||
:blocking:
|
||||
- [b, c]
|
||||
:queues:
|
||||
- a
|
||||
- b
|
||||
- c
|
||||
- d
|
||||
:blocking:
|
||||
- [b, c]
|
||||
```
|
||||
|
||||
In this case tasks from `d` will be blocked when a task from queue `b` or `c` is executed.
|
||||
|
|
@ -132,7 +136,7 @@ In this case tasks from `d` will be blocked when a task from queue `b` or `c` is
|
|||
You can dynamically set exceptions for queue blocking:
|
||||
|
||||
```ruby
|
||||
Sidekiq::Queue['queue1'].block_except 'queue2'
|
||||
Sidekiq::Queue['queue1'].block_except 'queue2'
|
||||
```
|
||||
|
||||
### Dynamic queues
|
||||
|
|
@ -143,11 +147,11 @@ that have tasks pushed to them (usually with `Sidekiq::Client.push`)).
|
|||
To use this mode you need to specify a following line in sidekiq.yml:
|
||||
|
||||
```yaml
|
||||
:dynamic: true
|
||||
:dynamic: true
|
||||
```
|
||||
|
||||
Dynamic queues will be ran at the lowest priority.
|
||||
|
||||
### Maintenance
|
||||
|
||||
If you use ```flushdb```, restart the sidekiq process to re-populate the dynamic configuration.
|
||||
If you use `flushdb`, restart the sidekiq process to re-populate the dynamic configuration.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
Gem::Specification.new do |gem|
|
||||
gem.name = 'sidekiq-limit_fetch'
|
||||
gem.version = '4.0.0'
|
||||
gem.version = '4.1.0'
|
||||
gem.license = 'MIT'
|
||||
gem.authors = ['Dean Perry', 'brainopia']
|
||||
gem.email = 'dean@deanpcmad.com'
|
||||
|
|
@ -8,6 +8,10 @@ Gem::Specification.new do |gem|
|
|||
gem.homepage = 'https://github.com/deanpcmad/sidekiq-limit_fetch'
|
||||
gem.description = "Sidekiq strategy to restrict number of workers which are able to run specified queues simultaneously."
|
||||
|
||||
gem.metadata["homepage_uri"] = spec.homepage
|
||||
gem.metadata["source_code_uri"] = "https://github.com/deanpcmad/sidekiq-limit_fetch"
|
||||
gem.metadata["changelog_uri"] = "https://github.com/deanpcmad/sidekiq-limit_fetch/blob/master/CHANGELOG.md"
|
||||
|
||||
gem.files = `git ls-files`.split($/)
|
||||
gem.test_files = gem.files.grep %r{^spec/}
|
||||
gem.require_paths = %w(lib)
|
||||
|
|
|
|||
Loading…
Reference in a new issue