mirror of
https://github.com/deanpcmad/sidekiq-limit_fetch.git
synced 2022-11-09 13:54:36 -05:00
v4.1.0 and updated readme
This commit is contained in:
parent
afb6c259bb
commit
5ff70b18fc
3 changed files with 49 additions and 41 deletions
|
@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
- [#101](deanpcmad/sidekiq-limit_fetch#101) Fix stuck queues bug on Redis restart from [@907th](https://github.com/907th).
|
- #101 - Fix stuck queues bug on Redis restart from [@907th](https://github.com/907th).
|
||||||
|
|
||||||
## [4.0.0] - 2022-03-26
|
## [4.0.0] - 2022-03-26
|
||||||
|
|
||||||
|
@ -17,6 +17,6 @@ This project was taken over by [@deanpcmad](https://github.com/deanpcmad)
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
- [#120](deanpcmad/sidekiq-limit_fetch#120) - Migrate CI to GitHub Actions from [@petergoldstein](https://github.com/petergoldstein).
|
- #120 - Migrate CI to GitHub Actions from [@petergoldstein](https://github.com/petergoldstein).
|
||||||
- [#124](deanpcmad/sidekiq-limit_fetch#124) - Fixed redis v4.6.0 pipelines deprecation warning from [@iurev](https://github.com/iurev).
|
- #124 - Fixed redis v4.6.0 pipelines deprecation warning from [@iurev](https://github.com/iurev).
|
||||||
- [#83](deanpcmad/sidekiq-limit_fetch#83) - Processing dynamic queues from [@alexey-yanchenko](https://github.com/alexey-yanchenko).
|
- #83 - Processing dynamic queues from [@alexey-yanchenko](https://github.com/alexey-yanchenko).
|
52
README.md
52
README.md
|
@ -1,6 +1,6 @@
|
||||||
## Description
|
## Description
|
||||||
|
|
||||||
*This project is currently being taken over by [@deanpcmad](https://github.com/deanpcmad) and will be updated soon*
|
*This project has been taken over by [@deanpcmad](https://github.com/deanpcmad)*
|
||||||
|
|
||||||
Sidekiq strategy to support a granular queue control – limiting, pausing, blocking, querying.
|
Sidekiq strategy to support a granular queue control – limiting, pausing, blocking, querying.
|
||||||
|
|
||||||
|
@ -11,7 +11,11 @@ Sidekiq strategy to support a granular queue control – limiting, pausing, bloc
|
||||||
|
|
||||||
Add this line to your application's Gemfile:
|
Add this line to your application's Gemfile:
|
||||||
|
|
||||||
gem 'sidekiq-limit_fetch'
|
```
|
||||||
|
gem 'sidekiq-limit_fetch'
|
||||||
|
```
|
||||||
|
|
||||||
|
Then `bundle install`.
|
||||||
|
|
||||||
### Limitations
|
### Limitations
|
||||||
|
|
||||||
|
@ -31,19 +35,19 @@ To use this Gem in other Ruby projects, just add `require 'sidekiq-limit_fetch'`
|
||||||
Specify limits which you want to place on queues inside sidekiq.yml:
|
Specify limits which you want to place on queues inside sidekiq.yml:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
:limits:
|
:limits:
|
||||||
queue_name1: 5
|
queue_name1: 5
|
||||||
queue_name2: 10
|
queue_name2: 10
|
||||||
```
|
```
|
||||||
|
|
||||||
Or set it dynamically in your code:
|
Or set it dynamically in your code:
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['queue_name1'].limit = 5
|
Sidekiq::Queue['queue_name1'].limit = 5
|
||||||
Sidekiq::Queue['queue_name2'].limit = 10
|
Sidekiq::Queue['queue_name2'].limit = 10
|
||||||
```
|
```
|
||||||
|
|
||||||
In these examples, tasks for the ```queue_name1``` will be run by at most 5
|
In these examples, tasks for the `queue_name1` will be run by at most 5
|
||||||
workers at the same time and the ```queue_name2``` will have no more than 10
|
workers at the same time and the `queue_name2` will have no more than 10
|
||||||
workers simultaneously.
|
workers simultaneously.
|
||||||
|
|
||||||
Ability to set limits dynamically allows you to resize worker
|
Ability to set limits dynamically allows you to resize worker
|
||||||
|
@ -54,14 +58,14 @@ distribution among queues any time you want.
|
||||||
If you use multiple sidekiq processes then you can specify limits per process:
|
If you use multiple sidekiq processes then you can specify limits per process:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
:process_limits:
|
:process_limits:
|
||||||
queue_name: 2
|
queue_name: 2
|
||||||
```
|
```
|
||||||
|
|
||||||
Or set it in your code:
|
Or set it in your code:
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['queue_name'].process_limit = 2
|
Sidekiq::Queue['queue_name'].process_limit = 2
|
||||||
```
|
```
|
||||||
|
|
||||||
### Busy workers by queue
|
### Busy workers by queue
|
||||||
|
@ -69,7 +73,7 @@ Or set it in your code:
|
||||||
You can see how many workers currently handling a queue:
|
You can see how many workers currently handling a queue:
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['name'].busy # number of busy workers
|
Sidekiq::Queue['name'].busy # number of busy workers
|
||||||
```
|
```
|
||||||
|
|
||||||
### Pauses
|
### Pauses
|
||||||
|
@ -78,10 +82,10 @@ You can also pause your queues temporarily. Upon continuing their limits
|
||||||
will be preserved.
|
will be preserved.
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
|
Sidekiq::Queue['name'].pause # prevents workers from running tasks from this queue
|
||||||
Sidekiq::Queue['name'].paused? # => true
|
Sidekiq::Queue['name'].paused? # => true
|
||||||
Sidekiq::Queue['name'].unpause # allows workers to use the queue
|
Sidekiq::Queue['name'].unpause # allows workers to use the queue
|
||||||
Sidekiq::Queue['name'].pause_for_ms(1000) # will pause for a second
|
Sidekiq::Queue['name'].pause_for_ms(1000) # will pause for a second
|
||||||
```
|
```
|
||||||
|
|
||||||
### Blocking queue mode
|
### Blocking queue mode
|
||||||
|
@ -92,11 +96,11 @@ queue task is executing then no new task from lesser priority queues will
|
||||||
be ran. Eg,
|
be ran. Eg,
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
:queues:
|
:queues:
|
||||||
- a
|
- a
|
||||||
- b
|
- b
|
||||||
- c
|
- c
|
||||||
:blocking:
|
:blocking:
|
||||||
- b
|
- b
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -106,9 +110,9 @@ will be started.
|
||||||
You can also enable and disable blocking mode for queues on the fly:
|
You can also enable and disable blocking mode for queues on the fly:
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['name'].block
|
Sidekiq::Queue['name'].block
|
||||||
Sidekiq::Queue['name'].blocking? # => true
|
Sidekiq::Queue['name'].blocking? # => true
|
||||||
Sidekiq::Queue['name'].unblock
|
Sidekiq::Queue['name'].unblock
|
||||||
```
|
```
|
||||||
|
|
||||||
### Advanced blocking queues
|
### Advanced blocking queues
|
||||||
|
@ -118,12 +122,12 @@ running only queues higher and queues from their blocking group can
|
||||||
run. It will be easier to understand with an example:
|
run. It will be easier to understand with an example:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
:queues:
|
:queues:
|
||||||
- a
|
- a
|
||||||
- b
|
- b
|
||||||
- c
|
- c
|
||||||
- d
|
- d
|
||||||
:blocking:
|
:blocking:
|
||||||
- [b, c]
|
- [b, c]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -132,7 +136,7 @@ In this case tasks from `d` will be blocked when a task from queue `b` or `c` is
|
||||||
You can dynamically set exceptions for queue blocking:
|
You can dynamically set exceptions for queue blocking:
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Sidekiq::Queue['queue1'].block_except 'queue2'
|
Sidekiq::Queue['queue1'].block_except 'queue2'
|
||||||
```
|
```
|
||||||
|
|
||||||
### Dynamic queues
|
### Dynamic queues
|
||||||
|
@ -143,11 +147,11 @@ that have tasks pushed to them (usually with `Sidekiq::Client.push`)).
|
||||||
To use this mode you need to specify a following line in sidekiq.yml:
|
To use this mode you need to specify a following line in sidekiq.yml:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
:dynamic: true
|
:dynamic: true
|
||||||
```
|
```
|
||||||
|
|
||||||
Dynamic queues will be ran at the lowest priority.
|
Dynamic queues will be ran at the lowest priority.
|
||||||
|
|
||||||
### Maintenance
|
### Maintenance
|
||||||
|
|
||||||
If you use ```flushdb```, restart the sidekiq process to re-populate the dynamic configuration.
|
If you use `flushdb`, restart the sidekiq process to re-populate the dynamic configuration.
|
||||||
|
|
|
@ -1,6 +1,6 @@
|
||||||
Gem::Specification.new do |gem|
|
Gem::Specification.new do |gem|
|
||||||
gem.name = 'sidekiq-limit_fetch'
|
gem.name = 'sidekiq-limit_fetch'
|
||||||
gem.version = '4.0.0'
|
gem.version = '4.1.0'
|
||||||
gem.license = 'MIT'
|
gem.license = 'MIT'
|
||||||
gem.authors = ['Dean Perry', 'brainopia']
|
gem.authors = ['Dean Perry', 'brainopia']
|
||||||
gem.email = 'dean@deanpcmad.com'
|
gem.email = 'dean@deanpcmad.com'
|
||||||
|
@ -8,6 +8,10 @@ Gem::Specification.new do |gem|
|
||||||
gem.homepage = 'https://github.com/deanpcmad/sidekiq-limit_fetch'
|
gem.homepage = 'https://github.com/deanpcmad/sidekiq-limit_fetch'
|
||||||
gem.description = "Sidekiq strategy to restrict number of workers which are able to run specified queues simultaneously."
|
gem.description = "Sidekiq strategy to restrict number of workers which are able to run specified queues simultaneously."
|
||||||
|
|
||||||
|
gem.metadata["homepage_uri"] = spec.homepage
|
||||||
|
gem.metadata["source_code_uri"] = "https://github.com/deanpcmad/sidekiq-limit_fetch"
|
||||||
|
gem.metadata["changelog_uri"] = "https://github.com/deanpcmad/sidekiq-limit_fetch/blob/master/CHANGELOG.md"
|
||||||
|
|
||||||
gem.files = `git ls-files`.split($/)
|
gem.files = `git ls-files`.split($/)
|
||||||
gem.test_files = gem.files.grep %r{^spec/}
|
gem.test_files = gem.files.grep %r{^spec/}
|
||||||
gem.require_paths = %w(lib)
|
gem.require_paths = %w(lib)
|
||||||
|
|
Loading…
Add table
Reference in a new issue