* Add keyword arguments handler in Sidekiq::JobRecord#display_args
* Update Sidekiq::JobRecord#display_args kwargs's presence check so that we are not dependent on Rails.
Context: https://github.com/redis/redis-rb/pull/1059
The following is deprecated
```ruby
redis.pipelined do
redis.get(key)
end
```
And should be rewritten as:
```ruby
redis.pipelined do |pipeline|
pipeline.get(key)
end
```
Functionally it makes no difference.
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
* Add keyword arguments support in extensions
* Check that keywords arguments are passed
* Test perform calls with keyword arguments
* Fix kwargs forwarding on Ruby 2.6
If we raised Sidekiq::Shutdown in job threads, sleep for a while
so they have time to run `ensure` blocks. Right now we immediately
return from hard_shutdown and call `exit` which immediately raises SystemExit on every thread which can stop ensure blocks.
* Add the outline of failing tests
* Implement argument checking
* Move argument checking into Sidekiq::JobUtil#validate
* Refactor acceptable class definition into a constant to cut down on array allocations
* Improve error message, match raise call formatting to other errors in the class
* Address feedback in the Pull Request to use the JSON round-trip method
of confirming the safety of job argument payloads.
Cleanup commented-out code from a few years back.
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Swap out JSON.load for JSON.parse per the security CI check
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Add a few more tests cases to build up confidence around our JSON.parse/dump approach and deep structures
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Improve test case description
* Warn when job arguments do not serialize safely and point folks toward how to enable strict_mode and the best practice
* Reconfigure the options-hash based approach to a global Sidekiq.strict_mode! method
* Add a note in the raised error on how to disable the error
* Let the error message breathe a little bit
* Toggle strict_mode! off to suss out a test flake
* Capitalize the start of a sentence
* Rename job_is_json_safe to json_safe?
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Refactor a few tests to test a single argument at a time
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Break out test cases to exercise each individual intersting case instead of all at once
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Change formatting to be more consistent, tighter when arguments are simple
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
* Add a flag to disable the warning message for development warning messages
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
We need to call the `script` command on the raw redis connection because
passthrough for administrative commands has been deprecated and will be
removed in redis-namespace 2.0
Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
* Create failing tests to capture the cases we care about
Co-authored-by: Noa Elad <noa.elad@zenpayroll.com>
* Call .force on args if they are a lazy enumerator to evaluate them immediately
Co-authored-by: Noa Elad <noa.elad@zenpayroll.com>
* Change implementation of `.perform_bulk` to lazily pull values from a lazy enumerator.
This prevents us from loading the entire contents of a lazy enumerator into memory while providing the
"unsurprising" behavior of enqueueing jobs when `.perform_bulk` is called.
Note: We'll still load all JIDs into memory.
Co-authored-by: Noa Elad <noa.elad@zenpayroll.com>
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
Co-authored-by: Noa Elad <noa.elad@zenpayroll.com>
Co-authored-by: Eda Zhou <eda.zhou@gusto.com>
When there are scheduled jobs that are due, the scheduler thread will
keep going instead of waiting for the next poll.
Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
This allows us to pop jobs atomically so that we remove redundant ZREMs
from multiple processes trying to process the same job.
Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
* Sketch out some failing tests to capture the behavior
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Implement Sidekiq::Client.perform_bulk
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Allow .perform_bulk to operate on different batch sizes
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Write a failing test to capture the Sidekiq::Worker::Setter.perform_bulk behavior
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Implement Sidekiq::Worker::Setter.perform_bulk
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Write a small comment for to document the method
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
* Add a Changes.md entry
Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
We don't need the commands here to be atomic so we use pipelined instead
to save us some overhead from MULTI
Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
