1
0
Fork 0
mirror of https://github.com/mperham/sidekiq.git synced 2022-11-09 13:52:34 -05:00
Commit graph

852 commits

Author SHA1 Message Date
Mike Perham
26f5f3373b Fix keyword args issue on Ruby 3.0 2021-11-06 18:12:09 -07:00
Mike Perham
0723abd071 Failing test on 3.0 2021-11-06 18:07:02 -07:00
Kelly Sutton
4a0432622f
Implement Sidekiq::Worker.perform_bulk ()
* Sketch out some failing tests to capture the behavior

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Implement Sidekiq::Client.perform_bulk

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Allow .perform_bulk to operate on different batch sizes

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Write a failing test to capture the Sidekiq::Worker::Setter.perform_bulk behavior

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Implement Sidekiq::Worker::Setter.perform_bulk

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Write a small comment for to document the method

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>

* Add a Changes.md entry

Co-authored-by: jeffcarbs <jeff.carbonella@gmail.com>
2021-11-02 09:54:44 -07:00
Heinrich Lee Yu
fd76471cea
Use Redis.pipelined instead of Redis.multi ()
We don't need the commands here to be atomic so we use pipelined instead
to save us some overhead from MULTI

Co-authored-by: Heinrich Lee Yu <heinrich@gitlab.com>
2021-11-02 07:49:34 -07:00
Mike Perham
3b233c558a Add support for AJ's :wait option 2021-10-05 14:04:04 -07:00
Mike Perham
2b2390d579
Support ActiveJob’s wait_until ()
* Implement support for `set(wait_until: <interval>)`

* Implement `queue_as`, docs

* Rollback implementing `perform_later`, it's a footgun

* changes
2021-09-28 11:24:17 -07:00
Cameron McCord
e9d16dbdb8
updated Sidekiq::Client.push_bulk to support ActiveSupport::Duration in the at argument () 2021-09-20 17:56:07 -07:00
Mike Perham
9c46df0a15
Helper middleware to preserve implicit context? ()
* Add support for round-tripping AS::CurrentAttributes

* Add tests
2021-09-08 15:24:41 -07:00
Franz Liedke
237c70fe37
Reconnect on "UNBLOCKED force unblock" errors ()
These errors can occur during Sidekiq's long-running job fetching
command. This uses Redis' blocking BRPOP primitive. On failover in a
cluster setup, these commands are interrupted by the server.

This error causes the worker threads to be restarted, but as they are
bubbled up to the top, they cause a lot of spam in our error logging
systems. As related errors from other commands are being handled (see
2021-09-07 15:04:01 -07:00
Teo Ljungberg
0bdec6f144
Fix faulty environment variable value () 2021-09-01 11:07:19 -07:00
Teo Ljungberg
1daed42060
Disable new minitest parallelism () 2021-09-01 07:04:09 -07:00
Stephen Humphries
84dd20d397
Persist live poll status in browser localStorage ()
* Persist live poll status in browser localStorage

* Prefix localStorage variables with 'sidekiq'

* Minor fix for default handling in localStorage timeInterval parsing
2021-08-30 12:31:41 -07:00
Mike Perham
8e36432662 Add optional Sidekiq::Job alias, see 2021-08-20 11:06:00 -07:00
Mike Perham
f1b24da9c2 Rename Sidekiq::Job, reserve for future use 2021-08-20 10:43:55 -07:00
Matt Duszynski
69b2aec57b
Use TIMEOUT constant instead of hardcoded 2 second sleep ()
* Use TIMEOUT constant instead of hardcoded 2 second sleep

* Add a test
2021-08-19 10:52:24 -07:00
Mike Perham
3575ccb44c Reduce retry jitter, add jitter to sidekiq_retry_in, closes 2021-08-09 10:48:16 -07:00
Michael Hauser-Raspe
47ce2a0b38
Quick stats api. ()
* Lazily fetch slow stats later.

* Put calls to lazy load into stat method.

* Add test for workers_size.
2021-07-06 09:13:52 -07:00
Matt Duszynski
02348cdc97
Add Sidekiq::Process#queues method () 2021-06-03 10:02:48 -07:00
Mike Perham
614d41585b Improve logging of .delay jobs, closes 2021-05-24 12:29:45 -07:00
Mike Perham
64f70339d1 Add pessimistic regexp on queue name input to avoid XSS, fixes 2021-03-25 16:16:55 -07:00
Mike Perham
285d6541a5 Log poor Redis RTT, fixes 2021-02-22 15:48:38 -08:00
Mike Perham
50740f253e Restore class-level middleware APIs for backwards compat, fixes 2021-02-21 11:59:01 -08:00
Mike Perham
968bc81043
Improve Web UI session experience ()
* Simplify Web UI sessions

Remove all of the hacks and support infrastructure around Rack sessions. Rails provides this by default so we don't need it for 95% of users. The other 5% need to provide a Rack session.

This is a big change and has the potential to break installs so it deserves at least a minor version bump.

See also ,  and many others.
2021-02-12 14:50:51 -08:00
Matt Robinson
d4fca4683c
Fix logger with from deleting existing context ()
If using `with` to add logging context, any context that has the same
keys as existing context will disappear after the with block completes,
instead of reverting back to the original context.

This fixes that by restoring the original context directly instead of
just deleting the keys from the `with` context.
2020-12-23 12:44:35 -08:00
Mike Perham
4a43f598fb Warn louder 2020-12-02 09:29:10 -08:00
Oliver Peate
c4c01f8894
Warn on boot if maxmemory policy is not 'noeviction' () 2020-12-02 08:50:00 -08:00
Jared Szechy
09305a0909
Fix handling of no session () 2020-11-04 09:09:30 -08:00
Mike Perham
3b5ae30c4e Add process RSS to the Busy page, fixes 2020-10-14 15:11:40 -07:00
Mike Perham
97b9f255a8 handle test edge case 2020-08-28 13:17:03 -07:00
Eugene Kenny
d9cc977f1a
Allow middleware to yield arguments ()
The arguments are always ignored, but this allows the block to be passed
to a method that yields arguments:

    class TaggedLoggingMiddleware
      def call(*, &block)
        Rails.logger.tagged("my cool job", &block)
      end
    end
2020-08-25 09:26:42 -07:00
basherru
df702a16ce
check if session object is not empty in csrf protection ()
Co-authored-by: basherru <alexander.baz@okwork.io>
2020-08-19 10:39:30 -07:00
Mike Perham
2139254f77 Sort busy workers by run_at, fixes
We sort oldest first so long-running jobs will appear at the top; those jobs are typically the ones that developers want to see and diagnose.
2020-07-13 12:42:36 -07:00
Mike Perham
a2374039ef Integrate codecov 2020-06-19 17:10:01 -07:00
Mike Perham
fce05c9d4b
Fetch API refactor, WIP ()
* Fetch API refactor, WIP

* save options for later

* changes

* Fix test failures
2020-06-19 08:39:18 -07:00
Ulysse Buonomo
50b9e67655
raise ArgumentError if 'at' size is wrong ()
In `Sidekiq::Client.bulk_push`, now
'at' size must be the same size as
'args' (or just be a single numeric).

Fixes 
2020-06-18 07:33:13 -07:00
Mike Perham
6a334042ee Migrate to exists? for redis-rb 4.2, fixes 2020-06-09 15:14:02 -07:00
Mike Perham
eb2a6ab497 refactoring 2020-06-03 22:11:18 -07:00
Mike Perham
93bfa9d40e disable backtrace cleaners, dunno where they are coming from... 2020-06-03 16:06:57 -07:00
Mike Perham
47bc87034f Refactor and clean up CSRF protection code
1. Remove unused methods
2. Remove unused and unnecessary features
3. Add basic tests

The session token is stored server-side and never given to the client.
A masked token is generated for each request which can be inserted into forms to be POSTed.
2020-06-03 16:06:36 -07:00
Manoj M J
b75b9491af
Refactor tests () 2020-05-26 11:38:09 -07:00
Manoj M J
a1805e36a0
Validate that args for bulk push is an Array of Arrays () 2020-05-26 11:01:22 -07:00
Manoj M J
26f983b39a
Do not allow to set strict via the config file () 2020-05-07 09:00:19 -07:00
Manoj M J
fe2915ab8b
Fix: Strict ordering must be followed for queues without weights specified by -q ()
Co-authored-by: Mike Perham <mperham@gmail.com>
2020-05-05 13:00:48 -07:00
Manoj M J
316a281d7a
Refactor Sidekiq::CLI to remove a redundant .empty? method call () 2020-05-05 12:57:21 -07:00
Manoj M J
7843815554
Add tests for invalid concurrency and timeout setting () 2020-05-04 09:51:32 -07:00
Geoff Harcourt
dd0a8476ea
Remove SSL parameters from Redis connection logging to avoid exception ()
In 3f9c4bf9 the Redis connection options began to be cloned (via dumping
and re-marshalling) to avoid issues with password redaction in logging
altering the connection options and breaking authentication with Sentinels.

Unfortunately, this change caused an exception on boot for users of
Redis over SSL. The `OpenSSL::X509::Store` object used for SSL certs is
not yet dumpable in the bundled OpenSSL wrapper for current Rubies
(although it does in master as of ).

The fix here prunes the `ssl_params` options out of the Redis
configuration options before the dumping and marshalling. It's probably
better not to include those in logging anyway for privacy purposes.

Fix 
2020-04-18 13:00:29 -07:00
Mike Perham
e3c5551f71 misc, prepare for release 2020-04-17 07:44:51 -07:00
Mike Perham
3f9c4bf90b Deep clone options, fixes
Ensures any password mangling doesn't affect the runtime options
2020-04-01 08:11:44 -07:00
Mike Perham
05f874edea Remove connection naming, fixes 2020-03-23 10:40:44 -07:00
Mike Perham
d4d8b209de
systemd notification integration ()
systemd notification support, fixes 
2020-03-16 07:20:22 -07:00