3.6 KiB

Raw Permalink Blame History

Welcome to Sidekiq 7.0!

Sidekiq 7.0 contains major new features and some breaking changes which streamline proper operation of Sidekiq. Please read these notes carefully.

What's New

Job Metrics

Sidekiq 7.0 adds a new "Metrics" tab in the Web UI with high-resolution data on job execution times along with the ability to mark deploy times. This allows you to monitor job execution times minute by minute and see when a deployment causes a performance regression. See the Metrics wiki page for details.

Embedding

Embedding is an experimental new way to run Sidekiq. Previously, you could only run Sidekiq by starting a new process with bundle exec sidekiq. Now you can embed Sidekiq within another process with just a few lines of Ruby code. I'm labeling this "experimental" because it's quite possible that 3rd party plugins or various edge cases within Sidekiq could break. Feedback is very much appreciated if you try out embedding, even if just "works great for us!". See the Embedding wiki page for details.

Capsules

Sidekiq 7.0 allows you to define "capsules" which execute jobs from queues. The default capsule spins up 5 Threads to fetch jobs from the default queue. You can change these values and/or define your own separate Capsules if you wish to throttle execution from a given queue. For instance, here's how to define a single-threaded capsule for thread-unsafe jobs:

Sidekiq.configure_server do |config|
  # edits the default capsule
  config.queues = %w[critical default low]
  config.concurrency = 5

  # define a new capsule which processes jobs from the `unsafe` queue one at a time
  config.capsule("unsafe") do |cap|
    cap.concurrency = 1
    cap.queues = %w[unsafe]
  end
end

This Sidekiq instance will have 6 threads executing jobs.

Note that Sidekiq's default concurrency has been reduced from 10 to 5. This matches the Rails default concurrency and database pool size. Use RAILS_MAX_THREADS=10 bundle exec sidekiq to adjust Sidekiq's concurrency and the Rails DB pool size:

# config/database.yml
default: &default
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

Remember that your usable concurrency is limited by Ruby's GVL. If you increase your concurrency, you should be monitoring CPU usage to ensure you aren't saturating those CPUs.

redis-client

The redis-client gem is a new Rubygem which uses the RESP3 protocol found in Redis 6.0+. Sidekiq 6.5 introduced beta support for the redis-client gem while defaulting to using the redis gem. Sidekiq 7.0 completes this transition and no longer uses the redis gem internally. Your app can still continue to use redis. If you use Sidekiq.redis to access Redis connections, that API is now exposing a connection based on redis-client.

redis-namespace

Support for redis-namespace has been removed. I have advised against its usage for many years now. One option is to use Redis's numbered databases instead.

Version Support

Redis 6.2+ is now required
Ruby 2.7+ is now required
Rails 6.0+ is now supported

Support is only guaranteed for the current and previous major versions. With the release of Sidekiq 7, Sidekiq 5.x is no longer supported.

Upgrade

As always, please upgrade Sidekiq one major version at a time. If you are already running Sidekiq 6.x, then:

Upgrade to the latest Sidekiq 6.x.

gem 'sidekiq', '< 7'

Fix any deprecation warnings you see.
Upgrade to 7.x.

gem 'sidekiq', '< 8'

3.6 KiB Raw Permalink Blame History