Redis stores its loaded drivers in `Redis::Connection.drivers` and uses the
last one of them when initializing a new client. Sidekiq always uses `'ruby'`
(or `Redis::Connection::Ruby`) per default, though.
With this commit we are following redis' default by passing the last
loaded driver per default (or a given `:driver`).
This is a follow up to #2985 (52828e4) adding similar support for the
client connection pool. For Rails servers, Sidekiq is not loaded from
the CLI so the prior change to support setting the concurrency via
`RAILS_MAX_THREADS` is not applied to the web server process. This means
for Rails servers which do not configure a custom size through an
initializer they will run with the default connection pool size of 5.
When the Rails server runs the initial Redis connection may be made
through `Sidekiq::Client` (e.g. from [`ActiveJob::QueueAdapters::SidekiqAdapter`](https://github.com/rails/rails/blob/v5.1.5/activejob/lib/active_job/queue_adapters/sidekiq_adapter.rb#L20)).
This causes the `redis_pool` to be initialized without any options,
setting the pool size to the default of 5.
.gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq.rb:125:in `redis_pool'
.gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:42:in `initialize'
.gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:131:in `new'
.gem/ruby/2.5.0/gems/sidekiq-5.1.1/lib/sidekiq/client.rb:131:in `push'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/queue_adapters/sidekiq_adapter.rb:20:in `enqueue'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:51:in `block in enqueue'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:108:in `block in run_callbacks'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:15:in `block (3 levels) in <module:Logging>'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:44:in `block in tag_logger'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `block in tagged'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:26:in `tagged'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `tagged'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:44:in `tag_logger'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/logging.rb:14:in `block (2 levels) in <module:Logging>'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:117:in `instance_exec'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
.gem/ruby/2.5.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:135:in `run_callbacks'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:47:in `enqueue'
.gem/ruby/2.5.0/gems/activejob-5.1.5/lib/active_job/enqueuing.rb:18:in `perform_later'
For the majority of cases, a client pool size of 5 is sufficient.
However, servers which utilize a high number of threads, with large job
payloads, and which may experience some network latency issues can see
`Timeout::Error` crashes. This may be further exacerbated by the ~2-20x
performance decrease through `ActiveJob` (#3782). Rails addresses this
general type of connection issue for the main database by suggesting
that the DB pool size match the number of threads running. This change
applies that logic to the default client pool size by leveraging the
same environment setting; this way there's a connection available per
thread.
This may also have the side effect of a slight performance boost, as
there is less of a chance that threads will be blocked waiting on
connections. The trade-off is that there may be a memory profile
increase to handle the additional Redis connections in the pool; note
the pool only creates new connections as necessary to handle the
requests.
Resolves#3806
* Loosen Redis version dep to `>= 3.3.5, < 5`
* Bump redis-namespace for looser Redis version dep. Pending
https://github.com/resque/redis-namespace/pull/136 gem release.
Use `redis.connection` where we can and fall back to `redis._client`
where we need (to inspect timeout and scheme).
References #3617
Fixes an edge case where a stringified `namespace` key
is ignored. A `namespace` key may be a string if
the Sidekiq Redis config is loaded from a yml file.
Also minor, syntactical improvements to error messaging.
Existing tests pass. Added a new test to address
this edge-case.
Replaced with `__send__` from std lib
* `send` can now be overridden to be more semantically meaningful
* `message.send(user_id)` as opposed to `message.send_message(user_id)`
* `__send__` makes it clear that the reflective version is intended
