Commit Graph

19 Commits

Author SHA1 Message Date
Patricio Cano b95216aaba Allow the Sidekiq queues to throttle and the factor by which to throttle them to be configurable 2016-11-10 11:38:11 -06:00
Patricio Cano 9e2964c15a Allow certain Sidekiq jobs to be throttled 2016-11-10 11:38:10 -06:00
Drew Blessing d498ec98e0 Set default Sidekiq retries to 3
By default, Sidekiq will retry 25 times with an exponential backoff.
This may result in jobs retrying for up to 21 days. Most Sidekiq
failures occur when attempting to connect to external services -
Project service hooks, web hooks, mailers, mirror updates, etc.
We should set a default retry of 3, and if that's not sufficient
individual workers can override this in the worker class.
2016-11-04 14:58:02 -05:00
Stan Hu cde3963dae Initialize Sidekiq with the list of queues used by GitLab
The Sidekiq client API adds an entry to the Sidekiq "queues" list,
but mail_room and gitlab-shell use redis-rb directly to insert jobs
into Redis and thus do not make an extra "sadd" call to Redis
each time a job is inserted. To make it possible to monitor
these queues via the API, add an initialization step to
set up the list at startup.

Closes gitlab-com/infrastructure#682
2016-11-01 07:03:25 -07:00
Gabriel Mazetto 2625a4ccbd Make sidekiq get config settings from Gitlab::RedisConfig 2016-08-04 18:55:37 +02:00
Stan Hu 22386040fe Enable SIDEKIQ_REQUEST_STORE by default 2016-07-25 18:05:14 -07:00
Stan Hu f4aac77389 Add support for using RequestStore within Sidekiq tasks via SIDEKIQ_REQUEST_STORE env variable
This significantly reduces the DB churn in the PostReceive task when it
performs reference extraction.

See #18663
2016-07-25 17:59:09 -07:00
Gabriel Mazetto d7ab81cf7d Log cron_jobs configuration instead of raising exception 2016-07-20 12:50:42 +02:00
Gabriel Mazetto aaba9ebbb1 Improve cron_jobs loading error messages 2016-07-18 23:52:58 +02:00
Grzegorz Bizon 28bafd5354 Enable Style/SpaceAfterComma Rubocop cop 2016-06-29 15:23:44 +02:00
Rémy Coutable 3cb1e96ef6
Avoid autoload issue such as 'Mail::Parsers::AddressStruct'
By eager-loading the Mail gem in the Sidekiq initializer.

Signed-off-by: Rémy Coutable <remy@rymai.me>
2016-06-20 12:43:29 +02:00
Valery Sizov 2244aaf98f Redis configuration consistency 2016-04-13 20:57:19 +03:00
Jacob Vosmaer 0163e27631 Add Gitlab::Redis connection pool 2016-04-04 17:00:09 +02:00
Jacob Vosmaer c46e6eff10 Parse config/resque.yml in one place only 2016-03-09 15:16:46 +01:00
Gabriel Mazetto a7a52f7567 Hotfix for sidekiq-cron being loaded from Settingslogic with defaults 2015-12-22 13:02:42 -02:00
Gabriel Mazetto 51ef4a2fed Sidekiq-cron configuration moved to gitlab.yml 2015-12-14 14:17:20 -02:00
Gabriel Mazetto 3e5b24d46a Upgraded Sidekiq to 4.x 2015-12-10 17:18:03 -02:00
Gabriel Mazetto 496870ddec Migrate from Sidetiq to Sidekiq-cron
Updated Sidekiq to 3.5.x
2015-12-04 11:29:45 -02:00
Douwe Maan 046b283127 Groundwork for merging CI into CE 2015-08-25 18:42:46 -07:00