Commit graph

18 commits

Author SHA1 Message Date
Yorick Peterse
97731760d7
Re-organize queues to use for Sidekiq
Dumping too many jobs in the same queue (e.g. the "default" queue) is a
dangerous setup. Jobs that take a long time to process can effectively
block any other work from being performed given there are enough of
these jobs.

Furthermore it becomes harder to monitor the jobs as a single queue
could contain jobs for different workers. In such a setup the only
reliable way of getting counts per job is to iterate over all jobs in a
queue, which is a rather time consuming process.

By using separate queues for various workers we have better control over
throughput, we can add weight to queues, and we can monitor queues
better. Some workers still use the same queue whenever their work is
related. For example, the various CI pipeline workers use the same
"pipeline" queue.

This commit includes a Rails migration that moves Sidekiq jobs from the
old queues to the new ones. This migration also takes care of doing the
inverse if ever needed. This does require downtime as otherwise new jobs
could be scheduled in the old queues after this migration completes.

This commit also includes an RSpec test that blacklists the use of the
"default" queue and ensures cron workers use the "cronjob" queue.

Fixes gitlab-org/gitlab-ce#23370
2016-10-21 18:17:07 +02:00
Lin Jen-Shin
c55eebb283 An instruction for what to do 2016-06-15 15:49:01 +08:00
Lin Jen-Shin
cc69bd07e7 This is easier to read for me. No early return
76dbafba86 (note_4575102)
2016-05-24 15:16:16 +08:00
Lin Jen-Shin
1f5d55907a Merge the places where exceptions could be raised 2016-05-24 01:23:07 +08:00
Lin Jen-Shin
c337e748d3 so we use separate classes to handle different tasks 2016-05-18 17:25:45 -05:00
Lin Jen-Shin
8156475ea5 Report better errors. TODO: Enable skipped test 2016-05-16 21:27:16 +00:00
Lin Jen-Shin
eac3583340 Update wording because we're overloading UserNotAuthorizedError 2016-05-16 21:27:16 +00:00
Lin Jen-Shin
034e752aa4 Handle InvalidIssueError as InvalidNoteError 2016-05-16 21:27:16 +00:00
Valery Sizov
e92ceb7b57 fix specs 2015-11-30 16:12:31 +02:00
Valery Sizov
7f214cee74 Migrate mailers to ActiveJob 2015-11-26 17:03:43 +02:00
Douwe Maan
ee028d9d60 Rename reply_by_email to incoming_email to prepare for the future. 2015-09-21 10:35:37 +02:00
Douwe Maan
2088880383 Ignore empty incoming messages. 2015-08-24 10:57:35 -07:00
Douwe Maan
69708dab9f Block blocked users from replying to threads by email. 2015-08-21 10:14:45 -07:00
Douwe Maan
2f78b5e8af Make error class names more consistent. 2015-08-20 11:33:18 -07:00
Douwe Maan
e9972efc2f Extract ReplyParser and AttachmentUploader from Receiver. 2015-08-20 11:05:06 -07:00
Douwe Maan
76dbafba86 Send a rejection email when the incoming email couldn't be processed. 2015-08-19 11:10:21 -07:00
Douwe Maan
8906cabae7 Changes and stuff. 2015-08-18 17:02:26 -07:00
Douwe Maan
f76eac56b9 Reply by email POC 2015-08-18 15:46:36 -07:00