Commit Graph

19 Commits

Author SHA1 Message Date
Yorick Peterse d83ee2bbd1
Add the ability to perform background migrations
Background migrations can be used to perform long running data
migrations without these blocking a deployment procedure.

See MR https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11854 for
more information.
2017-06-12 13:24:04 +02:00
Alexander Randa 330789c23c Implement web hooks logging
* implemented logging of project and system web hooks
* implemented UI for user area (project hooks)
* implemented UI for admin area (system hooks)
* implemented retry of logged webhook
* NOT imeplemented log remover
2017-05-25 10:07:52 +03:00
Toon Claes 0ad80cab40 Use worker to destroy namespaceless projects in post-deploy
Destroying projects can be very time consuming. So instead of destroying them in
the post-deploy, just schedule them and make Sidekiq do the hard work.

They are scheduled in batches of 5000 records. This way the number of database
requests is limited while also the amount data read to memory is limited.
2017-05-10 15:01:27 +02:00
James Lopez 6ecf16b8f7 refactor code based on feedback 2017-05-05 18:01:33 +02:00
James Lopez 2f7f1ce4e6 fix sidekiq spec, add changelog 2017-05-04 11:32:39 +02:00
Nick Thomas e9819de10a Remove the ClearDatabaseCacheWorker 2017-04-21 11:52:28 +01:00
Rémy Coutable d4da926f48 Add new ScheduleUpdateUserActivityWorker and UpdateUserActivityWorker
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-04-14 15:20:55 +02:00
gpongelli 4bcd900f14 Moved call of SystemHooksService from UpdateMergeRequestsWorker to GitPushServic… 2017-03-07 12:59:20 +00:00
Robert Speicher 4c622b71fd Add Upload model and UploadChecksumWorker worker 2017-03-06 14:41:09 -05:00
blackst0ne ff7f74a2f6 Increase process_commit queue weight from 2 to 3 2017-02-17 22:38:22 +11:00
Kamil Trzcinski 732a821d4f Fix specs 2017-01-31 22:50:40 +00:00
Vincent Wong b6df93a51f Record and show last used date of SSH Keys
Addresses: Issue #13810

1. Adds a last_used_at attribute to the Key table/model
2. Update a key's last_used_at whenever it gets used
3. Display how long ago an ssh key was last used
2017-01-09 04:15:39 +11:00
Nick Thomas 5378302763 Add a ReactiveCaching concern for use in the KubernetesService 2016-12-19 19:53:03 +00:00
Ahmad Sherif fd05e26618 Precalculate user's authorized projects in database
Closes #23150
2016-11-18 20:25:45 +02:00
Douwe Maan 374b8e95c3 Merge branch '21076-deleted-merged-branches' into 'master'
Add button to delete all merged branches

## What does this MR do?

It adds a button to the branches page that the user can use to delete all the branches that are already merged. This can be used to clean up all the branches that were forgotten to delete while merging MRs.

**Note**  
~~This MR is WIP until MR !6408 is merged.~~

## Are there points in the code the reviewer needs to double check?

The UX of the actual "Delete merged branches" button.

## Why was this MR needed?

Fixes #21076

## Screenshots
![Branches page without "Delete all merged" button](/uploads/3a2936a83c3547a0fce92a74af880a2d/Screen_Shot_2016-10-17_at_20.06.30.png)

Before:
![Screen_Shot_2016-10-17_at_20.07.11](/uploads/55efcebf4e0a45dbfc70ba4a11ca152c/Screen_Shot_2016-10-17_at_20.07.11.png)


After:

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [x] API support added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Closes #21076

See merge request !6449
2016-11-15 12:08:22 +00:00
Toon Claes 1afab9eb79 Add button to delete all merged branches
It adds a button to the branches page that the user can use to delete
all the branches that are already merged. This can be used to clean up
all the branches that were forgotten to delete while merging MRs.

Fixes #21076.
2016-11-09 21:04:03 +01:00
Yorick Peterse 509910b89f
Process commits in a separate worker
This moves the code used for processing commits from GitPushService to
its own Sidekiq worker: ProcessCommitWorker.

Using a Sidekiq worker allows us to process multiple commits in
parallel. This in turn will lead to issues being closed faster and cross
references being created faster. Furthermore by isolating this code into
a separate class it's easier to test and maintain the code.

The new worker also ensures it can efficiently check which issues can be
closed, without having to run numerous SQL queries for every issue.
2016-11-07 13:11:44 +01:00
Stan Hu d79c41e762 Fix bug where e-mails were not being sent out via Sidekiq
By default, ActionMailer uses the "mailers" queue, but this entry
was not included in the list of queues for Sidekiq to use.

For more details:

* https://github.com/plataformatec/devise/wiki/How-To:-Send-devise-emails-in-background-(Resque,-Sidekiq-and-Delayed::Job)
* http://guides.rubyonrails.org/active_job_basics.html
2016-10-21 22:28:39 -07:00
Yorick Peterse 97731760d7
Re-organize queues to use for Sidekiq
Dumping too many jobs in the same queue (e.g. the "default" queue) is a
dangerous setup. Jobs that take a long time to process can effectively
block any other work from being performed given there are enough of
these jobs.

Furthermore it becomes harder to monitor the jobs as a single queue
could contain jobs for different workers. In such a setup the only
reliable way of getting counts per job is to iterate over all jobs in a
queue, which is a rather time consuming process.

By using separate queues for various workers we have better control over
throughput, we can add weight to queues, and we can monitor queues
better. Some workers still use the same queue whenever their work is
related. For example, the various CI pipeline workers use the same
"pipeline" queue.

This commit includes a Rails migration that moves Sidekiq jobs from the
old queues to the new ones. This migration also takes care of doing the
inverse if ever needed. This does require downtime as otherwise new jobs
could be scheduled in the old queues after this migration completes.

This commit also includes an RSpec test that blacklists the use of the
"default" queue and ensures cron workers use the "cronjob" queue.

Fixes gitlab-org/gitlab-ce#23370
2016-10-21 18:17:07 +02:00