Commit Graph

31 Commits

Author SHA1 Message Date
Douglas Barbosa Alexandre 3c09000e18 Does not halt the GitHub import process when an error occurs 2016-08-24 12:16:42 -03:00
Yorick Peterse d345591fc8
Tracking of custom events
GitLab Performance Monitoring is now able to track custom events not
directly related to application performance. These events include the
number of tags pushed, repositories created, builds registered, etc.

The use of these events is to get a better overview of how a GitLab
instance is used and how that may affect performance. For example, a
large number of Git pushes may have a negative impact on the underlying
storage engine.

Events are stored in the "events" measurement and are not prefixed with
"rails_" or "sidekiq_", this makes it easier to query events with the
same name triggered from different parts of the application. All events
being stored in the same measurement also makes it easier to downsample
data.

Currently the following events are tracked:

* Creating repositories
* Removing repositories
* Changing the default branch of a repository
* Pushing a new tag
* Removing an existing tag
* Pushing a commit (along with the branch being pushed to)
* Pushing a new branch
* Removing an existing branch
* Importing a repository (along with the URL we're importing)
* Forking a repository (along with the source/target path)
* CI builds registered (and when no build could be found)
* CI builds being updated
* Rails and Sidekiq exceptions

Fixes gitlab-org/gitlab-ce#13720
2016-08-17 10:04:04 +02:00
James Lopez c9e8acd058 Update repository_import_worker.rb 2016-06-02 21:48:27 -04:00
James Lopez 64c3905523 some refactoring and fixing spec 2016-06-02 21:48:25 -04:00
Rubén Dávila 0c47b68d04 Mask credentials from URL when import of project has failed. 2016-05-18 21:16:36 -05:00
Yorick Peterse ff28a7cc36 Moved cache expiration code to Repository hooks
This keeps all the cache expiration code in a single file/class instead
of spreading it all across the codebase.
2016-02-23 12:02:59 +01:00
Yorick Peterse a9e0301c23 Expire caches after forking/importing a repository
This ensures the caches for Repository#empty? and
Repository#has_visible_content? are flushed after a repository has been
imported or forked.

Fixes gitlab-org/gitlab-ce#13505
2016-02-17 11:38:18 +01:00
Douglas Barbosa Alexandre 6dd88e090e Extract Projects::ImportService service from RepositoryImportWorker 2016-01-25 22:58:30 -02:00
Rubén Dávila 2ed48df79a Remove accidentally added line. #3598
It should exist in EE only.
2015-11-20 16:50:48 -05:00
Douwe Maan 5e8ddd3c2d Remove unused variable in repository import 2015-11-18 12:02:09 +01:00
Douwe Maan 841a7c6897 Store and show reason why import failed. 2015-11-18 12:00:56 +01:00
Jared Szechy e156f42079 FogBugz project import 2015-09-08 20:23:01 -04:00
Stan Hu ed1d4fa477 Remove user OAuth tokens stored in database for Bitbucket, GitHub, and GitLab
and request them each session. Pass these tokens to the project import data.

This prevents the need to encrypt these tokens and clear them in case they
expire or get revoked.

For example, if you deleted and re-created OAuth2 keys for Bitbucket, you would get
an Error 500 with no way to recover:

```
Started GET "/import/bitbucket/status" for x.x.x.x at 2015-08-07 05:24:10 +0000
Processing by Import::BitbucketController#status as HTML
Completed 500 Internal Server Error in 607ms (ActiveRecord: 2.3ms)

NameError (uninitialized constant Import::BitbucketController::Unauthorized):
  app/controllers/import/bitbucket_controller.rb:77:in `rescue in go_to_bitbucket_for_permissions'
  app/controllers/import/bitbucket_controller.rb:74:in `go_to_bitbucket_for_permissions'
  app/controllers/import/bitbucket_controller.rb:86:in `bitbucket_unauthorized'
```

Closes #1871
2015-08-23 09:23:44 -07:00
Dmitriy Zaporozhets 3e259ea8bc Revert "Merge branch 'revert-satellites' into 'master' "
This reverts commit 5daf44b7c8, reversing
changes made to 2f706fbd23.
2015-08-11 14:33:31 +02:00
Dmitriy Zaporozhets 9f10943c1a
Revert "Merge branch 'drop-satellites'"
This reverts commit 957e849f41, reversing
changes made to 6b9dbe9f5a.

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2015-08-11 10:28:42 +02:00
Dmitriy Zaporozhets b48b07044b
Merge branch 'master' into drop-satellites
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2015-08-04 16:42:36 +02:00
Dmitriy Zaporozhets f0116f1335
Store commit count in project table
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2015-07-17 15:22:56 +02:00
Dmitriy Zaporozhets a87989fb7d Remove satellites 2015-07-15 15:45:57 +02:00
Douwe Maan 7b5bc32cad Allow projects to be imported from Google Code. 2015-04-03 15:29:27 +02:00
Douwe Maan 6979b3afd5 Delete deploy key from Bitbucket after importing. 2015-02-24 15:07:24 +01:00
Douwe Maan ad6d623234 Add Bitbucket importer. 2015-02-24 15:07:24 +01:00
Valery Sizov 1ac20698a5 gitlab.com importer: refactorig 2015-02-05 17:03:43 -08:00
Valery Sizov 18231b0bb3 GitLab.com integration: refactoring 2015-02-05 12:50:34 -08:00
Valery Sizov 5194214e3a GitLab integration. Importer 2015-02-05 12:50:34 -08:00
Valery Sizov a9f7fd2c1a Github Importer 2015-01-10 09:51:43 -08:00
Ciro Santilli 2e9f5de868 Add parenthesis to function def with arguments. 2014-10-03 09:18:46 +02:00
CSchulz ac0763ac7a Added update repository size to repository import 2014-06-11 19:32:00 +00:00
Dmitriy Zaporozhets b47646ef13
Add retry feature to project import
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2014-03-12 19:26:09 +02:00
Dmitriy Zaporozhets 1436433c7b
Add project import state machine
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2014-03-12 17:15:03 +02:00
Dmitriy Zaporozhets 02484930e1
Removed Project#discover_default_branch since it not needed any more
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
2013-11-08 09:22:50 +02:00
Dmitriy Zaporozhets 7ecebdd02d Repository import during project creation often return timeout for medium and large repos.
So lets do it async. First create project, then import repo and create
satellite with Sidekiq
2013-08-12 18:21:47 +03:00