2014-11-11 09:49:26 -05:00
|
|
|
module MergeRequests
|
|
|
|
class RefreshService < MergeRequests::BaseService
|
|
|
|
def execute(oldrev, newrev, ref)
|
2015-03-10 06:51:36 -04:00
|
|
|
return true unless Gitlab::Git.branch_ref?(ref)
|
2014-11-11 09:49:26 -05:00
|
|
|
|
2014-12-02 10:42:56 -05:00
|
|
|
@oldrev, @newrev = oldrev, newrev
|
2015-03-10 06:51:36 -04:00
|
|
|
@branch_name = Gitlab::Git.ref_name(ref)
|
2015-10-21 11:34:12 -04:00
|
|
|
|
2017-11-08 15:39:29 -05:00
|
|
|
Gitlab::GitalyClient.allow_n_plus_1_calls(&method(:find_new_commits))
|
2015-11-06 02:47:04 -05:00
|
|
|
# Be sure to close outstanding MRs before reloading them to avoid generating an
|
|
|
|
# empty diff during a manual merge
|
2018-01-29 10:56:19 -05:00
|
|
|
close_upon_missing_source_branch_ref
|
|
|
|
post_merge_manually_merged
|
2015-10-21 11:24:09 -04:00
|
|
|
reload_merge_requests
|
2017-02-17 08:56:13 -05:00
|
|
|
reset_merge_when_pipeline_succeeds
|
2016-03-08 13:22:50 -05:00
|
|
|
mark_pending_todos_done
|
Improve performance of the cycle analytics page.
1. These changes bring down page load time for 100 issues from more than
a minute to about 1.5 seconds.
2. This entire commit is composed of these types of performance
enhancements:
- Cache relevant data in `IssueMetrics` wherever possible.
- Cache relevant data in `MergeRequestMetrics` wherever possible.
- Preload metrics
3. Given these improvements, we now only need to make 4 SQL calls:
- Load all issues
- Load all merge requests
- Load all metrics for the issues
- Load all metrics for the merge requests
4. A list of all the data points that are now being pre-calculated:
a. The first time an issue is mentioned in a commit
- In `GitPushService`, find all issues mentioned by the given commit
using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
flag for each of them.
- There seems to be a (pre-existing) bug here - files (and
therefore commits) created using the Web CI don't have
cross-references created, and issues are not closed even when
the commit title is "Fixes #xx".
b. The first time a merge request is deployed to production
When a `Deployment` is created, find all merge requests that
were merged in before the deployment, and set the
`first_deployed_to_production_at` flag for each of them.
c. The start / end time for a merge request pipeline
Hook into the `Pipeline` state machine. When the `status` moves to
`running`, find the merge requests whose tip commit matches the
pipeline, and record the `latest_build_started_at` time for each
of them. When the `status` moves to `success`, record the
`latest_build_finished_at` time.
d. The merge requests that close an issue
- This was a big cause of the performance problems we were having
with Cycle Analytics. We need to use `ReferenceExtractor` to make
this calculation, which is slow when we have to run it on a large
number of merge requests.
- When a merge request is created, updated, or refreshed, find the
issues it closes, and create an instance of
`MergeRequestsClosingIssues`, which acts as a join model between
merge requests and issues.
- If a `MergeRequestsClosingIssues` instance links a merge request
and an issue, that issue closes that merge request.
5. The `Queries` module was changed into a class, so we can cache the
results of `issues` and `merge_requests_closing_issues` across
various cycle analytics stages.
6. The code added in this commit is untested. Tests will be added in the
next commit.
2016-09-15 04:59:36 -04:00
|
|
|
cache_merge_requests_closing_issues
|
2015-10-15 04:41:46 -04:00
|
|
|
|
2015-10-21 11:34:12 -04:00
|
|
|
# Leave a system note if a branch was deleted/added
|
|
|
|
if branch_added? || branch_removed?
|
2015-10-16 01:45:06 -04:00
|
|
|
comment_mr_branch_presence_changed
|
2015-10-15 04:41:46 -04:00
|
|
|
end
|
2014-11-11 09:49:26 -05:00
|
|
|
|
2015-11-06 02:47:04 -05:00
|
|
|
comment_mr_with_commits
|
2016-12-15 15:48:26 -05:00
|
|
|
mark_mr_as_wip_from_commits
|
2015-02-26 03:17:25 -05:00
|
|
|
execute_mr_web_hooks
|
2014-11-11 09:49:26 -05:00
|
|
|
|
|
|
|
true
|
|
|
|
end
|
|
|
|
|
|
|
|
private
|
|
|
|
|
2018-01-29 10:56:19 -05:00
|
|
|
def close_upon_missing_source_branch_ref
|
|
|
|
# MergeRequest#reload_diff ignores not opened MRs. This means it won't
|
|
|
|
# create an `empty` diff for `closed` MRs without a source branch, keeping
|
|
|
|
# the latest diff state as the last _valid_ one.
|
|
|
|
merge_requests_for_source_branch.reject(&:source_branch_exists?).each do |mr|
|
|
|
|
MergeRequests::CloseService
|
|
|
|
.new(mr.target_project, @current_user)
|
|
|
|
.execute(mr)
|
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2014-11-11 09:49:26 -05:00
|
|
|
# Collect open merge requests that target same branch we push into
|
|
|
|
# and close if push to master include last commit from merge request
|
|
|
|
# We need this to close(as merged) merge requests that were merged into
|
|
|
|
# target branch manually
|
2018-01-29 10:56:19 -05:00
|
|
|
def post_merge_manually_merged
|
2014-11-11 09:49:26 -05:00
|
|
|
commit_ids = @commits.map(&:id)
|
Use latest_merge_request_diff association
Compared to the merge_request_diff association:
1. It's simpler to query. The query uses a foreign key to the
merge_request_diffs table, so no ordering is necessary.
2. It's faster for preloading. The merge_request_diff association has to load
every diff for the MRs in the set, then discard all but the most recent for
each. This association means that Rails can just query for N diffs from N
MRs.
3. It's more complicated to update. This is a bidirectional foreign key, so we
need to update two tables when adding a diff record. This also means we need
to handle this as a special case when importing a GitLab project.
There is some juggling with this association in the merge request model:
* `MergeRequest#latest_merge_request_diff` is _always_ the latest diff.
* `MergeRequest#merge_request_diff` reuses
`MergeRequest#latest_merge_request_diff` unless:
* Arguments are passed. These are typically to force-reload the association.
* It doesn't exist. That means we might be trying to implicitly create a
diff. This only seems to happen in specs.
* The association is already loaded. This is important for the reasons
explained in the comment, which I'll reiterate here: if we a) load a
non-latest diff, then b) get its `merge_request`, then c) get that MR's
`merge_request_diff`, we should get the diff we loaded in c), even though
that's not the latest diff.
Basically, `MergeRequest#merge_request_diff` is the latest diff in most cases,
but not quite all.
2017-11-15 12:22:18 -05:00
|
|
|
merge_requests = @project.merge_requests.preload(:latest_merge_request_diff).opened.where(target_branch: @branch_name).to_a
|
2016-06-20 12:48:04 -04:00
|
|
|
merge_requests = merge_requests.select(&:diff_head_commit)
|
2014-11-11 09:49:26 -05:00
|
|
|
|
|
|
|
merge_requests = merge_requests.select do |merge_request|
|
2017-07-07 10:09:29 -04:00
|
|
|
commit_ids.include?(merge_request.diff_head_sha) &&
|
|
|
|
merge_request.merge_request_diff.state != 'empty'
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
|
|
|
|
2017-01-13 08:35:33 -05:00
|
|
|
filter_merge_requests(merge_requests).each do |merge_request|
|
2017-06-21 09:48:12 -04:00
|
|
|
MergeRequests::PostMergeService
|
|
|
|
.new(merge_request.target_project, @current_user)
|
|
|
|
.execute(merge_request)
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2014-12-02 10:42:56 -05:00
|
|
|
def force_push?
|
2016-07-18 04:16:56 -04:00
|
|
|
Gitlab::Checks::ForcePush.force_push?(@project, @oldrev, @newrev)
|
2014-12-02 10:42:56 -05:00
|
|
|
end
|
|
|
|
|
2014-11-11 09:49:26 -05:00
|
|
|
# Refresh merge request diff if we push to source or target branch of merge request
|
|
|
|
# Note: we should update merge requests from forks too
|
|
|
|
def reload_merge_requests
|
2017-06-21 09:48:12 -04:00
|
|
|
merge_requests = @project.merge_requests.opened
|
|
|
|
.by_source_or_target_branch(@branch_name).to_a
|
2015-10-15 04:57:45 -04:00
|
|
|
|
2017-01-13 08:35:33 -05:00
|
|
|
# Fork merge requests
|
|
|
|
merge_requests += MergeRequest.opened
|
|
|
|
.where(source_branch: @branch_name, source_project: @project)
|
|
|
|
.where.not(target_project: @project).to_a
|
|
|
|
|
|
|
|
filter_merge_requests(merge_requests).each do |merge_request|
|
2016-11-18 05:14:44 -05:00
|
|
|
if merge_request.source_branch == @branch_name || force_push?
|
2017-05-21 16:38:33 -04:00
|
|
|
merge_request.reload_diff(current_user)
|
2016-11-18 05:14:44 -05:00
|
|
|
else
|
2017-06-16 10:00:58 -04:00
|
|
|
mr_commit_ids = merge_request.commit_shas
|
2016-11-18 05:14:44 -05:00
|
|
|
push_commit_ids = @commits.map(&:id)
|
|
|
|
matches = mr_commit_ids & push_commit_ids
|
2017-05-21 16:38:33 -04:00
|
|
|
merge_request.reload_diff(current_user) if matches.any?
|
2016-11-18 05:14:44 -05:00
|
|
|
end
|
|
|
|
|
2016-06-30 07:20:15 -04:00
|
|
|
merge_request.mark_as_unchecked
|
2017-09-07 03:58:15 -04:00
|
|
|
UpdateHeadPipelineForMergeRequestWorker.perform_async(merge_request.id)
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
2018-02-01 16:17:30 -05:00
|
|
|
|
|
|
|
# Upcoming method calls need the refreshed version of
|
|
|
|
# @source_merge_requests diffs (for MergeRequest#commit_shas for instance).
|
|
|
|
merge_requests_for_source_branch(reload: true)
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
|
|
|
|
2017-02-17 08:56:13 -05:00
|
|
|
def reset_merge_when_pipeline_succeeds
|
|
|
|
merge_requests_for_source_branch.each(&:reset_merge_when_pipeline_succeeds)
|
2015-11-02 11:27:38 -05:00
|
|
|
end
|
|
|
|
|
2016-03-08 13:22:50 -05:00
|
|
|
def mark_pending_todos_done
|
|
|
|
merge_requests_for_source_branch.each do |merge_request|
|
|
|
|
todo_service.merge_request_push(merge_request, @current_user)
|
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2015-10-21 11:34:12 -04:00
|
|
|
def find_new_commits
|
|
|
|
if branch_added?
|
|
|
|
@commits = []
|
2015-10-16 01:45:06 -04:00
|
|
|
|
2015-10-21 11:34:12 -04:00
|
|
|
merge_request = merge_requests_for_source_branch.first
|
|
|
|
return unless merge_request
|
2015-10-16 01:45:06 -04:00
|
|
|
|
2015-10-21 11:34:12 -04:00
|
|
|
begin
|
|
|
|
# Since any number of commits could have been made to the restored branch,
|
|
|
|
# find the common root to see what has been added.
|
2016-06-20 12:48:04 -04:00
|
|
|
common_ref = @project.repository.merge_base(merge_request.diff_head_sha, @newrev)
|
2015-10-21 11:34:12 -04:00
|
|
|
# If the a commit no longer exists in this repo, gitlab_git throws
|
|
|
|
# a Rugged::OdbError. This is fixed in https://gitlab.com/gitlab-org/gitlab_git/merge_requests/52
|
|
|
|
@commits = @project.repository.commits_between(common_ref, @newrev) if common_ref
|
|
|
|
rescue
|
2015-10-16 01:45:06 -04:00
|
|
|
end
|
2015-10-21 11:34:12 -04:00
|
|
|
elsif branch_removed?
|
|
|
|
# No commits for a deleted branch.
|
|
|
|
@commits = []
|
|
|
|
else
|
|
|
|
@commits = @project.repository.commits_between(@oldrev, @newrev)
|
|
|
|
end
|
|
|
|
end
|
2015-10-16 01:45:06 -04:00
|
|
|
|
2015-10-21 11:34:12 -04:00
|
|
|
# Add comment about branches being deleted or added to merge requests
|
|
|
|
def comment_mr_branch_presence_changed
|
|
|
|
presence = branch_added? ? :add : :delete
|
|
|
|
|
|
|
|
merge_requests_for_source_branch.each do |merge_request|
|
2015-10-15 04:41:46 -04:00
|
|
|
SystemNoteService.change_branch_presence(
|
2015-12-14 21:53:52 -05:00
|
|
|
merge_request, merge_request.project, @current_user,
|
2015-10-15 05:08:22 -04:00
|
|
|
:source, @branch_name, presence)
|
2015-10-15 04:41:46 -04:00
|
|
|
end
|
|
|
|
end
|
|
|
|
|
|
|
|
# Add comment about pushing new commits to merge requests
|
|
|
|
def comment_mr_with_commits
|
2015-10-21 11:34:12 -04:00
|
|
|
return unless @commits.present?
|
|
|
|
|
2015-10-15 05:24:30 -04:00
|
|
|
merge_requests_for_source_branch.each do |merge_request|
|
2017-06-16 10:00:58 -04:00
|
|
|
mr_commit_ids = Set.new(merge_request.commit_shas)
|
2015-03-06 11:14:19 -05:00
|
|
|
|
|
|
|
new_commits, existing_commits = @commits.partition do |commit|
|
|
|
|
mr_commit_ids.include?(commit.id)
|
|
|
|
end
|
|
|
|
|
2015-05-09 18:18:50 -04:00
|
|
|
SystemNoteService.add_commits(merge_request, merge_request.project,
|
|
|
|
@current_user, new_commits,
|
|
|
|
existing_commits, @oldrev)
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2016-12-15 15:48:26 -05:00
|
|
|
def mark_mr_as_wip_from_commits
|
|
|
|
return unless @commits.present?
|
|
|
|
|
|
|
|
merge_requests_for_source_branch.each do |merge_request|
|
2017-06-16 10:00:58 -04:00
|
|
|
commit_shas = merge_request.commit_shas
|
2017-02-09 05:34:02 -05:00
|
|
|
|
|
|
|
wip_commit = @commits.detect do |commit|
|
|
|
|
commit.work_in_progress? && commit_shas.include?(commit.sha)
|
|
|
|
end
|
2016-12-15 15:48:26 -05:00
|
|
|
|
|
|
|
if wip_commit && !merge_request.work_in_progress?
|
|
|
|
merge_request.update(title: merge_request.wip_title)
|
|
|
|
SystemNoteService.add_merge_request_wip_from_commit(
|
|
|
|
merge_request,
|
|
|
|
merge_request.project,
|
|
|
|
@current_user,
|
|
|
|
wip_commit
|
|
|
|
)
|
|
|
|
end
|
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2015-02-26 03:17:25 -05:00
|
|
|
# Call merge request webhook with update branches
|
|
|
|
def execute_mr_web_hooks
|
2015-10-15 05:24:30 -04:00
|
|
|
merge_requests_for_source_branch.each do |merge_request|
|
2017-09-15 13:08:27 -04:00
|
|
|
execute_hooks(merge_request, 'update', old_rev: @oldrev)
|
2015-02-26 03:17:25 -05:00
|
|
|
end
|
|
|
|
end
|
|
|
|
|
Improve performance of the cycle analytics page.
1. These changes bring down page load time for 100 issues from more than
a minute to about 1.5 seconds.
2. This entire commit is composed of these types of performance
enhancements:
- Cache relevant data in `IssueMetrics` wherever possible.
- Cache relevant data in `MergeRequestMetrics` wherever possible.
- Preload metrics
3. Given these improvements, we now only need to make 4 SQL calls:
- Load all issues
- Load all merge requests
- Load all metrics for the issues
- Load all metrics for the merge requests
4. A list of all the data points that are now being pre-calculated:
a. The first time an issue is mentioned in a commit
- In `GitPushService`, find all issues mentioned by the given commit
using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
flag for each of them.
- There seems to be a (pre-existing) bug here - files (and
therefore commits) created using the Web CI don't have
cross-references created, and issues are not closed even when
the commit title is "Fixes #xx".
b. The first time a merge request is deployed to production
When a `Deployment` is created, find all merge requests that
were merged in before the deployment, and set the
`first_deployed_to_production_at` flag for each of them.
c. The start / end time for a merge request pipeline
Hook into the `Pipeline` state machine. When the `status` moves to
`running`, find the merge requests whose tip commit matches the
pipeline, and record the `latest_build_started_at` time for each
of them. When the `status` moves to `success`, record the
`latest_build_finished_at` time.
d. The merge requests that close an issue
- This was a big cause of the performance problems we were having
with Cycle Analytics. We need to use `ReferenceExtractor` to make
this calculation, which is slow when we have to run it on a large
number of merge requests.
- When a merge request is created, updated, or refreshed, find the
issues it closes, and create an instance of
`MergeRequestsClosingIssues`, which acts as a join model between
merge requests and issues.
- If a `MergeRequestsClosingIssues` instance links a merge request
and an issue, that issue closes that merge request.
5. The `Queries` module was changed into a class, so we can cache the
results of `issues` and `merge_requests_closing_issues` across
various cycle analytics stages.
6. The code added in this commit is untested. Tests will be added in the
next commit.
2016-09-15 04:59:36 -04:00
|
|
|
# If the merge requests closes any issues, save this information in the
|
|
|
|
# `MergeRequestsClosingIssues` model (as a performance optimization).
|
|
|
|
def cache_merge_requests_closing_issues
|
2016-09-20 16:45:02 -04:00
|
|
|
@project.merge_requests.where(source_branch: @branch_name).each do |merge_request|
|
Improve performance of the cycle analytics page.
1. These changes bring down page load time for 100 issues from more than
a minute to about 1.5 seconds.
2. This entire commit is composed of these types of performance
enhancements:
- Cache relevant data in `IssueMetrics` wherever possible.
- Cache relevant data in `MergeRequestMetrics` wherever possible.
- Preload metrics
3. Given these improvements, we now only need to make 4 SQL calls:
- Load all issues
- Load all merge requests
- Load all metrics for the issues
- Load all metrics for the merge requests
4. A list of all the data points that are now being pre-calculated:
a. The first time an issue is mentioned in a commit
- In `GitPushService`, find all issues mentioned by the given commit
using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
flag for each of them.
- There seems to be a (pre-existing) bug here - files (and
therefore commits) created using the Web CI don't have
cross-references created, and issues are not closed even when
the commit title is "Fixes #xx".
b. The first time a merge request is deployed to production
When a `Deployment` is created, find all merge requests that
were merged in before the deployment, and set the
`first_deployed_to_production_at` flag for each of them.
c. The start / end time for a merge request pipeline
Hook into the `Pipeline` state machine. When the `status` moves to
`running`, find the merge requests whose tip commit matches the
pipeline, and record the `latest_build_started_at` time for each
of them. When the `status` moves to `success`, record the
`latest_build_finished_at` time.
d. The merge requests that close an issue
- This was a big cause of the performance problems we were having
with Cycle Analytics. We need to use `ReferenceExtractor` to make
this calculation, which is slow when we have to run it on a large
number of merge requests.
- When a merge request is created, updated, or refreshed, find the
issues it closes, and create an instance of
`MergeRequestsClosingIssues`, which acts as a join model between
merge requests and issues.
- If a `MergeRequestsClosingIssues` instance links a merge request
and an issue, that issue closes that merge request.
5. The `Queries` module was changed into a class, so we can cache the
results of `issues` and `merge_requests_closing_issues` across
various cycle analytics stages.
6. The code added in this commit is untested. Tests will be added in the
next commit.
2016-09-15 04:59:36 -04:00
|
|
|
merge_request.cache_merge_request_closes_issues!(@current_user)
|
|
|
|
end
|
|
|
|
end
|
|
|
|
|
2014-11-11 09:49:26 -05:00
|
|
|
def filter_merge_requests(merge_requests)
|
|
|
|
merge_requests.uniq.select(&:source_project)
|
|
|
|
end
|
2015-10-15 04:41:46 -04:00
|
|
|
|
2018-02-01 16:17:30 -05:00
|
|
|
def merge_requests_for_source_branch(reload: false)
|
|
|
|
@source_merge_requests = nil if reload
|
2017-01-13 08:35:33 -05:00
|
|
|
@source_merge_requests ||= merge_requests_for(@branch_name)
|
2015-10-21 11:34:12 -04:00
|
|
|
end
|
|
|
|
|
|
|
|
def branch_added?
|
|
|
|
Gitlab::Git.blank_ref?(@oldrev)
|
|
|
|
end
|
|
|
|
|
|
|
|
def branch_removed?
|
|
|
|
Gitlab::Git.blank_ref?(@newrev)
|
|
|
|
end
|
2014-11-11 09:49:26 -05:00
|
|
|
end
|
|
|
|
end
|