Instead of checking if a commit already exists in the upstream project
in its ProcessCommitWorker and bailing out if it does, we check the
existence of all commits in bulk in Git::BranchHooksService, so that we
can skip scheduling ProcessCommitWorker jobs for those commits
that already exist upstream entirely.
Whenever `PostReceive` is enqueued, `UpdateMergeRequestsWorker`
is enqueued and `MergeRequests::RefreshService` is called, it'll
check if the source branch of each MR asssociated to the push exists
or not via `MergeRequest#source_branch_exists?`. The said method will
call `Repository#branch_exists?` which is cached in `Rails.cache`.
When the cache contains outdated data and the source branch actually
exists, the `MergeRequests#RefreshService` job will close associated
MRs which is not correct.
The fix is to expire the branches cache of the project so we have
updated data during the post receive hook which will help in the
accuracy of the check if we need to close associated MRs or not.
In certain cases, GitLab can miss a PostReceive invocation the first
time a branch is pushed. When this happens, the "branch created" hooks
are not run, which means various features don't work until the branch
is deleted and pushed again.
This MR changes the `Git::BranchPushService` so it checks the cache of
existing branches in addition to the `oldrev` reported for the branch.
If the branch name isn't in the cache, chances are we haven't run the
service yet (it's what refreshes the cache), so we can go ahead and
run it, even through `oldrev` is set.
If the cache has been cleared by some other means in the meantime, then
we'll still fail to run the hooks when we should. Fixing that in the
general case is a larger problem, and we'd need to devote significant
engineering effort to it.
There's a chance that we'll run the relevant hooks *multiple times*
with this change, if there's a race between the branch being created,
and the `PostReceive` worker being run multiple times, but this can
already happen, since Sidekiq is "at-least-once" execution of jobs. So,
this should be safe.
Commit messages are not processed for references to issues when
creating the default branch on push. This was expected
behavior (probably to avoid performance problems when first pushing a
repository with thousands of commits). However, this is not an issue
because we always limit the number of commits to process to 100
regardless of whether we are creating the default branch or not.
