Update automatic merging documentation
We're moving from merging + reverting to only merging with the `ours` strategy, removing the need for merge requests or reverting conflicts. For more details, refer to Merge Train merge request https://gitlab.com/gitlab-org/merge-train/merge_requests/6.
This commit is contained in:
Yorick Peterse
parent
955b4b9264
commit
8d99636eff
1 changed files with 171 additions and 77 deletions
|
|
@ -1,33 +1,27 @@
|
|||
# Automatic CE->EE merge
|
||||
|
||||
Whenever a commit is pushed to the CE `master` branch, it is automatically
|
||||
merged into the EE `master` branch. If the commit produces any conflicts, it is
|
||||
instead reverted from CE `master`. When this happens, a merge request will be
|
||||
set up automatically that can be used to reinstate the changes. This merge
|
||||
request will be assigned to the author of the conflicting commit, or the merge
|
||||
request author if the commit author could not be associated with a GitLab user.
|
||||
If no author could be found, the merge request is assigned to a random member of
|
||||
the Delivery team. It is then up to this team member to figure out who to assign
|
||||
the merge request to.
|
||||
Commits pushed to CE `master` are automatically merged into EE `master` roughly
|
||||
every 5 minutes. Changes are merged using the `ours` merge strategy in the
|
||||
context of EE. This means that any merge conflicts are resolved by taking the EE
|
||||
changes and discarding the CE changes. This removes the need for resolving
|
||||
conflicts or reverting changes, at the cost of **absolutely requiring** EE merge
|
||||
requests to be created whenever a CE merge request causes merge conflicts.
|
||||
Failing to do so can result in changes not making their way into EE.
|
||||
|
||||
Because some commits can not be reverted if new commits depend on them, we also
|
||||
run a job periodically that processes a range of commits and tries to merge or
|
||||
revert them. This should ensure that all commits are either merged into EE
|
||||
`master`, or reverted, instead of just being left behind in CE.
|
||||
## Always create an EE merge request if there are conflicts
|
||||
|
||||
In CI there is a job called `ee_compat_check`, which checks if a CE MR causes
|
||||
merge conflicts with EE. If this job reports conflicts, you **must** create an
|
||||
EE merge request. If you are an external contributor you can ask the reviewer to
|
||||
do this for you.
|
||||
|
||||
## Always merge EE merge requests before their CE counterparts
|
||||
|
||||
**In order to avoid conflicts in the CE->EE merge, you should always merge the
|
||||
EE version of your CE merge request first, if present.**
|
||||
|
||||
The rationale for this is that as CE->EE merges are done automatically, it can
|
||||
happen that:
|
||||
|
||||
1. A CE merge request that needs EE-specific changes is merged.
|
||||
1. The automatic CE->EE merge happens.
|
||||
1. Conflicts due to the CE merge request occur since its EE merge request isn't
|
||||
merged yet.
|
||||
1. The CE changes are reverted.
|
||||
Failing to do so will lead to CE changes being discarded when merging into EE,
|
||||
if they cause merge conflicts.
|
||||
|
||||
## Avoiding CE->EE merge conflicts beforehand
|
||||
|
||||
|
|
@ -45,76 +39,184 @@ detect if the current branch's changes will conflict during the CE->EE merge.
|
|||
The job reports what files are conflicting and how to set up a merge request
|
||||
against EE.
|
||||
|
||||
## How to reinstate changes
|
||||
#### How the job works
|
||||
|
||||
When a commit is reverted, the corresponding merge request to reinstate the
|
||||
changes will include all the details necessary to ensure the changes make it
|
||||
back into CE and EE. However, you still need to manually set up an EE merge
|
||||
request that resolves the conflicts.
|
||||
1. Generates the diff between your branch and current CE `master`
|
||||
1. Tries to apply it to current EE `master`
|
||||
1. If it applies cleanly, the job succeeds, otherwise...
|
||||
1. Detects a branch with the `ee-` prefix or `-ee` suffix in EE
|
||||
1. If it exists, generate the diff between this branch and current EE `master`
|
||||
1. Tries to apply it to current EE `master`
|
||||
1. If it applies cleanly, the job succeeds
|
||||
|
||||
Each merge request used to reinstate changes will have the "reverted" label
|
||||
applied. Please do not remove this label, as it will be used to determine how
|
||||
many times commits are reverted and how long it takes to reinstate the changes.
|
||||
In the case where the job fails, it means you should create an `ee-<ce_branch>`
|
||||
or `<ce_branch>-ee` branch, push it to EE and open a merge request against EE
|
||||
`master`.
|
||||
At this point if you retry the failing job in your CE merge request, it should
|
||||
now pass.
|
||||
|
||||
An example merge request can be found in [CE merge request
|
||||
23280](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23280).
|
||||
Notes:
|
||||
|
||||
## How it works
|
||||
- This task is not a silver-bullet, its current goal is to bring awareness to
|
||||
developers that their work needs to be ported to EE.
|
||||
- Community contributors shouldn't be required to submit merge requests against
|
||||
EE, but reviewers should take actions by either creating such EE merge request
|
||||
or asking a GitLab developer to do it **before the merge request is merged**.
|
||||
- If you branch is too far behind `master`, the job will fail. In that case you
|
||||
should rebase your branch upon latest `master`.
|
||||
- Code reviews for merge requests often consist of multiple iterations of
|
||||
feedback and fixes. There is no need to update your EE MR after each
|
||||
iteration. Instead, create an EE MR as soon as you see the
|
||||
`ee_compat_check` job failing. After you receive the final approval
|
||||
from a Maintainer (but **before the CE MR is merged**) update the EE MR.
|
||||
This helps to identify significant conflicts sooner, but also reduces the
|
||||
number of times you have to resolve conflicts.
|
||||
- Please remember to
|
||||
[always have your EE merge request merged before the CE version](#always-merge-ee-merge-requests-before-their-ce-counterparts).
|
||||
- You can use [`git rerere`](https://git-scm.com/docs/git-rerere)
|
||||
to avoid resolving the same conflicts multiple times.
|
||||
|
||||
The automatic merging is performed using a project called [Merge
|
||||
Train](https://gitlab.com/gitlab-org/merge-train/). For every commit to merge or
|
||||
revert, we generate patches using `git format-patch` which we then try to apply
|
||||
using `git am --3way`. If this succeeds we push the changes to EE, if this fails
|
||||
we decide what to do based on the failure reason:
|
||||
### Cherry-picking from CE to EE
|
||||
|
||||
1. If the patch could not be applied because it was already applied, we just
|
||||
skip it.
|
||||
1. If the patch caused conflicts, we revert the source commits.
|
||||
For avoiding merge conflicts, we use a method of creating equivalent branches
|
||||
for CE and EE. If the `ee-compat-check` job fails, this process is required.
|
||||
|
||||
Commits are reverted in reverse order, ensuring that if commit B depends on A,
|
||||
and both conflict, we first revert B followed by reverting A.
|
||||
This method only requires that you have cloned both CE and EE into your computer.
|
||||
If you don't have them yet, please go ahead and clone them:
|
||||
|
||||
- Clone CE repo: `git clone git@gitlab.com:gitlab-org/gitlab-ce.git`
|
||||
- Clone EE repo: `git clone git@gitlab.com:gitlab-org/gitlab-ee.git`
|
||||
|
||||
And the only additional setup we need is to add CE as remote of EE and vice-versa:
|
||||
|
||||
- Open two terminal windows, one in CE, and another one in EE:
|
||||
- In EE: `git remote add ce git@gitlab.com:gitlab-org/gitlab-ce.git`
|
||||
- In CE: `git remote add ee git@gitlab.com:gitlab-org/gitlab-ee.git`
|
||||
|
||||
That's all setup we need, so that we can cherry-pick a commit from CE to EE, and
|
||||
from EE to CE.
|
||||
|
||||
Now, every time you create an MR for CE and EE:
|
||||
|
||||
1. Open two terminal windows, one in CE, and another one in EE
|
||||
1. In the CE terminal:
|
||||
1. Create the CE branch, e.g., `branch-example`
|
||||
1. Make your changes and push a commit (commit A)
|
||||
1. Create the CE merge request in GitLab
|
||||
1. In the EE terminal:
|
||||
1. Create the EE-equivalent branch ending with `-ee`, e.g.,
|
||||
`git checkout -b branch-example-ee`
|
||||
1. Fetch the CE branch: `git fetch ce branch-example`
|
||||
1. Cherry-pick the commit A: `git cherry-pick commit-A-SHA`
|
||||
1. If Git prompts you to fix the conflicts, do a `git status`
|
||||
to check which files contain conflicts, fix them, save the files
|
||||
1. Add the changes with `git add .` but **DO NOT commit** them
|
||||
1. Continue cherry-picking: `git cherry-pick --continue`
|
||||
1. Push to EE: `git push origin branch-example-ee`
|
||||
1. Create the EE-equivalent MR and link to the CE MR from the
|
||||
description "Ports [CE-MR-LINK] to EE"
|
||||
1. Once all the jobs are passing in both CE and EE, you've addressed the
|
||||
feedback from your own team, and got them approved, the merge requests can be merged.
|
||||
1. When both MRs are ready, the EE merge request will be merged first, and the
|
||||
CE-equivalent will be merged next.
|
||||
|
||||
**Important notes:**
|
||||
|
||||
- The commit SHA can be easily found from the GitLab UI. From a merge request,
|
||||
open the tab **Commits** and click the copy icon to copy the commit SHA.
|
||||
- To cherry-pick a **commit range**, such as [A > B > C > D] use:
|
||||
|
||||
```shell
|
||||
git cherry-pick "oldest-commit-SHA^..newest-commit-SHA"
|
||||
```
|
||||
|
||||
For example, suppose the commit A is the oldest, and its SHA is `4f5e4018c09ed797fdf446b3752f82e46f5af502`,
|
||||
and the commit D is the newest, and its SHA is `80e1c9e56783bd57bd7129828ec20b252ebc0538`.
|
||||
The cherry-pick command will be:
|
||||
|
||||
```shell
|
||||
git cherry-pick "4f5e4018c09ed797fdf446b3752f82e46f5af502^..80e1c9e56783bd57bd7129828ec20b252ebc0538"
|
||||
```
|
||||
|
||||
- To cherry-pick a **merge commit**, use the flag `-m 1`. For example, suppose that the
|
||||
merge commit SHA is `138f5e2f20289bb376caffa0303adb0cac859ce1`:
|
||||
|
||||
```shell
|
||||
git cherry-pick -m 1 138f5e2f20289bb376caffa0303adb0cac859ce1
|
||||
```
|
||||
- To cherry-pick multiple commits, such as B and D in a range [A > B > C > D], use:
|
||||
|
||||
```shell
|
||||
git cherry-pick commmit-B-SHA commit-D-SHA
|
||||
```
|
||||
|
||||
For example, suppose commit B SHA = `4f5e4018c09ed797fdf446b3752f82e46f5af502`,
|
||||
and the commit D SHA = `80e1c9e56783bd57bd7129828ec20b252ebc0538`.
|
||||
The cherry-pick command will be:
|
||||
|
||||
```shell
|
||||
git cherry-pick 4f5e4018c09ed797fdf446b3752f82e46f5af502 80e1c9e56783bd57bd7129828ec20b252ebc0538
|
||||
```
|
||||
|
||||
This case is particularly useful when you have a merge commit in a sequence of
|
||||
commits and you want to cherry-pick all but the merge commit.
|
||||
|
||||
- If you push more commits to the CE branch, you can safely repeat the procedure
|
||||
to cherry-pick them to the EE-equivalent branch. You can do that as many times as
|
||||
necessary, using the same CE and EE branches.
|
||||
- If you submitted the merge request to the CE repo and the `ee-compat-check` job passed,
|
||||
you are not required to submit the EE-equivalent MR, but it's still recommended. If the
|
||||
job failed, you are required to submit the EE MR so that you can fix the conflicts in EE
|
||||
before merging your changes into CE.
|
||||
|
||||
---
|
||||
|
||||
[Return to Development documentation](README.md)
|
||||
|
||||
## FAQ
|
||||
|
||||
### Why?
|
||||
### How does automatic merging work?
|
||||
|
||||
We want to work towards being able to deploy continuously, but this requires
|
||||
that `master` is always stable and has all the changes we need. If CE `master`
|
||||
can not be merged into EE `master` due to merge conflicts, this prevents _any_
|
||||
change from CE making its way into EE. Since GitLab.com runs on EE, this
|
||||
effectively prevents us from deploying changes.
|
||||
The automatic merging is performed using a project called [Merge
|
||||
Train](https://gitlab.com/gitlab-org/merge-train/). This project will clone CE
|
||||
and EE master, and merge CE master into EE master using `git merge
|
||||
--strategy=ours`. This process runs roughly every 5 minutes.
|
||||
|
||||
Past experiences and data have shown that periodic CE to EE merge requests do
|
||||
not scale, and often take a very long time to complete. For example, [in this
|
||||
For more information on the exact implementation you can refer to the source
|
||||
code.
|
||||
|
||||
### Why merge automatically?
|
||||
|
||||
As we work towards continuous deployments and a single repository for both CE
|
||||
and EE, we need to first make sure that all CE changes make their way into CE as
|
||||
fast as possible. Past experiences and data have shown that periodic CE to EE
|
||||
merge requests do not scale, and often take a very long time to complete. For
|
||||
example, [in this
|
||||
comment](https://gitlab.com/gitlab-org/release/framework/issues/49#note_114614619)
|
||||
we determined that the average time to close an upstream merge request is around
|
||||
5 hours, with peaks up to several days. Periodic merge requests are also
|
||||
frustrating to work with, because they often include many changes unrelated to
|
||||
your own changes.
|
||||
|
||||
Automatically merging or reverting commits allows us to keep merging changes
|
||||
from CE into EE, as we never have to wait hours for somebody to resolve a set of
|
||||
merge conflicts.
|
||||
To resolve these problems, we now merge changes using the `ours` strategy to
|
||||
automatically resolve merge conflicts. This removes the need for resolving
|
||||
conflicts in a periodic merge request, and allows us to merge changes from CE
|
||||
into EE much faster.
|
||||
|
||||
### Does the CE to EE merge take into account merge commits?
|
||||
### My CE merge request caused conflicts after it was merged. What do I do?
|
||||
|
||||
No. When merging CE changes into EE, merge commits are ignored.
|
||||
If you notice this, you should set up an EE merge request that resolves these
|
||||
conflicts as **soon as possible**. Failing to do so can lead to your changes not
|
||||
being available in EE, which may break tests. This in turn would prevent us from
|
||||
being able to deploy.
|
||||
|
||||
### My changes are reverted, but I set up an EE MR to resolve conflicts
|
||||
### Won't this setup be risky?
|
||||
|
||||
Most likely the automatic merge job ran before the EE merge request was merged.
|
||||
If this keeps happening, consider reporting a bug in the [Merge Train issue
|
||||
tracker](https://gitlab.com/gitlab-org/merge-train/issues).
|
||||
|
||||
### My changes keep getting reverted, and this is really annoying!
|
||||
|
||||
This is understandable, but the solution to this is fairly straightforward:
|
||||
simply set up an EE merge request for every CE merge request, and resolve your
|
||||
conflicts before the changes are reverted.
|
||||
|
||||
### Will we allow certain people to still merge changes, even if they conflict?
|
||||
|
||||
No.
|
||||
No, not if there is an EE merge request for every CE merge request that causes
|
||||
conflicts _and_ that EE merge request is merged first. In the past we may have
|
||||
been a bit more relaxed when it comes to enforcing EE merge requests, but to
|
||||
enable automatic merging have to start requiring such merge requests even for
|
||||
the smallest conflicts.
|
||||
|
||||
### Some files I work with often conflict, how can I best deal with this?
|
||||
|
||||
|
|
@ -123,11 +225,3 @@ so that the EE specific changes are not intertwined with CE code. For Ruby code
|
|||
you can do this by moving the EE code to a separate module, which can then be
|
||||
injected into the appropriate classes or modules. See [Guidelines for
|
||||
implementing Enterprise Edition features](ee_features.md) for more information.
|
||||
|
||||
### Will changelog entries be reverted automatically?
|
||||
|
||||
Only if the changelog was added in the commit that was reverted. If a changelog
|
||||
entry was added in a separate commit, it is possible for it to be left behind.
|
||||
Since changelog entries are related to the changes in question, there is no real
|
||||
reason to commit the changelog separately, and as such this should not be a big
|
||||
problem.
|
||||
|
|
|
|||
Loading…
Reference in a new issue