kotovalexarian-likes-gitlab
/
gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity

Reorganize feature flag documentation 

Split the FF docs in 3 separate docs to cover 3 different use-cases.
This commit is contained in:
Marin Jankovski 2019-06-27 17:52:02 +00:00 committed by Marcia Ramos
parent 01c917e358
commit fda3149dda
9 changed files with 425 additions and 393 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
PROCESS.md
View File

@ -84,44 +84,29 @@ star, smile, etc.). Some good tips about code reviews can be found in our
[Code Review Guidelines]: https://docs.gitlab.com/ce/development/code_review.html
## Feature flags
Overview and details of feature flag processes in development of GitLab itself is described in [feature flags process documentation](https://docs.gitlab.com/ee/development/feature_flags/process.html).
Guides on how to include feature flags in your backend/frontend code while developing GitLab are described in [developing with feature flags documentation](https://docs.gitlab.com/ee/development/feature_flags/developing.html).
Getting access and how to expose the feature to users is detailed in [controlling feature flags documentation](https://docs.gitlab.com/ee/development/feature_flags/controls.html).
## Feature proposals from the 22nd to the 1st
To allow the Product and Engineering teams time to discuss issues that will be placed into an upcoming milestone,
Product Managers must have their proposal for that milestone ready by the 22nd of each month.
This proposal will be shared with Engineering for discussion, feedback, and planning.
The plan for the upcoming milestone must be finalized by the 1st of the month, one week before kickoff on the 8th.
## Feature freeze on the 7th for the release on the 22nd
The feature freeze on the 7th has been discontinued. The [transition period overview](https://gitlab.com/gitlab-org/release/docs/blob/21cbd409dd5f157fe252f254f3e897f01908abe2/general/deploy/auto-deploy-transition.md#transition)
The feature freeze on the 7th has been discontinued. [Transition period overview](https://gitlab.com/gitlab-org/release/docs/blob/21cbd409dd5f157fe252f254f3e897f01908abe2/general/deploy/auto-deploy-transition.md#transition)
describes the change to this process. During the transition period, the only guarantee that
a change will be included in the release on the 22nd is if the change has been
deployed to GitLab.com prior to this date.
### Feature flags
Merge requests that make changes hidden behind a feature flag, or remove an
existing feature flag because a feature is deemed stable, may be merged (and
picked into the stable branches) up to the 19th of the month. Such merge
requests should have the ~"feature flag" label assigned, and don't require a
corresponding exception request to be created.
A level of common sense should be applied when deciding whether to have a feature
behind a feature flag off or on by default.
The following guidelines can be applied to help make this decision:
* If the feature is not fully ready or functioning, the feature flag should be disabled by default.
* If the feature is ready but there are concerns about performance or impact, the feature flag should be enabled by default, but
disabled via chatops before deployment on GitLab.com environments. If the performance concern is confirmed, the final release should have the feature flag disabled by default.
* In most other cases, the feature flag can be enabled by default.
For more information on rolling out changes using feature flags, read [through the documentation](https://docs.gitlab.com/ee/development/rolling_out_changes_using_feature_flags.html).
In order to build the final package and present the feature for self-hosted
customers, the feature flag should be removed. This should happen before the
22nd, ideally _at least_ 2 days before. That means MRs with feature
flags being picked at the 19th would have quite a tight schedule, so picking
these _earlier_ is preferable.
While rare, release managers may decide to reject picking a change into a stable
branch, even when feature flags are used. This might be necessary if the changes
are deemed problematic, too invasive, or there simply isn't enough time to
properly test how the changes behave on GitLab.com.
### Between the 1st and the 7th
These types of merge requests for the upcoming release need special consideration:

9
doc/development/chatops_on_gitlabcom.md
View File

@ -1,12 +1,13 @@
# Chatops on GitLab.com
Chatops on GitLab.com allows GitLabbers to run various automation tasks on GitLab.com using Slack.
ChatOps on GitLab.com allows GitLab team members to run various automation tasks on GitLab.com using Slack.
## Requesting access
GitLabbers may need access to Chatops on GitLab.com for administration tasks such as:
GitLab team-members may need access to Chatops on GitLab.com for administration
tasks such as:
- Configuring feature flags on staging.
- Configuring feature flags.
- Running `EXPLAIN` queries against the GitLab.com production replica.
To request access to Chatops on GitLab.com:
@ -18,4 +19,4 @@ To request access to Chatops on GitLab.com:
- [Chatops Usage](https://docs.gitlab.com/ee/ci/chatops/README.html)
- [Understanding EXPLAIN plans](understanding_explain_plans.md)
- [Feature Groups](feature_flags.md#feature-groups)
- [Feature Groups](feature_flags/development.md#feature-groups)

128
doc/development/feature_flags.md
View File

@ -1,127 +1 @@
# Manage feature flags
Starting from GitLab 9.3 we support feature flags for features in GitLab via
[Flipper](https://github.com/jnunemaker/flipper/). You should use the `Feature`
class (defined in `lib/feature.rb`) in your code to get, set and list feature
flags.
During runtime you can set the values for the gates via the
[features API](../api/features.md) (accessible to admins only).
## Feature groups
Starting from GitLab 9.4 we support feature groups via
[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group).
Feature groups must be defined statically in `lib/feature.rb` (in the
`.register_feature_groups` method), but their implementation can obviously be
dynamic (querying the DB etc.).
Once defined in `lib/feature.rb`, you will be able to activate a
feature for a given feature group via the [`feature_group` param of the features API](../api/features.md#set-or-create-a-feature)
For GitLab.com, [team members have access to feature flags through Chatops](chatops_on_gitlabcom.md). Only
percentage gates are supported at this time. Setting a feature to be used 50% of
the time, you should execute `/chatops run feature set my_feature_flag 50`.
## Feature flags for user applications
This document only covers feature flags used in the development of GitLab
itself. Feature flags in deployed user applications can be found at
[Feature Flags](../user/project/operations/feature_flags.md)
## Developing with feature flags
In general, it's better to have a group- or user-based gate, and you should prefer
it over the use of percentage gates. This would make debugging easier, as you
filter for example logs and errors based on actors too. Furthermore, this allows
for enabling for the `gitlab-org` group first, while the rest of the users
aren't impacted.
```ruby
# Good
Feature.enabled?(:feature_flag, project)
# Avoid, if possible
Feature.enabled?(:feature_flag)
```
To use feature gates based on actors, the model needs to respond to
`flipper_id`. For example, to enable for the Foo model:
```ruby
class Foo < ActiveRecord::Base
include FeatureGate
end
```
Features that are developed and are intended to be merged behind a feature flag
should not include a changelog entry. The entry should be added in the merge
request removing the feature flags.
In the rare case that you need the feature flag to be on automatically, use
`default_enabled: true` when checking:
```ruby
Feature.enabled?(:feature_flag, project, default_enabled: true)
```
For more information about rolling out changes using feature flags, refer to the
[Rolling out changes using feature flags](rolling_out_changes_using_feature_flags.md)
guide.
### Frontend
For frontend code you can use the method `push_frontend_feature_flag`, which is
available to all controllers that inherit from `ApplicationController`. Using
this method you can expose the state of a feature flag as follows:
```ruby
before_action do
push_frontend_feature_flag(:vim_bindings)
end
def index
# ...
end
def edit
# ...
end
```
You can then check for the state of the feature flag in JavaScript as follows:
```javascript
if ( gon.features.vimBindings ) {
// ...
}
```
The name of the feature flag in JavaScript will always be camelCased, meaning
that checking for `gon.features.vim_bindings` would not work.
### Specs
In the test environment `Feature.enabled?` is stubbed to always respond to `true`,
so we make sure behavior under feature flag doesn't go untested in some non-specific
contexts.
Whenever a feature flag is present, make sure to test _both_ states of the
feature flag.
See the
[testing guide](testing_guide/best_practices.md#feature-flags-in-tests)
for information and examples on how to stub feature flags in tests.
## Enabling a feature flag (in development)
In the rails console (`rails c`), enter the following command to enable your feature flag
```ruby
Feature.enable(:feature_flag_name)
```
## Enabling a feature flag (in production)
Check how to [roll out changes using feature flags](rolling_out_changes_using_feature_flags.md).
This document was moved to [another location](feature_flags/index.md).

123
doc/development/feature_flags/controls.md Normal file
View File

@ -0,0 +1,123 @@
# Access for enabling a feature flag in production
In order to be able to turn on/off features behind feature flags in any of the
GitLab Inc. provided environments such as staging and production, you need to
have access to the chatops bot. Chatops bot is currently running on the ops instance,
which is different from GitLab.com or dev.gitlab.org.
Follow the Chatops document to [request access](https://docs.gitlab.com/ee/development/chatops_on_gitlabcom.html#requesting-access).
Once you are added to the project test if your access propagated,
run:
```
/chatops run feature --help
```
## Rolling out changes
When the changes are deployed to the environments it is time to start
rolling out the feature to our users. The exact procedure of rolling out a
change is unspecified, as this can vary from change to change. However, in
general we recommend rolling out changes incrementally, instead of enabling them
for everybody right away. We also recommend you to _not_ enable a feature
_before_ the code is being deployed.
This allows you to separate rolling out a feature from a deploy, making it
easier to measure the impact of both separately.
GitLab's feature library (using
[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature
Flags process](process.md) guide) supports rolling out changes to a percentage of
users. This in turn can be controlled using [GitLab chatops](../../ci/chatops/README.md).
For an up to date list of feature flag commands please see [the source
code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb).
Note that all the examples in that file must be preceded by
`/chatops run`.
If you get an error "Whoops! This action is not allowed. This incident
will be reported." that means your Slack account is not allowed to
change feature flags or you do not [have access](#access-for-enabling-a-feature-flag-in-production).
### Enabling feature for staging and dev.gitlab.org
As a first step in a feature rollout, you should enable the feature on <https://staging.gitlab.com>
and <https://dev.gitlab.org>.
For example, to enable a feature for 25% of all users, run the following in
Slack:
```
/chatops run feature set new_navigation_bar 25 --dev
/chatops run feature set new_navigation_bar 25 --staging
```
These two environments have different scopes.
`dev.gitlab.org` is a production CE environment that has internal GitLab Inc.
traffic and is used for some development and other related work.
`staging.gitlab.com` has a smaller subset of GitLab.com database and repositories
and does not have regular traffic. Staging is an EE instance and can give you
a (very) rough estimate of how your feature will look/behave on GitLab.com.
Both of these instances are connected to Sentry so make sure you check the projects
there for any exceptions while testing your feature after enabling the feature flag.
Once you are confident enough that these environments are in a good state with your
feature enabled, you can roll out the change to GitLab.com.
## Enabling feature for GitLab.com
Similar to above, to enable a feature for 25% of all users, run the following in
Slack:
```
/chatops run feature set new_navigation_bar 25
```
This will enable the feature for GitLab.com, with `new_navigation_bar` being the
name of the feature.
If you are not certain what percentages to use, simply use the following steps:
1. 25%
1. 50%
1. 75%
1. 100%
Between every step you'll want to wait a little while and monitor the
appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait
may differ. For some features a few minutes is enough, while for others you may
want to wait several hours or even days. This is entirely up to you, just make
sure it is clearly communicated to your team, and the Production team if you
anticipate any potential problems.
Feature gates can also be actor based, for example a feature could first be
enabled for only the `gitlab-ce` project. The project is passed by supplying a
`--project` flag:
```
/chatops run feature set --project=gitlab-org/gitlab-ce some_feature true
```
For groups the `--group` flag is available:
```
/chatops run feature set --group=gitlab-org some_feature true
```
## Cleaning up
Once the change is deemed stable, submit a new merge request to remove the
feature flag. This ensures the change is available to all users and self-hosted
instances. Make sure to add the ~"feature flag" label to this merge request so
release managers are aware the changes are hidden behind a feature flag. If the
merge request has to be picked into a stable branch, make sure to also add the
appropriate "Pick into X" label (e.g. "Pick into XX.X").
See [the process document](https://docs.gitlab.com/ee/development/feature_flags/process.html#including-a-feature-behind-feature-flag-in-the-final-release) for further details.
When a feature gate has been removed from the code base, the value still exists
in the database.
This can be removed through ChatOps:
```
/chatops run feature delete some_feature
```

131
doc/development/feature_flags/development.md Normal file
View File

@ -0,0 +1,131 @@
# Developing with feature flags
In general, it's better to have a group- or user-based gate, and you should prefer
it over the use of percentage gates. This would make debugging easier, as you
filter for example logs and errors based on actors too. Furthermore, this allows
for enabling for the `gitlab-org` or `gitlab-com` group first, while the rest of
the users aren't impacted.
```ruby
# Good
Feature.enabled?(:feature_flag, project)
# Avoid, if possible
Feature.enabled?(:feature_flag)
```
To use feature gates based on actors, the model needs to respond to
`flipper_id`. For example, to enable for the Foo model:
```ruby
class Foo < ActiveRecord::Base
include FeatureGate
end
```
Features that are developed and are intended to be merged behind a feature flag
should not include a changelog entry. The entry should be added in the merge
request removing the feature flags.
In the rare case that you need the feature flag to be on automatically, use
`default_enabled: true` when checking:
```ruby
Feature.enabled?(:feature_flag, project, default_enabled: true)
```
The [`Project#feature_available?`][project-fa],
[`Namespace#feature_available?`][namespace-fa] (EE), and
[`License.feature_available?`][license-fa] (EE) methods all implicitly check for
a feature flag by the same name as the provided argument.
For example if a feature is license-gated, there's no need to add an additional
explicit feature flag check since the flag will be checked as part of the
`License.feature_available?` call. Similarly, there's no need to "clean up" a
feature flag once the feature has reached general availability.
You'd still want to use an explicit `Feature.enabled?` check if your new feature
isn't gated by a License or Plan.
[project-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68
[namespace-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85
[license-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300
An important side-effect of the implicit feature flags mentioned above is that
unless the feature is explicitly disabled or limited to a percentage of users,
the feature flag check will default to `true`.
As an example, if you were to ship the backend half of a feature behind a flag,
you'd want to explicitly disable that flag until the frontend half is also ready
to be shipped. [You can do this via Chatops](https://docs.gitlab.com/ee/development/feature_flags/controls.html):
```
/chatops run feature set some_feature 0
```
Note that you can do this at any time, even before the merge request using the
flag has been merged!
## Feature groups
Starting from GitLab 9.4 we support feature groups via
[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group).
Feature groups must be defined statically in `lib/feature.rb` (in the
`.register_feature_groups` method), but their implementation can obviously be
dynamic (querying the DB etc.).
Once defined in `lib/feature.rb`, you will be able to activate a
feature for a given feature group via the [`feature_group` param of the features API](../../api/features.md#set-or-create-a-feature)
### Frontend
For frontend code you can use the method `push_frontend_feature_flag`, which is
available to all controllers that inherit from `ApplicationController`. Using
this method you can expose the state of a feature flag as follows:
```ruby
before_action do
push_frontend_feature_flag(:vim_bindings)
end
def index
# ...
end
def edit
# ...
end
```
You can then check for the state of the feature flag in JavaScript as follows:
```javascript
if ( gon.features.vimBindings ) {
// ...
}
```
The name of the feature flag in JavaScript will always be camelCased, meaning
that checking for `gon.features.vim_bindings` would not work.
### Specs
In the test environment `Feature.enabled?` is stubbed to always respond to `true`,
so we make sure behavior under feature flag doesn't go untested in some non-specific
contexts.
Whenever a feature flag is present, make sure to test _both_ states of the
feature flag.
See the
[testing guide](../testing_guide/best_practices.md#feature-flags-in-tests)
for information and examples on how to stub feature flags in tests.
### Enabling a feature flag (in development)
In the rails console (`rails c`), enter the following command to enable your feature flag
```ruby
Feature.enable(:feature_flag_name)
```

12
doc/development/feature_flags/index.md Normal file
View File

@ -0,0 +1,12 @@
# Feature flags in development of GitLab
Feature flags can be used to gradually roll out changes, be
it a new feature, or a performance improvement. By using feature flags, we can
comfortably measure the impact of our changes, while still being able to easily
disable those changes, without having to revert an entire release.
Before using feature flags for GitLab's development, read through the following:
- [Process for using features flags](process.md).
- [Developing with feature flags documentation](development.md).
- [Controlling feature flags documentation](controls.md).

130
doc/development/feature_flags/process.md Normal file
View File

@ -0,0 +1,130 @@
# Feature flags process
## Feature flags for user applications
This document only covers feature flags used in the development of GitLab
itself. Feature flags in deployed user applications can be found at
[Feature Flags feature documentation](../../user/project/operations/feature_flags.md).
## Feature flags in GitLab development
The following highlights should be considered when deciding if feature flags
should be leveraged:
- By default, the feature flags should be **off**.
- Feature flags should remain in the codebase for as short period as possible
to reduce the need for feature flag accounting.
- The person operating with feature flags is responsible for clearly communicating
the status of a feature behind the feature flag with responsible stakeholders.
- Merge requests that make changes hidden behind a feature flag, or remove an
existing feature flag because a feature is deemed stable must have the
~"feature flag" label assigned.
One might be tempted to think that feature flags will delay the release of a
feature by at least one month (= one release). This is not the case. A feature
flag does not have to stick around for a specific amount of time
(e.g. at least one release), instead they should stick around until the feature
is deemed stable. Stable means it works on GitLab.com without causing any
problems, such as outages.
### When to use feature flags
Starting with GitLab 11.4, developers are required to use feature flags for
non-trivial changes. Such changes include:
- New features (e.g. a new merge request widget, epics, etc).
- Complex performance improvements that may require additional testing in
production, such as rewriting complex queries.
- Invasive changes to the user interface, such as a new navigation bar or the
removal of a sidebar.
- Adding support for importing projects from a third-party service.
In all cases, those working on the changes can best decide if a feature flag is
necessary. For example, changing the color of a button doesn't need a feature
flag, while changing the navigation bar definitely needs one. In case you are
uncertain if a feature flag is necessary, simply ask about this in the merge
request, and those reviewing the changes will likely provide you with an answer.
When using a feature flag for UI elements, make sure to _also_ use a feature
flag for the underlying backend code, if there is any. This ensures there is
absolutely no way to use the feature until it is enabled.
### Including a feature behind feature flag in the final release
In order to build a final release and present the feature for self-hosted
users, the feature flag should be at least defaulted to **on**. If the feature
is deemed stable and there is confidence that removing the feature flag is safe,
consider removing the feature flag altogether. Take into consideration that such
action can make the feature available on GitLab.com shortly after the change to
the feature flag is merged.
Changing the default state or removing the feature flag has to be done before
the 22nd of the month, _at least_ 2 working days before, in order for the change
to be included in the final self-managed release.
In addition to this, the feature behind feature flag should:
- Run in all GitLab.com environments for a sufficient period of time. This time
period depends on the feature behind the feature flag, but as a general rule of
thumb 2-4 working days should be sufficient to gather enough feedback.
- The feature should be exposed to all users within the GitLab.com plan during
the above mentioned period of time. Exposing the feature to a smaller percentage
or only a group of users might not expose a sufficient amount of information to aid in
making a decision on feature stability.
While rare, release managers may decide to reject picking or revert a change in
a stable branch, even when feature flags are used. This might be necessary if
the changes are deemed problematic, too invasive, or there simply isn't enough
time to properly measure how the changes behave on GitLab.com.
### The cost of feature flags
When reading the above, one might be tempted to think this procedure is going to
add a lot of work. Fortunately, this is not the case, and we'll show why. For
this example we'll specify the cost of the work to do as a number, ranging from
0 to infinity. The greater the number, the more expensive the work is. The cost
does _not_ translate to time, it's just a way of measuring complexity of one
change relative to another.
Let's say we are building a new feature, and we have determined that the cost of
this is 10. We have also determined that the cost of adding a feature flag check
in a variety of places is 1. If we do not use feature flags, and our feature
works as intended, our total cost is 10. This however is the best case scenario.
Optimizing for the best case scenario is guaranteed to lead to trouble, whereas
optimizing for the worst case scenario is almost always better.
To illustrate this, let's say our feature causes an outage, and there's no
immediate way to resolve it. This means we'd have to take the following steps to
resolve the outage:
1. Revert the release.
1. Perform any cleanups that might be necessary, depending on the changes that
were made.
1. Revert the commit, ensuring the "master" branch remains stable. This is
especially necessary if solving the problem can take days or even weeks.
1. Pick the revert commit into the appropriate stable branches, ensuring we
don't block any future releases until the problem is resolved.
As history has shown, these steps are time consuming, complex, often involve
many developers, and worst of all: our users will have a bad experience using
GitLab.com until the problem is resolved.
Now let's say that all of this has an associated cost of 10. This means that in
the worst case scenario, which we should optimize for, our total cost is now 20.
If we had used a feature flag, things would have been very different. We don't
need to revert a release, and because feature flags are disabled by default we
don't need to revert and pick any Git commits. In fact, all we have to do is
disable the feature, and in the worst case, perform cleanup. Let's say that
the cost of this is 2. In this case, our best case cost is 11: 10 to build the
feature, and 1 to add the feature flag. The worst case cost is now 13: 10 to
build the feature, 1 to add the feature flag, and 2 to disable and clean up.
Here we can see that in the best case scenario the work necessary is only a tiny
bit more compared to not using a feature flag. Meanwhile, the process of
reverting our changes has been made significantly and reliably cheaper.
In other words, feature flags do not slow down the development process. Instead,
they speed up the process as managing incidents now becomes _much_ easier. Once
continuous deployments are easier to perform, the time to iterate on a feature
is reduced even further, as you no longer need to wait weeks before your changes
are available on GitLab.com.

10
doc/development/new_fe_guide/tips.md
View File

@ -10,16 +10,16 @@ yarn clean
## Creating feature flags in development
The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags.md#enabling-a-feature-flag-in-development).
The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags/development.md#enabling-a-feature-flag-in-development).
Your feature flag can now be:
- [made available to the frontend](../feature_flags.md#frontend) via the `gon`
- queried in [tests](../feature_flags.md#specs)
- queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method
- [Made available to the frontend](../feature_flags/development.md#frontend) via the `gon`
- Queried in [tests](../feature_flags/development.md#specs)
- Queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method
### More on feature flags
- [Deleting a feature flag](../../api/features.md#delete-a-feature)
- [Manage feature flags](../feature_flags.md)
- [Manage feature flags](../feature_flags/process.md)
- [Feature flags API](../../api/features.md)

226
doc/development/rolling_out_changes_using_feature_flags.md
View File

@ -1,225 +1 @@
# Rolling out changes using feature flags
[Feature flags](feature_flags.md) can be used to gradually roll out changes, be
it a new feature, or a performance improvement. By using feature flags, we can
comfortably measure the impact of our changes, while still being able to easily
disable those changes, without having to revert an entire release.
## When to use feature flags
Starting with GitLab 11.4, developers are required to use feature flags for
non-trivial changes. Such changes include:
- New features (e.g. a new merge request widget, epics, etc).
- Complex performance improvements that may require additional testing in
production, such as rewriting complex queries.
- Invasive changes to the user interface, such as a new navigation bar or the
removal of a sidebar.
- Adding support for importing projects from a third-party service.
In all cases, those working on the changes can best decide if a feature flag is
necessary. For example, changing the color of a button doesn't need a feature
flag, while changing the navigation bar definitely needs one. In case you are
uncertain if a feature flag is necessary, simply ask about this in the merge
request, and those reviewing the changes will likely provide you with an answer.
When using a feature flag for UI elements, make sure to _also_ use a feature
flag for the underlying backend code, if there is any. This ensures there is
absolutely no way to use the feature until it is enabled.
## The cost of feature flags
When reading the above, one might be tempted to think this procedure is going to
add a lot of work. Fortunately, this is not the case, and we'll show why. For
this example we'll specify the cost of the work to do as a number, ranging from
0 to infinity. The greater the number, the more expensive the work is. The cost
does _not_ translate to time, it's just a way of measuring complexity of one
change relative to another.
Let's say we are building a new feature, and we have determined that the cost of
this is 10. We have also determined that the cost of adding a feature flag check
in a variety of places is 1. If we do not use feature flags, and our feature
works as intended, our total cost is 10. This however is the best case scenario.
Optimising for the best case scenario is guaranteed to lead to trouble, whereas
optimising for the worst case scenario is almost always better.
To illustrate this, let's say our feature causes an outage, and there's no
immediate way to resolve it. This means we'd have to take the following steps to
resolve the outage:
1. Revert the release.
1. Perform any cleanups that might be necessary, depending on the changes that
were made.
1. Revert the commit, ensuring the "master" branch remains stable. This is
especially necessary if solving the problem can take days or even weeks.
1. Pick the revert commit into the appropriate stable branches, ensuring we
don't block any future releases until the problem is resolved.
As history has shown, these steps are time consuming, complex, often involve
many developers, and worst of all: our users will have a bad experience using
GitLab.com until the problem is resolved.
Now let's say that all of this has an associated cost of 10. This means that in
the worst case scenario, which we should optimise for, our total cost is now 20.
If we had used a feature flag, things would have been very different. We don't
need to revert a release, and because feature flags are disabled by default we
don't need to revert and pick any Git commits. In fact, all we have to do is
disable the feature, and in the worst case, perform cleanup. Let's say that
the cost of this is 2. In this case, our best case cost is 11: 10 to build the
feature, and 1 to add the feature flag. The worst case cost is now 13: 10 to
build the feature, 1 to add the feature flag, and 2 to disable and clean up.
Here we can see that in the best case scenario the work necessary is only a tiny
bit more compared to not using a feature flag. Meanwhile, the process of
reverting our changes has been made significantly and reliably cheaper.
In other words, feature flags do not slow down the development process. Instead,
they speed up the process as managing incidents now becomes _much_ easier. Once
continuous deployments are easier to perform, the time to iterate on a feature
is reduced even further, as you no longer need to wait weeks before your changes
are available on GitLab.com.
## Rolling out changes
The procedure of using feature flags is straightforward, and similar to not
using them. You add the necessary tests (make sure to test both the on and off
states of your feature flag(s)), make sure they all pass, have the code
reviewed, etc. You then submit your merge request, and add the ~"feature flag"
label. This label is used to signal to release managers that your changes are
hidden behind a feature flag and that it is safe to pick the MR into a stable
branch, without the need for an exception request.
When the changes are deployed it is time to start rolling out the feature to our
users. The exact procedure of rolling out a change is unspecified, as this can
vary from change to change. However, in general we recommend rolling out changes
incrementally, instead of enabling them for everybody right away. We also
recommend you to _not_ enable a feature _before_ the code is being deployed.
This allows you to separate rolling out a feature from a deploy, making it
easier to measure the impact of both separately.
GitLab's feature library (using
[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature
Flags](feature_flags.md) guide) supports rolling out changes to a percentage of
users. This in turn can be controlled using [GitLab
chatops](../ci/chatops/README.md).
For an up to date list of feature flag commands please see [the source
code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb).
Note that all the examples in that file must be preceded by
`/chatops run`.
If you get an error "Whoops! This action is not allowed. This incident
will be reported." that means your Slack account is not allowed to
change feature flags. To test if you are allowed to do anything at all,
run:
```
/chatops run feature --help
```
For example, to enable a feature for 25% of all users, run the following in
Slack:
```
/chatops run feature set new_navigation_bar 25
```
This will enable the feature for GitLab.com, with `new_navigation_bar` being the
name of the feature. We can also enable the feature for <https://dev.gitlab.org>
or <https://staging.gitlab.com>:
```
/chatops run feature set new_navigation_bar 25 --dev
/chatops run feature set new_navigation_bar 25 --staging
```
If you are not certain what percentages to use, simply use the following steps:
1. 25%
1. 50%
1. 75%
1. 100%
Between every step you'll want to wait a little while and monitor the
appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait
may differ. For some features a few minutes is enough, while for others you may
want to wait several hours or even days. This is entirely up to you, just make
sure it is clearly communicated to your team, and the Production team if you
anticipate any potential problems.
Feature gates can also be actor based, for example a feature could first be
enabled for only the `gitlab-ce` project. The project is passed by supplying a
`--project` flag:
```
/chatops run feature set --project=gitlab-org/gitlab-ce some_feature true
```
For groups the `--group` flag is available:
```
/chatops run feature set --group=gitlab-org some_feature true
```
Once a change is deemed stable, submit a new merge request to remove the
feature flag. This ensures the change is available to all users and self-hosted
instances. Make sure to add the ~"feature flag" label to this merge request so
release managers are aware the changes are hidden behind a feature flag. If the
merge request has to be picked into a stable branch (e.g. after the 7th), make
sure to also add the appropriate "Pick into X" label (e.g. "Pick into 11.4").
One might be tempted to think this will delay the release of a feature by at
least one month (= one release). This is not the case. A feature flag does not
have to stick around for a specific amount of time (e.g. at least one release),
instead they should stick around until the feature is deemed stable. Stable
means it works on GitLab.com without causing any problems, such as outages. In
most cases this will translate to a feature (with a feature flag) being shipped
in RC1, followed by the feature flag being removed in RC2. This in turn means
the feature will be stable by the time we publish a stable package around the
22nd of the month.
## Implicit feature flags
The [`Project#feature_available?`][project-fa],
[`Namespace#feature_available?`][namespace-fa] (EE), and
[`License.feature_available?`][license-fa] (EE) methods all implicitly check for
a feature flag by the same name as the provided argument.
For example if a feature is license-gated, there's no need to add an additional
explicit feature flag check since the flag will be checked as part of the
`License.feature_available?` call. Similarly, there's no need to "clean up" a
feature flag once the feature has reached general availability.
You'd still want to use an explicit `Feature.enabled?` check if your new feature
isn't gated by a License or Plan.
[project-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68
[namespace-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85
[license-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300
### Undefined feature flags default to "on"
An important side-effect of the [implicit feature flags](#implicit-feature-flags)
mentioned above is that unless the feature is explicitly disabled or limited to a
percentage of users, the feature flag check will default to `true`.
As an example, if you were to ship the backend half of a feature behind a flag,
you'd want to explicitly disable that flag until the frontend half is also ready
to be shipped. You can do this via ChatOps:
```
/chatops run feature set some_feature 0
```
Note that you can do this at any time, even before the merge request using the
flag has been merged!
### Cleaning up
When a feature gate has been removed from the code base, the value still exists
in the database. This can be removed through ChatOps:
```
/chatops run feature delete some_feature
```
This document was moved to [another location](feature_flags/index.md).