In https://gitlab.com/gitlab-com/gl-infra/production/issues/928, we saw
a significant amount of network traffic and CPU usage due to Redis
checking feature flags via Flipper. Since these flags are hit with every
request, the overhead becomes significant. To alleviate Redis overhead,
we now cache the data in the following way:
* L1: A thread-local memory store for 1 minute
* L2: Redis for 1 hour
Now that application settings are no longer dominating network traffic,
we see that the Feature#persisted_names is using a significant amount of
CPU and network bandwidth for Redis. Move this cache into the
thread-local memory storage to reduce Redis overhead.
We saw on GitLab.com, the SQL query, `SELECT "features"."key" FROM
"features"` peaked at 2300 times per second.
We can quiet this down a bit by caching it in Redis for a minute.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/63435
Building on support for setting feature flags by project, this adds
support for setting them by GitLab group path.
This is different from setting them by Flipper feature_groups, which
are for batch updating pre-registered collections.
For features the feature gates are sometimes projects, not groups or
users. For example for git object pools:
https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5872
This commit allows for setting feature group gates based on projects, by its
path as that seems most convenient.
Previous code would not work with `disabled?` because that method would
send two parameters (second always `nil`) which we are not mocking.
Instead of mock yet another state, I decide to fix it where it belongs.
The GitHub importer (and probably other parts of our code) ends up
calling Feature.persisted? many times (via Gitaly). By storing this data
in RequestStore we can save ourselves _a lot_ of database queries.
Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/39361
Ash McKenzie