Bob Van Landuyt
705c15d7af
Allow enabling the circuitbreaker using an env variable
...
That way we can enable the circuitbreaker for just one host at a time.
2017-10-23 13:53:49 +03:00
Bob Van Landuyt
591ee4e361
Perform the stat check multiple times when checking a storage
...
Instead of only checking once within a timeout, check multiple times
within a timeout.
That means with a timeout of 30 seconds and 3 retries. Each try would
be allowed 20 seconds.
2017-10-23 13:53:49 +03:00
Bob Van Landuyt
430e767139
Implement backoff for the circuitbreaker
...
The circuitbreaker now has 2 failure modes:
- Backing off: This will raise the `Gitlab::Git::Storage::Failing`
exception. Access to the shard is blocked temporarily.
- Circuit broken: This will raise the
`Gitlab::Git::Storage::CircuitBroken` exception. Access to the shard
will be blocked until the failures are reset.
2017-10-23 12:02:23 +03:00
Rémy Coutable
89f5ede2a3
Merge branch 'bvl-do-not-use-redis-keys' into 'master'
...
Avoid using `Redis#keys`
See merge request gitlab-org/gitlab-ce!14889
2017-10-17 16:15:44 +00:00
Bob Van Landuyt
c365dea887
Don't use Redis#keys in the circuitbreaker
2017-10-17 12:47:20 +02:00
Bob Van Landuyt
619021fd7a
Read circuitbreaker settings from Gitlab::CurrentSettings
...
Instead of from the configuration file
2017-10-17 11:50:32 +02:00
Nick Thomas
ba0ebbb510
Allow the git circuit breaker to correctly handle missing repository storages
2017-09-22 10:05:28 +01:00
Robert Speicher
a6fbd8648e
Stub ForkedStorageCheck.storage_available? by default in all specs
...
Add `:broken_storage` metadata to examples to disable this behavior only
when necessary.
2017-08-22 13:06:25 -04:00
Bob Van Landuyt
ee603a0089
Allow a failure_wait_time of 0 for storage access
...
This allows testing every storage attempt after a failure. Which could
be useful for tests
2017-08-16 14:15:50 +02:00
Rémy Coutable
1b413bd62f
Enable Timecop safe mode
...
Signed-off-by: Rémy Coutable <remy@rymai.me>
2017-08-11 14:53:17 +02:00
Robert Speicher
c8b802471b
Enable the RSpec/HookArgument cop and auto-correct offenses
2017-08-10 19:29:42 -04:00
Bob Van Landuyt
4b34720c0c
Use ruby's File.stat to check storage availability
2017-08-04 15:38:50 +02:00
Bob Van Landuyt
0dd4c306ca
Only track accessibility once
2017-08-04 15:38:50 +02:00
Bob Van Landuyt
022c38e63e
Use keys instead of scan_each
2017-08-04 15:38:49 +02:00
Bob Van Landuyt
3899d07f9e
Move hostname to Gitlab::Environment
2017-08-04 15:38:49 +02:00
Bob Van Landuyt
0fa94a0ddd
Make sure the check works for paths with spaces.
2017-08-04 15:38:49 +02:00
Bob Van Landuyt
3598e60bf2
Add a Circuitbreaker for storage paths
2017-08-04 15:38:48 +02:00
