Commit graph

15 commits

Author SHA1 Message Date
Lin Jen-Shin
4b30aec0aa Allow rake cache:clear clearing pipeline status cache
* Use the correct key prefix
* Clear old cache keys

TODO:

At some point we could remove clearing old cache keys.
2018-04-10 20:57:00 +08:00
Paul Charlton
cb3b4a15e6 Support multiple Redis instances based on queue type 2017-07-11 03:35:47 +00:00
Nick Thomas
e9819de10a Remove the ClearDatabaseCacheWorker 2017-04-21 11:52:28 +01:00
Douwe Maan
87c9df29ce Don’t exclude some file in lib from rubocop 2017-02-23 09:32:42 -06:00
Nick Thomas
c0eb2cbbb4 Stop clearing the database cache on rake cache:clear 2016-10-22 12:42:19 +01:00
Nick Thomas
e94cd6fdfe Add markdown cache columns to the database, but don't use them yet
This commit adds a number of _html columns and, with the exception of Note,
starts updating them whenever the content of their partner fields changes.

Note has a collision with the note_html attr_accessor; that will be fixed later

A background worker for clearing these cache columns is also introduced - use
`rake cache:clear` to set it off. You can clear the database or Redis caches
separately by running `rake cache:clear:db` or `rake cache:clear:redis`,
respectively.
2016-10-07 02:54:25 +01:00
Jacob Vosmaer
cf669551f6 Put CACHE_NAMESPACE in the Gitlab::Redis module 2016-04-07 15:42:07 +02:00
Jacob Vosmaer
0163e27631 Add Gitlab::Redis connection pool 2016-04-04 17:00:09 +02:00
Jacob Vosmaer
c46e6eff10 Parse config/resque.yml in one place only 2016-03-09 15:16:46 +01:00
Jacob Vosmaer
a310901280 Batch size >1000 does not pay off
We did a small experiment to see how a full scan of the Redis keys on
gitlab.com speeds up as we increase the batch size. The values on the
right are time in seconds for a full scan (no delete operations).

count: 10);      284.500529021
count: 100);      86.21216934
count: 1_000);    60.931676195
count: 10_000);   60.96355610
count: 100_000);  62.378172541

It looks like 1,000 is a good number.
2016-02-25 13:50:08 +01:00
Jacob Vosmaer
44e4f07037 Improve readability of 'rake cache:clear' code 2016-02-19 14:18:57 +01:00
Jacob Vosmaer
9805fe1914 Use SCAN during 'rake cache:clear'
This allows 'rake cache:clear' to delete millions of keys without
choking. It requires Redis 2.8.0 or newer but we needed that already
anyway.
2016-02-18 16:55:19 +01:00
Dmitriy Zaporozhets
b8f12d1917 Replace GITLAB with GitLab in rake task descriptions 2015-06-23 16:52:40 +02:00
Jacob Vosmaer
cf0a370274 Patch rake cache:clear for large amounts of keys 2014-06-20 13:58:57 +02:00
Dmitriy Zaporozhets
2a6b4f965e rake task to clear redis cache 2013-03-31 23:45:58 +03:00