Clean up stale Prometheus metrics in before_exec unicorn hook

Issuing a SIGUSR2 to a unicorn involves a dance where new workers are
spun up before the old ones are shut down. To ensure Prometheus
.db files aren't deleted after the new workers are shut down, we
need to clean these files up before the unicorn process is run.

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2582
This commit is contained in:
Stan Hu 2018-06-11 21:45:11 -07:00
parent 26c9d71666
commit e9d4b08dc2

View file

@ -81,6 +81,17 @@ GC.respond_to?(:copy_on_write_friendly=) and
# fast LAN.
check_client_connection false
before_exec do |server|
# The following is necessary to ensure stale Prometheus metrics don't
# accumulate over time. It needs to be done in this hook as opposed to
# inside an init script to ensure metrics files aren't deleted after new
# unicorn workers start after a SIGUSR2 is received.
if ENV['prometheus_multiproc_dir']
old_metrics = Dir[File.join(ENV['prometheus_multiproc_dir'], '*.db')]
FileUtils.rm_rf(old_metrics)
end
end
before_fork do |server, worker|
# the following is highly recommended for Rails + "preload_app true"
# as there's no need for the master process to hold a connection