kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity
gitlab-org--gitlab-foss/.gitlab/ci/memory.gitlab-ci.yml

56 lines
2.1 KiB
YAML
Raw Normal View History

Run static benchmarks from 'derailed_benchmarks' Two static memory benchmarks will be included in our CI pipeline. It will load gems from the Gemfile and check the amount of RAM consumed as well as the number of objects allocated and retained. Aggregated values will be included as 'metrics' into MRs while full reports will be downloadable as job artifacts.
2019-06-13 11:14:06 -04:00
memory-static:
Reduce complexity of CI files Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-08-26 16:41:55 -04:00
extends:
- .default-tags
- .default-retry
- .default-cache
- .default-before_script
- .except-docs
variables:
SETUP_DB: "false"
Run static benchmarks from 'derailed_benchmarks' Two static memory benchmarks will be included in our CI pipeline. It will load gems from the Gemfile and check the amount of RAM consumed as well as the number of objects allocated and retained. Aggregated values will be included as 'metrics' into MRs while full reports will be downloadable as job artifacts.
2019-06-13 11:14:06 -04:00
script:
# Uses two different reports from the 'derailed_benchmars' gem.
# Loads each of gems in the Gemfile and checks how much memory they consume when they are required.
# 'derailed_benchmarks' internally uses 'get_process_mem'
Generate gem size metrics in memory CI This includes several changes: - Rename memory-static to generate-gems-size-metrics-static - Rename memory-static-objects to generate-gems-memory-metrics-static - Change generate-gems-size-metrics-static interface. The script now expect `bundle exec derailed bundle:mem` output as its input. The script output to stdout, or stderr for error message. - Change generate-gems-memory-metrics-static interface. The script now expect `bundle exec derailed bundle:objects` output as its input. The script output to stdout, or stderr for error message. - Generate gem size metrics. Script generate-gems-size-metrics-static extract each gem size from `bundle exec derailed bundle:mem` output. Save output to metrics file in format: 'gem_size_mb{name="zip"} 0.5'
2019-06-18 20:52:34 -04:00
- bundle exec derailed bundle:mem > tmp/memory_bundle_mem.txt
- scripts/generate-gems-size-metrics-static tmp/memory_bundle_mem.txt >> 'tmp/memory_metrics.txt'
Run static benchmarks from 'derailed_benchmarks' Two static memory benchmarks will be included in our CI pipeline. It will load gems from the Gemfile and check the amount of RAM consumed as well as the number of objects allocated and retained. Aggregated values will be included as 'metrics' into MRs while full reports will be downloadable as job artifacts.
2019-06-13 11:14:06 -04:00
# Outputs detailed information about objects created while gems are loaded.
# 'derailed_benchmarks' internally uses 'memory_profiler'
Generate gem size metrics in memory CI This includes several changes: - Rename memory-static to generate-gems-size-metrics-static - Rename memory-static-objects to generate-gems-memory-metrics-static - Change generate-gems-size-metrics-static interface. The script now expect `bundle exec derailed bundle:mem` output as its input. The script output to stdout, or stderr for error message. - Change generate-gems-memory-metrics-static interface. The script now expect `bundle exec derailed bundle:objects` output as its input. The script output to stdout, or stderr for error message. - Generate gem size metrics. Script generate-gems-size-metrics-static extract each gem size from `bundle exec derailed bundle:mem` output. Save output to metrics file in format: 'gem_size_mb{name="zip"} 0.5'
2019-06-18 20:52:34 -04:00
- bundle exec derailed bundle:objects > tmp/memory_bundle_objects.txt
- scripts/generate-gems-memory-metrics-static tmp/memory_bundle_objects.txt >> 'tmp/memory_metrics.txt'
Run static benchmarks from 'derailed_benchmarks' Two static memory benchmarks will be included in our CI pipeline. It will load gems from the Gemfile and check the amount of RAM consumed as well as the number of objects allocated and retained. Aggregated values will be included as 'metrics' into MRs while full reports will be downloadable as job artifacts.
2019-06-13 11:14:06 -04:00
artifacts:
paths:
Generate gem size metrics in memory CI This includes several changes: - Rename memory-static to generate-gems-size-metrics-static - Rename memory-static-objects to generate-gems-memory-metrics-static - Change generate-gems-size-metrics-static interface. The script now expect `bundle exec derailed bundle:mem` output as its input. The script output to stdout, or stderr for error message. - Change generate-gems-memory-metrics-static interface. The script now expect `bundle exec derailed bundle:objects` output as its input. The script output to stdout, or stderr for error message. - Generate gem size metrics. Script generate-gems-size-metrics-static extract each gem size from `bundle exec derailed bundle:mem` output. Save output to metrics file in format: 'gem_size_mb{name="zip"} 0.5'
2019-06-18 20:52:34 -04:00
- tmp/memory_*.txt
Run static benchmarks from 'derailed_benchmarks' Two static memory benchmarks will be included in our CI pipeline. It will load gems from the Gemfile and check the amount of RAM consumed as well as the number of objects allocated and retained. Aggregated values will be included as 'metrics' into MRs while full reports will be downloadable as job artifacts.
2019-06-13 11:14:06 -04:00
reports:
Generate gem size metrics in memory CI This includes several changes: - Rename memory-static to generate-gems-size-metrics-static - Rename memory-static-objects to generate-gems-memory-metrics-static - Change generate-gems-size-metrics-static interface. The script now expect `bundle exec derailed bundle:mem` output as its input. The script output to stdout, or stderr for error message. - Change generate-gems-memory-metrics-static interface. The script now expect `bundle exec derailed bundle:objects` output as its input. The script output to stdout, or stderr for error message. - Generate gem size metrics. Script generate-gems-size-metrics-static extract each gem size from `bundle exec derailed bundle:mem` output. Save output to metrics file in format: 'gem_size_mb{name="zip"} 0.5'
2019-06-18 20:52:34 -04:00
metrics: tmp/memory_metrics.txt
Get memory data after booting app in production Preload the app in `production` env, hit it with a single request, and gather total gem memory consumption data using `derailed exec perf:mem` from `derailed_benchmarks`. Present the result as MR metrics.
2019-06-25 08:11:06 -04:00
# Show memory usage caused by invoking require per gem.
# Unlike `memory-static`, it hits the app with one request to ensure that any last minute require-s have been called.
# The application is booted in `production` environment.
# All tests are run without a webserver (directly using Rack::Mock by default).
memory-on-boot:
Reduce complexity of CI files Signed-off-by: Rémy Coutable <remy@rymai.me>
2019-08-26 16:41:55 -04:00
extends:
- .default-tags
- .default-retry
- .default-cache
- .default-before_script
- .use-pg-10
- .except-docs-qa
Get memory data after booting app in production Preload the app in `production` env, hit it with a single request, and gather total gem memory consumption data using `derailed exec perf:mem` from `derailed_benchmarks`. Present the result as MR metrics.
2019-06-25 08:11:06 -04:00
variables:
NODE_ENV: "production"
RAILS_ENV: "production"
SETUP_DB: "true"
SKIP_STORAGE_VALIDATION: "true"
# we override the max_old_space_size to prevent OOM errors
NODE_OPTIONS: --max_old_space_size=3584
script:
# Both bootsnap and derailed monkey-patch Kernel#require, which leads to circular dependency
Make Bootsnap available via ENABLE_BOOTSNAP=1 Bootsnap speeds up Rails loading and now ships by default with Rails 5.2 apps. We should promote this to a default gem and test it out in production. This will also make it possible for the Helm Charts to take advantage of this. It appears that Bootsnap with Rails 5.2.3 and all the GitLab CE gems loads fine on an ARM platform now, so it's possible that https://gitlab.com/gitlab-org/gitlab-ce/issues/34799 has been resolved. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/45230
2019-07-19 16:00:46 -04:00
- ENABLE_BOOTSNAP=false PATH_TO_HIT="/users/sign_in" CUT_OFF=0.3 bundle exec derailed exec perf:mem >> 'tmp/memory_on_boot.txt'
Get memory data after booting app in production Preload the app in `production` env, hit it with a single request, and gather total gem memory consumption data using `derailed exec perf:mem` from `derailed_benchmarks`. Present the result as MR metrics.
2019-06-25 08:11:06 -04:00
- scripts/generate-memory-metrics-on-boot tmp/memory_on_boot.txt >> 'tmp/memory_on_boot_metrics.txt'
artifacts:
paths:
- tmp/memory_*.txt
reports:
metrics: tmp/memory_on_boot_metrics.txt
Copy permalink