kotovalexarian-likes-gitlab/gitlab-org--gitlab-foss
1
0
Fork 0
Code Releases Activity
gitlab-org--gitlab-foss/app/workers/namespaces/schedule_aggregation_worker.rb

44 lines
1.3 KiB
Ruby
Raw Normal View History

Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
# frozen_string_literal: true
module Namespaces
Add latest changes from gitlab-org/gitlab@master
2020-03-31 17:08:05 -04:00
class ScheduleAggregationWorker
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
include ApplicationWorker
Add latest changes from gitlab-org/gitlab@master
2021-07-21 08:09:35 -04:00
data_consistency :always
Add latest changes from gitlab-org/gitlab@master
2021-04-30 14:10:09 -04:00
sidekiq_options retry: 3
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
queue_namespace :update_namespace_statistics
Add latest changes from gitlab-org/gitlab@master
2019-10-18 07:11:44 -04:00
feature_category :source_code_management
Add latest changes from gitlab-org/gitlab@master
2020-03-31 17:08:05 -04:00
idempotent!
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
def perform(namespace_id)
return unless aggregation_schedules_table_exists?
namespace = Namespace.find(namespace_id)
root_ancestor = namespace.root_ancestor
Removes update_statistics_namespace feature flag After measuring the impact of the namespace storage on https://gitlab.com/gitlab-org/gitlab-ce/issues/64092. It was decided that it's performant enough. So we can freely remove the feature flag Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/64092
2019-08-01 16:14:21 -04:00
return if root_ancestor.aggregation_scheduled?
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
Namespace::AggregationSchedule.safe_find_or_create_by!(namespace_id: root_ancestor.id)
Add latest changes from gitlab-org/gitlab@master
2020-03-09 17:07:57 -04:00
rescue ActiveRecord::RecordNotFound => ex
Gitlab::ErrorTracking.track_exception(ex, namespace_id: namespace_id)
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
end
private
# On db/post_migrate/20180529152628_schedule_to_archive_legacy_traces.rb
# traces are archived through build.trace.archive, which in consequence
# calls UpdateProjectStatistics#schedule_namespace_statistics_worker.
#
# The migration and specs fails since NamespaceAggregationSchedule table
# does not exist at that point.
Add latest changes from gitlab-org/gitlab@master
2019-09-18 10:02:45 -04:00
# https://gitlab.com/gitlab-org/gitlab-foss/issues/50712
Includes logic to persist namespace statistics - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
2019-07-02 10:44:39 -04:00
def aggregation_schedules_table_exists?
return true unless Rails.env.test?
Namespace::AggregationSchedule.table_exists?
end
end
end
Copy permalink