Merge branch 'fix/use-fewer-queries-for-ci-charts' into 'master'

Use fewer queries for CI charts

## What does this MR do?
It reduces number of queries sent for aggregating counts for CI graphs.

## Are there points in the code the reviewer needs to double check?
N/A

## Why was this MR needed?
For this project (gitlab-ce), loading `/graphs/master/ci` is so slow it times out eventually. I did a quick benchmarking on production and found that it can take 72.5 seconds to only load the controller action variables (there are queries done from the view, didn't look into those). This MR reduces the time to about 2.5 seconds.

Extra improvement could be done by introducing an index on `gl_project_id` and `created_at` for `ci_builds` table, but I can't confirm that right  now.

## What are the relevant issue numbers?
#20262 

## Screenshots (if relevant)
N/A

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- ~~[ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- ~~[ ] API support added~~
- ~~Tests~~
  - ~~[ ] Added for this feature/bug~~
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5502

lib/ci/charts.rb

@@ -1,5 +1,37 @@
 module Ci
   module Charts
+    module DailyInterval
+      def grouped_count(query)
+        query.
+          group("DATE(#{Ci::Build.table_name}.created_at)").
+          count(:created_at).
+          transform_keys { |date| date.strftime(@format) }
+      end
+
+      def interval_step
+        @interval_step ||= 1.day
+      end
+    end
+
+    module MonthlyInterval
+      def grouped_count(query)
+        if Gitlab::Database.postgresql?
+          query.
+            group("to_char(#{Ci::Build.table_name}.created_at, '01 Month YYYY')").
+            count(:created_at).
+            transform_keys(&:squish)
+        else
+          query.
+            group("DATE_FORMAT(#{Ci::Build.table_name}.created_at, '01 %M %Y')").
+            count(:created_at)
+        end
+      end
+
+      def interval_step
+        @interval_step ||= 1.month
+      end
+    end
+
     class Chart
       attr_reader :labels, :total, :success, :project, :build_times
 
@@ -13,47 +45,59 @@ module Ci
         collect
       end
 
-      def push(from, to, format)
+      def collect
-        @labels << from.strftime(format)
+        query = project.builds.
-        @total << project.builds.
+          where("? > #{Ci::Build.table_name}.created_at AND #{Ci::Build.table_name}.created_at > ?", @to, @from)
-          where("? > #{Ci::Build.table_name}.created_at AND #{Ci::Build.table_name}.created_at > ?", to, from).
+
-          count(:all)
+        totals_count  = grouped_count(query)
-        @success << project.builds.
+        success_count = grouped_count(query.success)
-          where("? > #{Ci::Build.table_name}.created_at AND #{Ci::Build.table_name}.created_at > ?", to, from).
+
-          success.count(:all)
+        current = @from
+        while current < @to
+          label = current.strftime(@format)
+
+          @labels  << label
+          @total   << (totals_count[label] || 0)
+          @success << (success_count[label] || 0)
+
+          current += interval_step
+        end
       end
     end
 
     class YearChart < Chart
-      def collect
+      include MonthlyInterval
-        13.times do |i|
-          start_month = (Date.today.years_ago(1) + i.month).beginning_of_month
-          end_month = start_month.end_of_month
 
-          push(start_month, end_month, "%d %B %Y")
+      def initialize(*)
-        end
+        @to     = Date.today.end_of_month
+        @from   = @to.years_ago(1).beginning_of_month
+        @format = '%d %B %Y'
+
+        super
       end
     end
 
     class MonthChart < Chart
-      def collect
+      include DailyInterval
-        30.times do |i|
-          start_day = Date.today - 30.days + i.days
-          end_day = Date.today - 30.days + i.day + 1.day
 
-          push(start_day, end_day, "%d %B")
+      def initialize(*)
-        end
+        @to     = Date.today
+        @from   = @to - 30.days
+        @format = '%d %B'
+
+        super
       end
     end
 
     class WeekChart < Chart
-      def collect
+      include DailyInterval
-        7.times do |i|
-          start_day = Date.today - 7.days + i.days
-          end_day = Date.today - 7.days + i.day + 1.day
 
-          push(start_day, end_day, "%d %B")
+      def initialize(*)
-        end
+        @to     = Date.today
+        @from   = @to - 7.days
+        @format = '%d %B'
+
+        super
       end
     end