Commit Graph

17 Commits

Author SHA1 Message Date
Andreas Brandl 259493bb4c Enable tablesample count strategy by default
https://gitlab.com/gitlab-org/gitlab-ce/issues/58792
2019-07-24 17:00:34 +00:00
Jan Provaznik 3720d02b8b Use approximate counts for big tables 2018-12-06 16:07:14 +00:00
Andreas Brandl 474fd9138c
Move strategies in their own files
This improves readability quite a bit.
2018-12-03 21:26:53 +01:00
Andreas Brandl ff35cb45e9
Fix comment. 2018-12-03 21:26:53 +01:00
Andreas Brandl df6166f4b1
Refactor estimate query 2018-12-03 21:26:52 +01:00
Andreas Brandl 9bb7c690c4
Fix rubocop offenses. 2018-12-03 21:26:51 +01:00
Andreas Brandl 3266642739
Add feature flag for tablesample counting. 2018-12-03 21:26:51 +01:00
Andreas Brandl 3a7d9b4b02
Implement TablesampleCountStrategy.
A tablesample count executes in two phases:
* Estimate table sizes based on reltuples.
* Based on the estimate:
  * If the table is considered 'small', execute an exact relation count.
  * Otherwise, count on a sample of the table using TABLESAMPLE.

The size of the sample is chosen in a way that we always roughly scan
the same amount of rows (see TABLESAMPLE_ROW_TARGET).
2018-12-03 21:26:50 +01:00
Andreas Brandl 72347448db
More specs for fallback testing. 2018-12-03 21:26:49 +01:00
Andreas Brandl 01c7cb90da
Refactor specs to separate concerns. 2018-12-03 21:26:49 +01:00
Andreas Brandl c5fb468255
Flexible approximate counts with fallback strategies. 2018-12-03 21:26:48 +01:00
Andreas Brandl b6a530c9b1
Simplify to use models instead of tables. 2018-12-03 21:26:48 +01:00
Andreas Brandl f2ec9d2d34
Add ExactCountStrategy. 2018-12-03 21:26:48 +01:00
Andreas Brandl f3a9315177
Extract ReltuplesCountStrategy. 2018-12-03 21:26:47 +01:00
gfyoung 7e6f6e1603 Enable even more frozen string in lib/gitlab
Enables frozens string for the following:

* lib/gitlab/conflict/**/*.rb
* lib/gitlab/cross_project_access/**/*.rb
* lib/gitlab/cycle_analytics/**/*.rb
* lib/gitlab/data_builder/**/*.rb
* lib/gitlab/database/**/*.rb
* lib/gitlab/dependency_linker/**/*.rb
* lib/gitlab/diff/**/*.rb
* lib/gitlab/downtime_check/**/*.rb
* lib/gitlab/email/**/*.rb
* lib/gitlab/etag_caching/**/*.rb

Partially addresses gitlab-org/gitlab-ce#47424.
2018-11-06 22:47:32 -08:00
Stan Hu b6125f7045 Fix fast admin counters not working when PostgreSQL has secondaries
This commit does a number of things:

1. Reduces the number of queries needed by perform a single query to get all
the tuples for the relevant rows.

2. Uses a transaction to query the tuple counts to ensure that the data
is retrieved from the primary.

Closes #46742
2018-05-25 22:58:43 -07:00
Stan Hu e38938b332 Fix Error 500 viewing admin page due to statement timeouts
Uses PostgreSQL tuple estimates to provide a much faster yet approximate
count. See https://wiki.postgresql.org/wiki/Slow_Counting for more details.
We only use this fast method if the table has been analyzed or vacuumed
within the last hour.

Closes #46255
2018-05-16 08:27:48 -07:00