Avoid hitting statement timeout finding MR pipelines
For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very slow. What we can't do to fix this: 1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony. 2. Do the `uniq` in Ruby. That can still be very slow with hundreds of thousands of commits. 3. Use a subquery. We haven't removed the `st_commits` column yet, but we will soon. Until 3 is available to us, we can just do 2, but also add a limit clause. There is no ordering, so this may return different results, but our goal with these MRs is just to get them to load, so it's not a huge deal.
This commit is contained in:
parent
e9d3526123
commit
ba9b4c4de8
2 changed files with 6 additions and 1 deletions
|
@ -874,7 +874,7 @@ class MergeRequest < ActiveRecord::Base
|
|||
#
|
||||
def all_commit_shas
|
||||
if persisted?
|
||||
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).pluck('DISTINCT(sha)')
|
||||
column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).limit(10_000).pluck('sha')
|
||||
serialised_shas = merge_request_diffs.where.not(st_commits: nil).flat_map(&:commit_shas)
|
||||
|
||||
(column_shas + serialised_shas).uniq
|
||||
|
|
|
@ -0,0 +1,5 @@
|
|||
---
|
||||
title: Stop merge requests with thousands of commits from timing out
|
||||
merge_request: 15063
|
||||
author:
|
||||
type: performance
|
Loading…
Reference in a new issue