Search query is especially slow if a user searches a generic string
which matches many records, in such case search can take tens of
seconds or time out. To speed up the search query, we search only for
first 1000 records, if there is >1000 matching records we just display
"1000+" instead of precise total count supposing that with such amount
the exact count is not so important for the user.
Because for issues even limited search was not fast enough, 2-phase
approach is used for issues: first we use simpler/faster query to get
all public issues, if this exceeds the limit, we just return the limit.
If the amount of matching results is lower than limit, we re-run more
complex search query (which includes also confidential issues).
Re-running the complex query should be fast enough in such case because the
amount of matching issues is lower than limit.
Because exact total_count is now limited, this patch also switches to
to "prev/next" pagination.
Related #40540
We're going to cache the total open count separately, and then just perform
these counts on the list. We already do that to get the pagination information,
through Kaminari, and a future change will make Kaminari reuse the query results
from earlier in the request.
When an issuable's state changes, or one is created, we should clear the cache
counts for a user's assigned issuables, and also the project-wide caches for
this user type.
This runs a slightly slower query to get the issue and MR counts in the
navigation, but caches by user type (can see all / none confidential issues) for
two minutes.
When we are filtering by a single project, and the current user has access to
see confidential issues on that project, we don't need to filter by
confidentiality at all - just as if the user were an admin.
The filter by confidentiality often picks a non-optimal query plan: for
instance, AND-ing the results of all issues in the project (a relatively small
set), and all issues in the states requested (a huge set), rather than just
starting small and winnowing further.
In CE only the admin has access to all private groups & projects. In EE also an
auditor can have full private access.
To overcome merge conflicts, or accidental incorrect access rights, abstract
this out in `User#full_private_access?`.
`User#admin?` now only should be used for admin-only features. For private
access-related features `User#full_private_access?` should be used.
Backported from gitlab-org/gitlab-ee!2199
Jan Provaznik