Previously when a user requested a list of projects,
`Project#public_or_visible_to_user` would search all authorized projects
and public/internal projects as well. However, when a user requests a
specific `visibility_level` (e.g. private), that should reduce the
search space, and we shouldn't need to load public/internal projects.
Improves https://gitlab.com/gitlab-org/gitlab-ce/issues/59329
