This refactors the Markdown pipeline so it supports the rendering of
multiple documents that may belong to different projects. An example of
where this happens is when displaying the event feed of a group. In this
case we retrieve events for all projects in the group. Previously we
would group events per project and render these chunks separately, but
this would result in many SQL queries being executed. By extending the
Markdown pipeline to support this out of the box we can drastically
reduce the number of SQL queries.
To achieve this we introduce a new object to the pipeline:
Banzai::RenderContext. This object simply wraps two other objects: an
optional Project instance, and an optional User instance. On its own
this wouldn't be very helpful, but a RenderContext can also be used to
associate HTML documents with specific Project instances. This work is
done in Banzai::ObjectRenderer and allows us to reuse as many queries
(and results) as possible.
Example: for issues that are closed, the links will now show '[closed]'
following the issue number. This is done as post-process after the markdown has
been loaded from the cache as the status of the issue may change between
the cache being populated and the content being displayed.
In order to avoid N+1 queries problem when rendering notes ObjectRenderer
populates the cache of referenced issuables for all notes at once,
before the post processing phase.
As a part of this change, the Banzai BaseParser#grouped_objects_for_nodes
method has been refactored to return a Hash utilising the node itself as the
key, since this was a common pattern of usage for this method.
This commit changes the way certain documents are rendered (currently
only Notes) and how documents are redacted. Previously both rendering
and redacting would run on a per document basis. The result of this was
that for every document we'd have to run countless queries just to
figure out if we could display a set of links or not.
This commit changes things around so that redacting Markdown documents
is no longer tied into the html-pipeline Gem. This in turn allows it to
redact multiple documents in a single pass, thus reducing the number of
queries needed.
In turn rendering issue/merge request notes has been adjusted to take
advantage of this new setup. Instead of rendering Markdown somewhere
deep down in a view the Markdown is rendered and redacted in the
controller (taking the current user and all that into account). This has
been done in such a way that the "markdown()" helper method can still be
used on its own.
This particular commit also paves the way for caching rendered HTML on
object level. Right now there's an accessor method Note#note_html which
is used for setting/getting the rendered HTML. Once we cache HTML on row
level we can simply change this field to be a column and call a "save"
whenever needed and we're pretty much done.
This splits the Markdown rendering and reference extraction phases into
two distinct code bases. The reference extraction phase no longer relies
on the html-pipeline Gem (and any related code) and allows for
extracting of references from multiple HTML nodes in a single pass. This
means that if you want to extract user references from 200 comments you
no longer need to run 200 times N number of queries, instead only a
handful of queries may be needed.
Nokogiri produces inefficient XPath expressions when given CSS
expressions such as "a.gfm". Luckily these expressions can be optimized
quite easily while still achieving the same results.
In the two cases where this optimization is applied the run time has
been reduced from around 170 ms to around 15 ms.
Yorick Peterse