gitlab-org--gitlab-foss/app/serializers/README.md

325 lines
12 KiB
Markdown

# Serializers
This is a documentation for classes located in `app/serializers` directory.
In GitLab, we use [grape-entities][grape-entity-project], accompanied by a
serializer, to convert a Ruby object to its JSON representation.
Serializers are typically used in controllers to build a JSON response
that is usually consumed by a frontend code.
## Why using a serializer is important?
Using serializers, instead of `to_json` method, has several benefits:
* it helps to prevent exposure of a sensitive data stored in the database
* it makes it easier to test what should and should not be exposed
* it makes it easier to reuse serialization entities that are building blocks
* it makes it easier to move complexity from controllers to easily testable
classes
* it encourages hiding complexity behind intentions-revealing interfaces
* it makes it easier to take care about serialization performance concerns
* it makes it easier to reduce merge conflicts between CE -> EE
* it makes it easier to benefit from domain driven development techniques
## What is a serializer?
A serializer is a class that encapsulates all business rules for building a
JSON response using serialization entities.
It is designed to be testable and to support passing additional context from
the controller.
## What is a serialization entity?
Entities are lightweight structures that allow to represent domain models
in a consistent and abstracted way, and reuse them as building blocks to
create a payload.
Entities located in `app/serializers` are usually derived from a
[`Grape::Entity`][grape-entity-class] class.
Serialization entities that do require to have a knowledge about specific
elements of the request, need to mix `RequestAwareEntity` in.
A serialization entity usually maps a domain model class into its JSON
representation. It rarely happens that a serialization entity exists without
a corresponding domain model class. As an example, we have an `Issue` class and
a corresponding `IssueSerializer`.
Serialization entities are designed to reuse other serialization entities, which
is a convenient way to create a multi-level JSON representation of a piece of
a domain model you want to serialize.
See [documentation for Grape Entities][grape-entity-readme] for more details.
## How to implement a serializer?
### Base implementation
In order to effectively implement a serializer it is necessary to create a new
class in `app/serializers`. See existing serializers as an example.
A new serializer should inherit from a `BaseSerializer` class. It is necessary
to specify which serialization entity will be used to serialize a resource.
```ruby
class MyResourceSerializer < BaseSerializer
entity MyResourceEntity
end
```
The example above shows how a most simple serializer can look like.
Given that the entity `MyResourceEntity` exists, you can now use
`MyResourceSerializer` in the controller by creating an instance of it, and
calling `MyResourceSerializer#represent(resource)` method.
Note that a `resource` can be either a single object, an array of objects or an
`ActiveRecord::Relation` object. A serialization entity should be smart enough
to accurately represent each of these.
It should not be necessary to use `Enumerable#map`, and it should be avoided
from the performance reasons.
### Choosing what gets serialized
It often happens that you might want to use the same serializer in many places,
but sometimes the intention is to only expose a small subset of object's
attributes in one place, and a different subset in another.
`BaseSerializer#represent(resource, opts = {})` method can take an additional
hash argument, `opts`, that defines what is going to be serialized.
`BaseSerializer` will pass these options to a serialization entity. See
how it is [documented in the upstream project][grape-entity-only].
With this approach you can extend the serializer to respond to methods that will
create a JSON response according to your needs.
```ruby
class PipelineSerializer < BaseSerializer
entity Ci::PipelineEntity
def represent_details(resource)
represent(resource, only: [:details])
end
def represent_status(resource)
represent(resource, only: [:status])
end
end
```
It is possible to use `only` and `except` keywords. Both keywords do support
nested attributes, like `except: [:id, { user: [:id] }]`.
Passing `only` and `except` to the `represent` method from a controller is
possible, but it defies principles of encapsulation and testability, and it is
better to avoid it, and to add a specific method to the serializer instead.
### Reusing serialization entities from the API
Public API in GitLab is implemented using [Grape][grape-project].
Under the hood it also uses [`Grape::Entity`][grape-entity-class] classes.
This means that it is possible to reuse these classes to implement internal
serializers.
You can either use such entity directly:
```ruby
class MyResourceSerializer < BaseSerializer
entity API::Entities::SomeEntity
end
```
Or derive a new serialization entity class from it:
```ruby
class MyEntity < API::Entities::SomeEntity
include RequestAwareEntity
unexpose :something
end
```
It might be a good idea to write specs for entities that do inherit from
the API, because when API payloads are changed / extended, it is easy to forget
about the impact on the internal API through a serializer that reuses API
entities.
It is usually safe to do that, because API entities rarely break backward
compatibility, but additional exposure may have a performance impact when API
gets extended significantly. Write tests that check if only necessary data is
exposed.
## How to write tests for a serializer?
Like every other class in the project, creating a serializer warrants writing
tests for it.
It is usually a good idea to test each public method in the serializer against
a valid payload. `BaseSerializer#represent` returns a hash, so it is possible
to use usual RSpec matchers like `include`.
Sometimes, when the payload is large, it makes sense to validate it entirely
using `match_response_schema` matcher along with a new fixture that can be
stored in `spec/fixtures/api/schemas/`. This matcher is using a `json-schema`
gem, which is quite flexible, see a [documentation][json-schema-gem] for it.
## How to use a serializer in a controller?
Once a new serializer is implemented, it is possible to use it in a controller.
Create an instance of the serializer and render the response.
```ruby
def index
format.json do
render json: MyResourceSerializer
.new(current_user: @current_user)
.represent_details(@project.resources)
end
end
```
If it is necessary to include additional information in the payload, it is
possible to extend what is going to be rendered, the usual way:
```ruby
def index
format.json do
render json: {
resources: MyResourceSerializer
.new(current_user: @current_user)
.represent_details(@project.resources),
count: @project.resources.count
}
end
end
```
Note that in these examples an additional context is being passed to the
serializer (`current_user: @current_user`).
## How to pass an additional context from the controller?
It is possible to pass an additional context from a controller to a
serializer and each serialization entity that is used in the process.
Serialization entities that do require an additional context have
`RequestAwareEntity` concern mixed in. This piece of the code exposes a method
called `request` in every serialization entity that is instantiated during
serialization.
An object returned by this method is an instance of `EntityRequest`, which
behaves like an `OpenStruct` object, with the difference that it will raise
an error if an unknown method is called.
In other words, in the previous example, `request` method will return an
instance of `EntityRequest` that responds to `current_user` method. It will be
available in every serialization entity instantiated by `MyResourceSerializer`.
`EntityRequest` is a workaround for [#20045][issue-20045] and is meant to be
refactored soon. Please avoid passing an additional context that is not
required by a serialization entity.
At the moment, the context that is passed to entities most often is
`current_user` and `project`.
## How is this related to using presenters?
Payload created by a serializer is usually a representation of the backed code,
combined with the current request data. Therefore, technically, serializers
are presenters that create payload consumed by a frontend code, usually Vue
components.
In GitLab, it is possible to use [presenters][presenters-readme], but
`BaseSerializer` still needs to learn how to use it, see [#30898][issue-30898].
It is possible to use presenters when serializer is used to represent only
a single object. It is not supported when `ActiveRecord::Relation` is being
serialized.
```ruby
MyObjectSerializer.new.represent(object.present)
```
## Best practices
1. Do not invoke a serializer from within a serialization entity.
If you need to use a serializer from within a serialization entity, it is
possible that you are missing a class for an important domain concept.
Consider creating a new domain class and a corresponding serialization
entity for it.
1. Use only one approach to switch behavior of the serializer.
It is possible to use a few approaches to switch a behavior of the
serializer. Most common are using a [Fluent Interface][fluent-interface]
and creating a separate `represent_something` methods.
Whatever you choose, it might be better to use only one approach at a time.
1. Do not forget about creating specs for serialization entities.
Writing tests for the serializer indeed does cover testing a behavior of
serialization entities that the serializer instantiates. However it might
be a good idea to write separate tests for entities as well, because these
are meant to be reused in different serializers, and a serializer can
change a behavior of a serialization entity.
1. Use `ActiveRecord::Relation` where possible
Using an `ActiveRecord::Relation` might help from the performance perspective.
1. Be diligent about passing an additional context from the controller.
Using `EntityRequest` and `RequestAwareEntity` is a workaround for the lack
of high-level mechanism. It is meant to be refactored, and current
implementation is error prone. Imagine the situation that one serialization
entity requires `request.user` attribute, but the second one wants
`request.current_user`. When it happens that these two entities are used in
the same serialization request, you might need to pass both parameters to
the serializer, which is obviously not a perfect situation.
When in doubt, pass only `current_user` and `project` if these are required.
1. Keep performance concerns in mind
Using a serializer incorrectly can have significant impact on the
performance.
Because serializers are technically presenters, it is often necessary
to calculate, for example, paths to various controller-actions.
Since using URL helpers usually involve passing `project` and `namespace`
adding `includes(project: :namespace)` in the serializer, can help to avoid
N+1 queries.
Also, try to avoid using `Enumerable#map` or other methods that will
execute a database query eagerly.
1. Avoid passing `only` and `except` from the controller.
1. Write tests checking for N+1 queries.
1. Write controller tests for actions / formats using serializers.
1. Write tests that check if only necessary data is exposed.
1. Write tests that check if no sensitive data is exposed.
## Future
* [Next iteration of serializers][issue-27569]
[grape-project]: http://www.ruby-grape.org
[grape-entity-project]: https://github.com/ruby-grape/grape-entity
[grape-entity-readme]: https://github.com/ruby-grape/grape-entity/blob/master/README.md
[grape-entity-class]: https://github.com/ruby-grape/grape-entity/blob/master/lib/grape_entity/entity.rb
[grape-entity-only]: https://github.com/ruby-grape/grape-entity/blob/master/README.md#returning-only-the-fields-you-want
[presenters-readme]: https://gitlab.com/gitlab-org/gitlab-foss/blob/master/app/presenters/README.md
[fluent-interface]: https://en.wikipedia.org/wiki/Fluent_interface
[json-schema-gem]: https://github.com/ruby-json-schema/json-schema
[issue-20045]: https://gitlab.com/gitlab-org/gitlab-foss/issues/20045
[issue-30898]: https://gitlab.com/gitlab-org/gitlab-foss/issues/30898
[issue-27569]: https://gitlab.com/gitlab-org/gitlab-foss/issues/27569