2017-05-04 14:28:28 +00:00
|
|
|
# Serializers
|
|
|
|
|
|
|
|
This is a documentation for classes located in `app/serializers` directory.
|
|
|
|
|
|
|
|
In GitLab, we use [grape-entities][grape-entity-project], accompanied by a
|
|
|
|
serializer, to convert a Ruby object to its JSON representation.
|
|
|
|
|
|
|
|
Serializers are typically used in controllers to build a JSON response
|
|
|
|
that is usually consumed by a frontend code.
|
|
|
|
|
|
|
|
## Why using a serializer is important?
|
|
|
|
|
|
|
|
Using serializers, instead of `to_json` method, has several benefits:
|
|
|
|
|
|
|
|
* it helps to prevent exposure of a sensitive data stored in the database
|
|
|
|
* it makes it easier to test what should and should not be exposed
|
|
|
|
* it makes it easier to reuse serialization entities that are building blocks
|
|
|
|
* it makes it easier to move complexity from controllers to easily testable
|
|
|
|
classes
|
|
|
|
* it encourages hiding complexity behind intentions-revealing interfaces
|
|
|
|
* it makes it easier to take care about serialization performance concerns
|
|
|
|
* it makes it easier to reduce merge conflicts between CE -> EE
|
|
|
|
* it makes it easier to benefit from domain driven development techniques
|
|
|
|
|
|
|
|
## What is a serializer?
|
|
|
|
|
|
|
|
A serializer is a class that encapsulates all business rules for building a
|
|
|
|
JSON response using serialization entities.
|
|
|
|
|
|
|
|
It is designed to be testable and to support passing additional context from
|
|
|
|
the controller.
|
|
|
|
|
|
|
|
## What is a serialization entity?
|
|
|
|
|
|
|
|
Entities are lightweight structures that allow to represent domain models
|
|
|
|
in a consistent and abstracted way, and reuse them as building blocks to
|
|
|
|
create a payload.
|
|
|
|
|
|
|
|
Entities located in `app/serializers` are usually derived from a
|
|
|
|
[`Grape::Entity`][grape-entity-class] class.
|
|
|
|
|
|
|
|
Serialization entities that do require to have a knowledge about specific
|
|
|
|
elements of the request, need to mix `RequestAwareEntity` in.
|
|
|
|
|
|
|
|
A serialization entity usually maps a domain model class into its JSON
|
|
|
|
representation. It rarely happens that a serialization entity exists without
|
|
|
|
a corresponding domain model class. As an example, we have an `Issue` class and
|
|
|
|
a corresponding `IssueSerializer`.
|
|
|
|
|
|
|
|
Serialization entites are designed to reuse other serialization entities, which
|
|
|
|
is a convenient way to create a multi-level JSON representation of a piece of
|
|
|
|
a domain model you want to serialize.
|
|
|
|
|
|
|
|
See [documentation for Grape Entites][grape-entity-readme] for more details.
|
|
|
|
|
|
|
|
## How to implement a serializer?
|
|
|
|
|
|
|
|
### Base implementation
|
|
|
|
|
|
|
|
In order to effectively implement a serializer it is necessary to create a new
|
|
|
|
class in `app/serializers`. See existing serializers as an example.
|
|
|
|
|
|
|
|
A new serializer should inherit from a `BaseSerializer` class. It is necessary
|
|
|
|
to specify which serialization entity will be used to serialize a resource.
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
class MyResourceSerializer < BaseSerialize
|
|
|
|
entity MyResourceEntity
|
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
The example above shows how a most simple serializer can look like.
|
|
|
|
|
|
|
|
Given that the entity `MyResourceEntity` exists, you can now use
|
|
|
|
`MyResourceSerializer` in the controller by creating an instance of it, and
|
|
|
|
calling `MyResourceSerializer#represent(resource)` method.
|
|
|
|
|
|
|
|
Note that a `resource` can be either a single object, an array of objects or an
|
|
|
|
`ActiveRecord::Relation` object. A serialization entity should be smart enough
|
|
|
|
to accurately represent each of these.
|
|
|
|
|
|
|
|
It should not be necessary to use `Enumerable#map`, and it should be avoided
|
|
|
|
from the performance reasons.
|
|
|
|
|
|
|
|
### Choosing what gets serialized
|
|
|
|
|
|
|
|
It often happens that you might want to use the same serializer in many places,
|
|
|
|
but sometimes the intention is to only expose a small subset of object's
|
|
|
|
attributes in one place, and a different subset in another.
|
|
|
|
|
|
|
|
`BaseSerializer#represent(resource, opts = {})` method can take an additional
|
|
|
|
hash argument, `opts`, that defines what is going to be serialized.
|
|
|
|
|
|
|
|
`BaseSerializer` will pass these options to a serialization entity. See
|
|
|
|
how it is [documented in the upstream project][grape-entity-only].
|
|
|
|
|
|
|
|
With this approach you can extend the serializer to respond to methods that will
|
|
|
|
create a JSON response according to your needs.
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
class PipelineSerializer < BaseSerializer
|
|
|
|
entity PipelineEntity
|
|
|
|
|
|
|
|
def represent_details(resource)
|
|
|
|
represent(resource, only: [:details])
|
|
|
|
end
|
|
|
|
|
|
|
|
def represent_status(resource)
|
|
|
|
represent(resource, only: [:status])
|
|
|
|
end
|
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
It is possible to use `only` and `except` keywords. Both keywords do support
|
|
|
|
nested attributes, like `except: [:id, { user: [:id] }]`.
|
|
|
|
|
|
|
|
Passing `only` and `except` to the `represent` method from a controller is
|
|
|
|
possible, but it defies principles of encapsulation and testability, and it is
|
|
|
|
better to avoid it, and to add a specific method to the serializer instead.
|
|
|
|
|
|
|
|
### Reusing serialization entities from the API
|
|
|
|
|
|
|
|
Public API in GitLab is implemented using [Grape][grape-project].
|
|
|
|
|
|
|
|
Under the hood it also uses [`Grape::Entity`][grape-entity-class] classes.
|
|
|
|
This means that it is possible to reuse these classes to implement internal
|
|
|
|
serializers.
|
|
|
|
|
|
|
|
You can either use such entity directly:
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
class MyResourceSerializer < BaseSerializer
|
|
|
|
entity API::Entities::SomeEntity
|
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
Or derive a new serialization entity class from it:
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
class MyEntity < API::Entities::SomeEntity
|
|
|
|
include RequestAwareEntity
|
|
|
|
|
|
|
|
unexpose :something
|
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
It might be a good idea to write specs for entities that do inherit from
|
|
|
|
the API, because when API payloads are changed / extended, it is easy to forget
|
|
|
|
about the impact on the internal API through a serializer that reuses API
|
|
|
|
entities.
|
|
|
|
|
|
|
|
It is usually safe to do that, because API entities rarely break backward
|
|
|
|
compatibility, but additional exposure may have a performance impact when API
|
|
|
|
gets extended significantly. Write tests that check if only necessary data is
|
|
|
|
exposed.
|
|
|
|
|
|
|
|
## How to write tests for a serializer?
|
|
|
|
|
|
|
|
Like every other class in the project, creating a serializer warrants writing
|
|
|
|
tests for it.
|
|
|
|
|
|
|
|
It is usually a good idea to test each public method in the serializer against
|
|
|
|
a valid payload. `BaseSerializer#represent` returns a hash, so it is possible
|
|
|
|
to use usual RSpec matchers like `include`.
|
|
|
|
|
|
|
|
Sometimes, when the payload is large, it makes sense to validate it entirely
|
|
|
|
using `match_response_schema` matcher along with a new fixture that can be
|
|
|
|
stored in `spec/fixtures/api/schemas/`. This matcher is using a `json-schema`
|
|
|
|
gem, which is quite flexible, see a [documentation][json-schema-gem] for it.
|
|
|
|
|
|
|
|
## How to use a serializer in a controller?
|
|
|
|
|
|
|
|
Once a new serializer is implemented, it is possible to use it in a controller.
|
|
|
|
|
|
|
|
Create an instance of the serializer and render the response.
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
def index
|
|
|
|
format.json do
|
|
|
|
render json: MyResourceSerializer
|
|
|
|
.new(current_user: @current_user)
|
|
|
|
.represent_details(@project.resources)
|
2018-11-09 00:19:48 +00:00
|
|
|
nd
|
2017-05-04 14:28:28 +00:00
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
If it is necessary to include additional information in the payload, it is
|
|
|
|
possible to extend what is going to be rendered, the usual way:
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
def index
|
|
|
|
format.json do
|
|
|
|
render json: {
|
|
|
|
resources: MyResourceSerializer
|
|
|
|
.new(current_user: @current_user)
|
|
|
|
.represent_details(@project.resources),
|
|
|
|
count: @project.resources.count
|
|
|
|
}
|
2018-11-09 00:19:48 +00:00
|
|
|
nd
|
2017-05-04 14:28:28 +00:00
|
|
|
end
|
|
|
|
```
|
|
|
|
|
|
|
|
Note that in these examples an additional context is being passed to the
|
|
|
|
serializer (`current_user: @current_user`).
|
|
|
|
|
|
|
|
## How to pass an additional context from the controller?
|
|
|
|
|
|
|
|
It is possible to pass an additional context from a controller to a
|
|
|
|
serializer and each serialization entity that is used in the process.
|
|
|
|
|
|
|
|
Serialization entities that do require an additional context have
|
|
|
|
`RequestAwareEntity` concern mixed in. This piece of the code exposes a method
|
|
|
|
called `request` in every serialization entity that is instantiated during
|
|
|
|
serialization.
|
|
|
|
|
|
|
|
An object returned by this method is an instance of `EntityRequest`, which
|
|
|
|
behaves like an `OpenStruct` object, with the difference that it will raise
|
|
|
|
an error if an unknown method is called.
|
|
|
|
|
|
|
|
In other words, in the previous example, `request` method will return an
|
|
|
|
instance of `EntityRequest` that responds to `current_user` method. It will be
|
|
|
|
available in every serialization entity instantiated by `MyResourceSerializer`.
|
|
|
|
|
|
|
|
`EntityRequest` is a workaround for [#20045][issue-20045] and is meant to be
|
|
|
|
refactored soon. Please avoid passing an additional context that is not
|
|
|
|
required by a serialization entity.
|
|
|
|
|
|
|
|
At the moment, the context that is passed to entities most often is
|
|
|
|
`current_user` and `project`.
|
|
|
|
|
|
|
|
## How is this related to using presenters?
|
|
|
|
|
|
|
|
Payload created by a serializer is usually a representation of the backed code,
|
|
|
|
combined with the current request data. Therefore, technically, serializers
|
|
|
|
are presenters that create payload consumed by a frontend code, usually Vue
|
|
|
|
components.
|
|
|
|
|
|
|
|
In GitLab, it is possible to use [presenters][presenters-readme], but
|
|
|
|
`BaseSerializer` still needs to learn how to use it, see [#30898][issue-30898].
|
|
|
|
|
|
|
|
It is possible to use presenters when serializer is used to represent only
|
|
|
|
a single object. It is not supported when `ActiveRecord::Relation` is being
|
|
|
|
serialized.
|
|
|
|
|
|
|
|
```ruby
|
|
|
|
MyObjectSerializer.new.represent(object.present)
|
|
|
|
```
|
|
|
|
|
|
|
|
## Best practices
|
|
|
|
|
|
|
|
1. Do not invoke a serializer from within a serialization entity.
|
|
|
|
|
|
|
|
If you need to use a serializer from within a serialization entity, it is
|
|
|
|
possible that you are missing a class for an important domain concept.
|
|
|
|
|
|
|
|
Consider creating a new domain class and a corresponding serialization
|
|
|
|
entity for it.
|
|
|
|
|
|
|
|
1. Use only one approach to switch behavior of the serializer.
|
|
|
|
|
|
|
|
It is possible to use a few approaches to switch a behavior of the
|
|
|
|
serializer. Most common are using a [Fluent Interface][fluent-interface]
|
|
|
|
and creating a separate `represent_something` methods.
|
|
|
|
|
|
|
|
Whatever you choose, it might be better to use only one approach at a time.
|
|
|
|
|
|
|
|
1. Do not forget about creating specs for serialization entities.
|
|
|
|
|
|
|
|
Writing tests for the serializer indeed does cover testing a behavior of
|
|
|
|
serialization entities that the serializer instantiates. However it might
|
|
|
|
be a good idea to write separate tests for entities as well, because these
|
|
|
|
are meant to be reused in different serializers, and a serializer can
|
|
|
|
change a behavior of a serialization entity.
|
|
|
|
|
|
|
|
1. Use `ActiveRecord::Relation` where possible
|
|
|
|
|
|
|
|
Using an `ActiveRecord::Relation` might help from the performance perspective.
|
|
|
|
|
|
|
|
1. Be diligent about passing an additional context from the controller.
|
|
|
|
|
|
|
|
Using `EntityRequest` and `RequestAwareEntity` is a workaround for the lack
|
|
|
|
of high-level mechanism. It is meant to be refactored, and current
|
|
|
|
implementation is error prone. Imagine the situation that one serialization
|
|
|
|
entity requires `request.user` attribute, but the second one wants
|
|
|
|
`request.current_user`. When it happens that these two entities are used in
|
|
|
|
the same serialization request, you might need to pass both parameters to
|
|
|
|
the serializer, which is obviously not a perfect situation.
|
|
|
|
|
|
|
|
When in doubt, pass only `current_user` and `project` if these are required.
|
|
|
|
|
|
|
|
1. Keep performance concerns in mind
|
|
|
|
|
|
|
|
Using a serializer incorrectly can have significant impact on the
|
|
|
|
performance.
|
|
|
|
|
|
|
|
Because serializers are technically presenters, it is often necessary
|
|
|
|
to calculate, for example, paths to various controller-actions.
|
|
|
|
Since using URL helpers usually involve passing `project` and `namespace`
|
|
|
|
adding `includes(project: :namespace)` in the serializer, can help to avoid
|
|
|
|
N+1 queries.
|
|
|
|
|
|
|
|
Also, try to avoid using `Enumerable#map` or other methods that will
|
|
|
|
execute a database query eagerly.
|
|
|
|
|
|
|
|
1. Avoid passing `only` and `except` from the controller.
|
|
|
|
1. Write tests checking for N+1 queries.
|
|
|
|
1. Write controller tests for actions / formats using serializers.
|
|
|
|
1. Write tests that check if only necessary data is exposed.
|
|
|
|
1. Write tests that check if no sensitive data is exposed.
|
|
|
|
|
|
|
|
## Future
|
|
|
|
|
|
|
|
* [Next iteration of serializers][issue-27569]
|
|
|
|
|
|
|
|
[grape-project]: http://www.ruby-grape.org
|
|
|
|
[grape-entity-project]: https://github.com/ruby-grape/grape-entity
|
|
|
|
[grape-entity-readme]: https://github.com/ruby-grape/grape-entity/blob/master/README.md
|
|
|
|
[grape-entity-class]: https://github.com/ruby-grape/grape-entity/blob/master/lib/grape_entity/entity.rb
|
|
|
|
[grape-entity-only]: https://github.com/ruby-grape/grape-entity/blob/master/README.md#returning-only-the-fields-you-want
|
|
|
|
[presenters-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/presenters/README.md
|
|
|
|
[fluent-interface]: https://en.wikipedia.org/wiki/Fluent_interface
|
|
|
|
[json-schema-gem]: https://github.com/ruby-json-schema/json-schema
|
|
|
|
[issue-20045]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20045
|
|
|
|
[issue-30898]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30898
|
|
|
|
[issue-27569]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27569
|