12 KiB
stage | group | info |
---|---|---|
none | unassigned | To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments |
API style guide
This style guide recommends best practices for API development.
Instance variables
Please do not use instance variables, there is no need for them (we don't need to access them as we do in Rails views), local variables are fine.
Entities
Always use an Entity to present the endpoint's payload.
Documentation
Each new or updated API endpoint must come with documentation, unless it is internal or behind a feature flag. The docs should be in the same merge request, or, if strictly necessary, in a follow-up with the same milestone as the original merge request.
See the Documentation Style Guide RESTful API page for details on documenting API resources in Markdown as well as in OpenAPI definition files.
Methods and parameters description
Every method must be described using the Grape DSL (see https://gitlab.com/gitlab-org/gitlab/blob/master/lib/api/environments.rb for a good example):
-
desc
for the method summary. You should pass it a block for additional details such as:- The GitLab version when the endpoint was added. If it is behind a feature flag, mention that instead: This feature is gated by the :feature_flag_symbol feature flag.
- If the endpoint is deprecated, and if so, when will it be removed
-
params
for the method parameters. This acts as description, validation, and coercion of the parameters
A good example is as follows:
desc 'Get all broadcast messages' do
detail 'This feature was introduced in GitLab 8.12.'
success Entities::BroadcastMessage
end
params do
optional :page, type: Integer, desc: 'Current page number'
optional :per_page, type: Integer, desc: 'Number of messages per page'
end
get do
messages = BroadcastMessage.all
present paginate(messages), with: Entities::BroadcastMessage
end
Declared parameters
Grape allows you to access only the parameters that have been declared by your
params
block. It filters out the parameters that have been passed, but are not allowed.
– https://github.com/ruby-grape/grape#declared
Exclude parameters from parent namespaces
By default
declared(params)
includes parameters that were defined in all parent namespaces.
– https://github.com/ruby-grape/grape#include-parent-namespaces
In most cases you will want to exclude parameters from the parent namespaces:
declared(params, include_parent_namespaces: false)
When to use declared(params)
You should always use declared(params)
when you pass the parameters hash as
arguments to a method call.
For instance:
# bad
User.create(params) # imagine the user submitted `admin=1`... :)
# good
User.create(declared(params, include_parent_namespaces: false).to_h)
NOTE:
declared(params)
return a Hashie::Mash
object, on which you will have to
call .to_h
.
But we can use params[key]
directly when we access single elements.
For instance:
# good
Model.create(foo: params[:foo])
Array types
With Grape v1.3+, Array types must be defined with a coerce_with
block, or parameters will fail to validate when passed a string from an
API request. See the Grape upgrading
documentation
for more details.
Automatic coercion of nil inputs
Prior to Grape v1.3.3, Array parameters with nil
values would
automatically be coerced to an empty Array. However, due to this pull
request in v1.3.3, this
is no longer the case. For example, suppose you define a PUT /test
request that has an optional parameter:
optional :user_ids, type: Array[Integer], coerce_with: ::API::Validations::Types::CommaSeparatedToIntegerArray.coerce, desc: 'The user ids for this rule'
Normally, a request to PUT /test?user_ids
would cause Grape to pass
params
of { user_ids: nil }
.
This may introduce errors with endpoints that expect a blank array and
do not handle nil
inputs properly. To preserve the previous behavior,
there is a helper method coerce_nil_params_to_array!
that is used
in the before
block of all API calls:
before do
coerce_nil_params_to_array!
end
With this change, a request to PUT /test?user_ids
will cause Grape to
pass params
to be { user_ids: [] }
.
There is an open issue in the Grape tracker to make this easier.
Using HTTP status helpers
For non-200 HTTP responses, use the provided helpers in lib/api/helpers.rb
to ensure correct behavior (not_found!
, no_content!
etc.). These will throw
inside Grape and abort the execution of your endpoint.
For DELETE
requests, you should also generally use the destroy_conditionally!
helper which by default returns a 204 No Content
response on success, or a 412 Precondition Failed
response if the given If-Unmodified-Since
header is out of range. This helper calls #destroy
on the passed resource, but you can also implement a custom deletion method by passing a block.
Using API path helpers in GitLab Rails codebase
Because we support installing GitLab under a relative URL, one must take this
into account when using API path helpers generated by Grape. Any such API path
helper usage must be in wrapped into the expose_path
helper call.
For instance:
- endpoint = expose_path(api_v4_projects_issues_related_merge_requests_path(id: @project.id, issue_iid: @issue.iid))
Custom Validators
In order to validate some parameters in the API request, we validate them before sending them further (say Gitaly). The following are the custom validators, which we have added so far and how to use them. We also wrote a guide on how you can add a new custom validator.
Using custom validators
-
FilePath
:GitLab supports various functionalities where we need to traverse a file path. The
FilePath
validator validates the parameter value for different cases. Mainly, it checks whether a path is relative and does it contain../../
relative traversal usingFile::Separator
or not, and whether the path is absolute, for example/etc/passwd/
. By default, absolute paths are not allowed. However, you can optionally pass in an allowlist for allowed absolute paths in the following way:requires :file_path, type: String, file_path: { allowlist: ['/foo/bar/', '/home/foo/', '/app/home'] }
-
Git SHA
:The
Git SHA
validator checks whether the Git SHA parameter is a valid SHA. It checks by using the regex mentioned incommit.rb
file. -
Absence
:The
Absence
validator checks whether a particular parameter is absent in a given parameters hash. -
IntegerNoneAny
:The
IntegerNoneAny
validator checks if the value of the given parameter is either anInteger
,None
, orAny
. It allows only either of these mentioned values to move forward in the request. -
ArrayNoneAny
:The
ArrayNoneAny
validator checks if the value of the given parameter is either anArray
,None
, orAny
. It allows only either of these mentioned values to move forward in the request. -
EmailOrEmailList
:The
EmailOrEmailList
validator checks if the value of a string or a list of strings contains only valid email addresses. It allows only lists with all valid email addresses to move forward in the request.
Adding a new custom validator
Custom validators are a great way to validate parameters before sending them to platform for further processing. It saves some back-and-forth from the server to the platform if we identify invalid parameters at the beginning.
If you need to add a custom validator, it would be added to
it's own file in the validators
directory.
Since we use Grape to add our API
we inherit from the Grape::Validations::Base
class in our validator class.
Now, all you have to do is define the validate_param!
method which takes
in two parameters: the params
hash and the param
name to validate.
The body of the method does the hard work of validating the parameter value and returns appropriate error messages to the caller method.
Lastly, we register the validator using the line below:
Grape::Validations.register_validator(<validator name as symbol>, ::API::Helpers::CustomValidators::<YourCustomValidatorClassName>)
Once you add the validator, make sure you add the rspec
s for it into
it's own file in the validators
directory.
Internal API
The internal API is documented for internal use. Please keep it up to date so we know what endpoints different components are making use of.
Avoiding N+1 problems
In order to avoid N+1 problems that are common when returning collections of records in an API endpoint, we should use eager loading.
A standard way to do this within the API is for models to implement a
scope called with_api_entity_associations
that will preload the
associations and data returned in the API. An example of this scope can
be seen in
the Issue
model.
In situations where the same model has multiple entities in the API
(for instance, UserBasic
, User
and UserPublic
) you should use your
discretion with applying this scope. It may be that you optimize for the
most basic entity, with successive entities building upon that scope.
The with_api_entity_associations
scope will also automatically preload
data
for Todo
targets when returned in the to-dos API.
For more context and discussion about preloading see this merge request which introduced the scope.
Verifying with tests
When an API endpoint returns collections, always add a test to verify
that the API endpoint does not have an N+1 problem, now and in the future.
We can do this using ActiveRecord::QueryRecorder
.
Example:
def make_api_request
get api('/foo', personal_access_token: pat)
end
it 'avoids N+1 queries', :request_store do
# Firstly, record how many PostgreSQL queries the endpoint will make
# when it returns a single record
create_record
control = ActiveRecord::QueryRecorder.new { make_api_request }
# Now create a second record and ensure that the API does not execute
# any more queries than before
create_record
expect { make_api_request }.not_to exceed_query_limit(control)
end
Testing
When writing tests for new API endpoints, consider using a schema fixture located in /spec/fixtures/api/schemas
. You can expect
a response to match a given schema:
expect(response).to match_response_schema('merge_requests')
Also see verifying N+1 performance in tests.
Include a changelog entry
All client-facing changes must include a changelog entry. This does not include internal APIs.