111 lines
3.8 KiB
Markdown
111 lines
3.8 KiB
Markdown
|
# Code Intelligence
|
||
|
|
|
||
|
|
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/1576) in GitLab 13.1.
|
||
|
|
|
||
|
|
This document describes the design behind [Code Intelligence](../../user/project/code_intelligence.md).
|
||
|
|
|
||
|
|
GitLab's built-in Code Intelligence is powered by
|
||
|
|
[LSIF](https://lsif.dev) and comes down to generating an LSIF document for a
|
||
|
|
project in a CI job, processing the data, uploading it as a CI artifact and
|
||
|
|
displaying this information for the files in the project.
|
||
|
|
|
||
|
|
Here is a sequence diagram for uploading an LSIF artifact:
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
sequenceDiagram
|
||
|
|
participant Runner
|
||
|
|
participant Workhorse
|
||
|
|
participant Rails
|
||
|
|
participant Object Storage
|
||
|
|
|
||
|
|
Runner->>+Workhorse: POST /v4/jobs/:id/artifacts
|
||
|
|
Workhorse->>+Rails: POST /:id/artifacts/authorize
|
||
|
|
Rails-->>-Workhorse: Respond with ProcessLsif header
|
||
|
|
Note right of Workhorse: Process LSIF file
|
||
|
|
Workhorse->>+Object Storage: Put file
|
||
|
|
Object Storage-->>-Workhorse: request results
|
||
|
|
Workhorse->>+Rails: POST /:id/artifacts
|
||
|
|
Rails-->>-Workhorse: request results
|
||
|
|
Workhorse-->>-Runner: request results
|
||
|
|
```
|
||
|
|
|
||
|
|
1. The CI/CD job generates a document in an LSIF format (usually `dump.lsif`) using [an
|
||
|
|
indexer](https://lsif.dev) for the language of a project. The format
|
||
|
|
[describes](https://github.com/sourcegraph/sourcegraph/blob/master/doc/user/code_intelligence/writing_an_indexer.md)
|
||
|
|
interactions between a method or function and its definition(s) or references. The
|
||
|
|
document is marked to be stored as an LSIF report artifact.
|
||
|
|
|
||
|
|
1. After receiving a request for storing the artifact, Workhorse asks
|
||
|
|
GitLab Rails to authorize the upload.
|
||
|
|
|
||
|
|
1. GitLab Rails validates whether the artifact can be uploaded and sends
|
||
|
|
`ProcessLsif: true` header if the lsif artifact can be processed.
|
||
|
|
|
||
|
|
1. Workhorse reads the LSIF document line by line and generates code intelligence
|
||
|
|
data for each file in the project. The output is a zipped directory of JSON
|
||
|
|
files which imitates the structure of the project:
|
||
|
|
|
||
|
|
Project:
|
||
|
|
|
||
|
|
```code
|
||
|
|
app
|
||
|
|
controllers
|
||
|
|
application_controller.rb
|
||
|
|
models
|
||
|
|
application.rb
|
||
|
|
```
|
||
|
|
|
||
|
|
Generated data:
|
||
|
|
|
||
|
|
```code
|
||
|
|
app
|
||
|
|
controllers
|
||
|
|
application_controller.rb.json
|
||
|
|
models
|
||
|
|
application.rb.json
|
||
|
|
```
|
||
|
|
|
||
|
|
1. The zipped directory is stored as a ZIP artifact. Workhorse replaces the
|
||
|
|
original LSIF document with a set of JSON files in the ZIP artifact and
|
||
|
|
generates metadata for it. The metadata makes it possible to view a single
|
||
|
|
file in a ZIP file without unpacking or loading the whole file. That allows us
|
||
|
|
to access code intelligence data for a single file.
|
||
|
|
|
||
|
|
1. When a file is viewed in the GitLab application, frontend fetches code
|
||
|
|
intelligence data for the file directly from the object storage. The file
|
||
|
|
contains information about code units in the file. For example:
|
||
|
|
|
||
|
|
```json
|
||
|
|
[
|
||
|
|
{
|
||
|
|
"definition_path": "cmd/check/main.go#L4",
|
||
|
|
"hover": [
|
||
|
|
{
|
||
|
|
"language": "go",
|
||
|
|
"tokens": [
|
||
|
|
[
|
||
|
|
{
|
||
|
|
"class": "kn",
|
||
|
|
"value": "package"
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"value": " "
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"class": "s",
|
||
|
|
"value": "\"fmt\""
|
||
|
|
}
|
||
|
|
]
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"value": "Package fmt implements formatted I/O with functions analogous to C's printf and scanf. The format 'verbs' are derived from C's but are simpler. \n\n### hdr-PrintingPrinting\nThe verbs: \n\nGeneral: \n\n```\n%v\tthe value in a default format\n\twhen printing st..."
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"start_char": 2,
|
||
|
|
"start_line": 33
|
||
|
|
}
|
||
|
|
...
|
||
|
|
]
|
||
|
|
```
|