Document all the available options for Dependency Scanning
Port all info from: - security-products/dependency-scanning/blob/master/docs/README.md - security-products/dependency-scanning/blob/master/docs/analyzers.md
This commit is contained in:
Achilleas Pipinellis
parent
4ec1720fdb
commit
60cd12a6d7
2 changed files with 174 additions and 13 deletions
133
doc/user/application_security/dependency_scanning/analyzers.md
Normal file
133
doc/user/application_security/dependency_scanning/analyzers.md
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
# Dependency Scanning Analyzers **[ULTIMATE]**
|
||||
|
||||
Dependency Scanning relies on underlying third party tools that are wrapped into
|
||||
what we call "Analyzers". An analyzer is a
|
||||
[dedicated project](https://gitlab.com/gitlab-org/security-products/analyzers)
|
||||
that wraps a particular tool to:
|
||||
|
||||
- Expose its detection logic.
|
||||
- Handle its execution.
|
||||
- Convert its output to the common format.
|
||||
|
||||
This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/security-products/analyzers/common).
|
||||
|
||||
Dependency Scanning supports the following official analyzers:
|
||||
|
||||
- [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit)
|
||||
- [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium)
|
||||
- [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven)
|
||||
- [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python)
|
||||
- [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js)
|
||||
|
||||
The analyzers are published as Docker images that Dependency Scanning will use
|
||||
to launch dedicated containers for each analysis.
|
||||
|
||||
Dependency Scanning is pre-configured with a set of **default images** that are
|
||||
maintained by GitLab, but users can also integrate their own **custom images**.
|
||||
|
||||
## Official default analyzers
|
||||
|
||||
Any custom change to the official analyzers can be achieved by using an
|
||||
[environment variable in your `.gitlab-ci.yml`](index.md#customizing-the-dependency-scanning-settings).
|
||||
|
||||
### Using a custom Docker mirror
|
||||
|
||||
You can switch to a custom Docker registry that provides the official analyzer
|
||||
images under a different prefix. For instance, the following instructs Dependency
|
||||
Scanning to pull `my-docker-registry/gl-images/gemnasium`
|
||||
instead of `registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium`.
|
||||
In `.gitlab-ci.yml` define:
|
||||
|
||||
```yaml
|
||||
include:
|
||||
template: Dependency-Scanning.gitlab-ci.yml
|
||||
|
||||
variables:
|
||||
DS_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images
|
||||
```
|
||||
|
||||
This configuration requires that your custom registry provides images for all
|
||||
the official analyzers.
|
||||
|
||||
### Selecting specific analyzers
|
||||
|
||||
You can select the official analyzers you want to run. Here's how to enable
|
||||
`bundler-audit` and `gemnasium` while disabling all the other default ones.
|
||||
In `.gitlab-ci.yml` define:
|
||||
|
||||
```yaml
|
||||
include:
|
||||
template: Dependency-Scanning.gitlab-ci.yml
|
||||
|
||||
variables:
|
||||
DS_DEFAULT_ANALYZERS: "bundler-audit,gemnasium"
|
||||
```
|
||||
|
||||
`bundler-audit` runs first. When merging the reports, Dependency Scanning will
|
||||
remove the duplicates and will keep the `bundler-audit` entries.
|
||||
|
||||
### Disabling default analyzers
|
||||
|
||||
Setting `DS_DEFAULT_ANALYZERS` to an empty string will disable all the official
|
||||
default analyzers. In `.gitlab-ci.yml` define:
|
||||
|
||||
```yaml
|
||||
include:
|
||||
template: Dependency-Scanning.gitlab-ci.yml
|
||||
|
||||
variables:
|
||||
DS_DEFAULT_ANALYZERS: ""
|
||||
```
|
||||
|
||||
That's needed when one totally relies on [custom analyzers](#custom-analyzers).
|
||||
|
||||
## Custom analyzers
|
||||
|
||||
You can provide your own analyzers as a comma separated list of Docker images.
|
||||
Here's how to add `analyzers/nugget` and `analyzers/perl` to the default images.
|
||||
In `.gitlab-ci.yml` define:
|
||||
|
||||
```yaml
|
||||
include:
|
||||
template: Dependency-Scanning.gitlab-ci.yml
|
||||
|
||||
variables:
|
||||
DS_ANALYZER_IMAGES: "my-docker-registry/analyzers/nugget,amy-docker-registry/nalyzers/perl"
|
||||
```
|
||||
|
||||
The values must be the full path to the container registry images,
|
||||
like what you would feed to the `docker pull` command.
|
||||
|
||||
NOTE: **Note:**
|
||||
This configuration doesn't benefit from the integrated detection step. Dependency
|
||||
Scanning has to fetch and spawn each Docker image to establish whether the
|
||||
custom analyzer can scan the source code.
|
||||
|
||||
## Analyzers data
|
||||
|
||||
The following table lists the data available for each official analyzer.
|
||||
|
||||
| Property \ Tool | Gemnasium | bundler-audit | Retire.js |
|
||||
|---------------------------------------|:------------------:|:------------------:|:------------------:|
|
||||
| Severity | 𐄂 | ✓ | ✓ |
|
||||
| Title | ✓ | ✓ | ✓ |
|
||||
| File | ✓ | ⚠ | ✓ |
|
||||
| Start line | 𐄂 | 𐄂 | 𐄂 |
|
||||
| End line | 𐄂 | 𐄂 | 𐄂 |
|
||||
| External ID (e.g., CVE) | ✓ | ✓ | ⚠ |
|
||||
| URLs | ✓ | ✓ | ✓ |
|
||||
| Internal doc/explanation | ✓ | 𐄂 | 𐄂 |
|
||||
| Solution | ✓ | ✓ | 𐄂 |
|
||||
| Confidence | 𐄂 | 𐄂 | 𐄂 |
|
||||
| Affected item (e.g. class or package) | ✓ | ✓ | ✓ |
|
||||
| Source code extract | 𐄂 | 𐄂 | 𐄂 |
|
||||
| Internal ID | ✓ | 𐄂 | 𐄂 |
|
||||
| Date | ✓ | 𐄂 | 𐄂 |
|
||||
| Credits | ✓ | 𐄂 | 𐄂 |
|
||||
|
||||
- ✓ => we have that data
|
||||
- ⚠ => we have that data but it's partially reliable, or we need to extract that data from unstructured content
|
||||
- 𐄂 => we don't have that data or it would need to develop specific or inefficient/unreliable logic to obtain it.
|
||||
|
||||
The values provided by these tools are heterogeneous so they are sometimes
|
||||
normalized into common values (e.g., `severity`, `confidence`, etc).
|
||||
|
|
@ -54,9 +54,22 @@ The following languages and dependency managers are supported.
|
|||
| Java ([Maven](https://maven.apache.org/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general) |
|
||||
| PHP ([Composer](https://getcomposer.org/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general) |
|
||||
|
||||
Some scanners require to send a list of project dependencies to GitLab's central
|
||||
servers to check for vulnerabilities. To learn more about this or to disable it,
|
||||
refer to the [GitLab Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).
|
||||
## Remote checks
|
||||
|
||||
While some tools pull a local database to check vulnerabilities, some others
|
||||
like Gemnasium require sending data to GitLab central servers to analyze them:
|
||||
|
||||
1. Gemnasium scans the dependencies of your project locally and sends a list of
|
||||
packages to GitLab central servers.
|
||||
1. The servers return the list of known vulnerabilities for all versions of
|
||||
these packages.
|
||||
1. The client picks up the relevant vulnerabilities by comparing with the versions
|
||||
of the packages that are used by the project.
|
||||
|
||||
The Gemnasium client does **NOT** send the exact package versions your project relies on.
|
||||
|
||||
You can disable the remote checks by [using](#customizing-the-dependency-scanning-settings)
|
||||
the `DS_DISABLE_REMOTE_CHECKS` environment variable and setting it to `true`.
|
||||
|
||||
## Configuring Dependency Scanning
|
||||
|
||||
|
|
@ -97,17 +110,10 @@ The report will be saved as a
|
|||
that you can later download and analyze. Due to implementation limitations, we
|
||||
always take the latest Dependency Scanning artifact available.
|
||||
|
||||
Some security scanners require to send a list of project dependencies to GitLab
|
||||
central servers to check for vulnerabilities. To learn more about this or to
|
||||
disable it, check the
|
||||
[GitLab Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).
|
||||
|
||||
#### Customizing the Dependency Scanning settings
|
||||
|
||||
The Dependency Scanning settings can be changed through environment variables by using the
|
||||
The Dependency Scanning settings can be changed through [environment variables](#available-variables) by using the
|
||||
[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`.
|
||||
These variables are documented in the
|
||||
[Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#settings).
|
||||
|
||||
For example:
|
||||
|
||||
|
|
@ -116,7 +122,7 @@ include:
|
|||
template: Dependency-Scanning.gitlab-ci.yml
|
||||
|
||||
variables:
|
||||
DEP_SCAN_DISABLE_REMOTE_CHECKS: true
|
||||
DS_DISABLE_REMOTE_CHECKS: true
|
||||
```
|
||||
|
||||
Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline
|
||||
|
|
@ -137,6 +143,24 @@ dependency_scanning:
|
|||
CI_DEBUG_TRACE: "true"
|
||||
```
|
||||
|
||||
#### Available variables
|
||||
|
||||
Dependency Scanning can be [configured](#customizing-the-dependency-scanning-settings)
|
||||
using environment variables.
|
||||
|
||||
| Environment variable | Function |
|
||||
|-------------------------------- |----------|
|
||||
| `DS_ANALYZER_IMAGES` | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). |
|
||||
| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). |
|
||||
| `DS_ANALYZER_IMAGE_TAG` | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). |
|
||||
| `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). |
|
||||
| `DS_DISABLE_REMOTE_CHECKS` | Do not send any data to GitLab. Used in the [Gemnasium analyzer](#remote-checks). |
|
||||
| `DS_PULL_ANALYZER_IMAGES` | Pull the images from the Docker registry (set to `0` to disable). |
|
||||
| `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, file or folder paths. Parent directories will also match patterns. |
|
||||
| `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. |
|
||||
| `DS_PULL_ANALYZER_IMAGE_TIMEOUT` | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. |
|
||||
| `DS_RUN_ANALYZER_TIMEOUT` | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. |
|
||||
|
||||
### Manual job definition for GitLab 11.5 and later
|
||||
|
||||
For GitLab 11.5 and GitLab Runner 11.5 and later, the following `dependency_scanning`
|
||||
|
|
@ -171,7 +195,7 @@ dependency_scanning:
|
|||
dependency_scanning: gl-dependency-scanning-report.json
|
||||
```
|
||||
|
||||
You can supply many other [settings variables](https://gitlab.com/gitlab-org/security-products/dependency-scanning#settings)
|
||||
You can supply many other [settings variables](#available-variables)
|
||||
via `docker run --env` to customize your job execution.
|
||||
|
||||
### Manual job definition for GitLab 11.4 and earlier (deprecated)
|
||||
|
|
@ -388,6 +412,10 @@ supported by Gemnasium.
|
|||
|
||||
To see the generated dependency list, navigate to your project's **Project > Dependency List**.
|
||||
|
||||
## Versioning and release process
|
||||
|
||||
Please check the [Release Process documentation](https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md).
|
||||
|
||||
## Contributing to the vulnerability database
|
||||
|
||||
You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-products/gemnasium-db) project
|
||||
|
|
|
|||
Loading…
Reference in a new issue