2021-11-09 03:42:22 +00:00
---
stage: Manage
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Audit event streaming **(ULTIMATE)**
2022-01-04 15:15:09 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. Disabled by default.
2022-01-17 00:14:49 +00:00
> - [Enabled on GitLab.com and by default on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338939) in GitLab 14.7.
2022-01-26 21:16:58 +00:00
> - [Feature flag `ff_external_audit_events_namespace`](https://gitlab.com/gitlab-org/gitlab/-/issues/349588) removed in GitLab 14.8.
2021-11-09 03:42:22 +00:00
2022-05-24 00:08:13 +00:00
Users can set an HTTP endpoint for a top-level group to receive all audit events about the group, its subgroups, and
2022-06-01 00:08:20 +00:00
projects as structured JSON.
2021-11-09 03:42:22 +00:00
2022-05-31 00:09:17 +00:00
Top-level group owners can manage their audit logs in third-party systems. Any service that can receive
2021-11-09 03:42:22 +00:00
structured JSON data can be used as the endpoint.
NOTE:
GitLab can stream a single event more than once to the same destination. Use the `id` key in the payload to deduplicate incoming data.
## Add a new event streaming destination
WARNING:
2022-04-12 12:08:50 +00:00
Event streaming destinations receive **all** audit event data, which could include sensitive information. Make sure you trust the destination endpoint.
2021-11-09 03:42:22 +00:00
2022-04-12 12:08:50 +00:00
### Use the GitLab UI
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
Users with at least the Owner role for a group can add event streaming destinations for it:
2022-03-16 12:07:24 +00:00
1. On the top bar, select **Menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**
1. On the main area, select **Streams** tab.
- When the destination list is empty, select **Add stream** activate edit mode and add a new destination.
- When the destination list is not empty, select ** {plus}** under the **Streams** tab to activate edit mode.
1. Enter the endpoint you wish to add and select **Add** .
Event streaming is enabled if:
- No warning is shown.
- The added endpoint is displayed in the UI.
2022-04-12 12:08:50 +00:00
### Use the API
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
To enable event streaming and add a destination, users with at least the Owner role for a group must use the
`externalAuditEventDestinationCreate` mutation in the GraphQL API.
2021-11-09 03:42:22 +00:00
```graphql
mutation {
externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", groupPath: "my-group" } ) {
errors
externalAuditEventDestination {
destinationUrl
2022-01-20 21:14:18 +00:00
verificationToken
2022-03-18 03:07:42 +00:00
group {
2021-11-09 03:42:22 +00:00
name
}
}
}
}
```
Event streaming is enabled if:
- The returned `errors` object is empty.
- The API responds with `200 OK` .
2022-04-12 12:08:50 +00:00
## List streaming destinations
2021-11-09 03:42:22 +00:00
2022-04-12 12:08:50 +00:00
Users with at least the Owner role for a group can list event streaming destinations.
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
### Use the GitLab UI
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336411) in GitLab 14.9.
Users with at least the Owner role for a group can list event streaming destinations:
2022-03-16 12:07:24 +00:00
1. On the top bar, select **Menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**
1. On the main area, select **Streams** tab.
2022-04-12 12:08:50 +00:00
### Use the API
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
Users with at least the Owner role for a group can view a list of event streaming destinations at any time using the
`externalAuditEventDestinations` query type.
2021-11-09 03:42:22 +00:00
```graphql
query {
group(fullPath: "my-group") {
id
externalAuditEventDestinations {
nodes {
destinationUrl
2022-01-20 21:14:18 +00:00
verificationToken
2021-11-09 03:42:22 +00:00
id
}
}
}
}
```
If the resulting list is empty, then audit event streaming is not enabled for that group.
2022-01-20 21:14:18 +00:00
2022-04-12 12:08:50 +00:00
## Delete streaming destinations
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
Users with at least the Owner role for a group can delete event streaming destinations using the
`deleteAuditEventDestinations` mutation type.
When the last destination is successfully deleted, event streaming is disabled for the group.
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
### Use the GitLab UI
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336411) in GitLab 14.9.
Users with at least the Owner role for a group can delete event streaming destinations.
2022-03-16 12:07:24 +00:00
1. On the top bar, select **Menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**
1. On the main area, select **Streams** tab.
1. Select ** {remove}** at the right side of each item.
2022-04-12 12:08:50 +00:00
The external streaming destination is deleted when:
2022-03-16 12:07:24 +00:00
- No warning is shown.
- The deleted endpoint is not displayed in the UI.
2022-04-12 12:08:50 +00:00
### Use the API
2022-03-16 12:07:24 +00:00
2022-04-12 12:08:50 +00:00
Delete an event streaming destination by specifying an ID. Get the required ID by
[listing the details ](audit_event_streaming.md#use-the-api-1 ) of event
streaming destinations.
2022-03-16 12:07:24 +00:00
```graphql
2022-05-20 12:08:50 +00:00
mutation {
2022-03-16 12:07:24 +00:00
externalAuditEventDestinationDestroy(input: { id: destination }) {
errors
}
}
```
Destination is deleted if:
- The returned `errors` object is empty.
- The API responds with `200 OK` .
2022-06-14 12:08:53 +00:00
## Custom HTTP header values
2022-06-28 00:09:28 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361216) in GitLab 15.1 [with a flag](feature_flags.md) named `streaming_audit_event_headers`. Disabled by default.
2022-06-29 09:09:41 +00:00
> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/362941) in GitLab 15.2.
2022-06-14 12:08:53 +00:00
FLAG:
2022-06-29 09:09:41 +00:00
On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag ](../administration/feature_flags.md ) named `streaming_audit_event_headers` .
2022-06-28 00:09:28 +00:00
On GitLab.com, this feature is available.
2022-06-14 12:08:53 +00:00
Each streaming destination can have up to 20 custom HTTP headers included with each streamed event.
### Add with the API
2022-06-28 03:09:38 +00:00
Group owners can add a HTTP header using the GraphQL `auditEventsStreamingHeadersCreate` mutation. You can retrieve the destination ID
by [listing the external audit destinations ](#list-streaming-destinations ) on the group.
2022-06-14 12:08:53 +00:00
```graphql
mutation {
auditEventsStreamingHeadersCreate(input: { destinationId: "gid://gitlab/AuditEvents::ExternalAuditEventDestination/24601", key: "foo", value: "bar" }) {
errors
}
}
```
2022-06-28 03:09:38 +00:00
The header is created if the returned `errors` object is empty.
2022-06-30 03:08:59 +00:00
### Update with the API
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361964) in GitLab 15.2.
Group owners can update a HTTP header using the GraphQL `auditEventsStreamingHeadersCreate` mutation.
```graphql
mutation {
auditEventsStreamingHeadersUpdate(input: { headerId: "gid://gitlab/AuditEvents::Streaming::Header/24255", key: "new-foo", value: "new-bar" }) {
errors
}
}
```
2022-06-17 06:08:15 +00:00
### Delete with the API
2022-06-28 03:09:38 +00:00
Group owners can remove a HTTP header using the GraphQL `auditEventsStreamingHeadersDestroy` mutation. You can retrieve the header ID
by [listing all the custom headers ](#list-all-custom-headers-with-the-api ) on the group.
2022-06-17 06:08:15 +00:00
```graphql
mutation {
2022-06-28 03:09:38 +00:00
auditEventsStreamingHeadersDestroy(input: { headerId: "gid://gitlab/AuditEvents::Streaming::Header/1" }) {
2022-06-17 06:08:15 +00:00
errors
}
}
```
2022-06-28 03:09:38 +00:00
The header is deleted if the returned `errors` object is empty.
### List all custom headers with the API
You can list all custom headers for a top-level group as well as their value and ID using the GraphQL `externalAuditEventDestinations` query. The ID
value returned by this query is what you need to pass to the `deletion` mutation.
```graphql
query {
group(fullPath: "your-group") {
id
externalAuditEventDestinations {
nodes {
destinationUrl
id
headers {
nodes {
key
value
id
}
}
}
}
}
}
```
2022-06-14 12:08:53 +00:00
2022-01-20 21:14:18 +00:00
## Verify event authenticity
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8.
Each streaming destination has a unique verification token (`verificationToken`) that can be used to verify the authenticity of the event. This
token is generated when the event destination is created and cannot be changed.
Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against
2022-04-12 12:08:50 +00:00
the destination's value when [listing streaming destinations ](#list-streaming-destinations ).
2022-03-09 03:09:18 +00:00
## Audit event streaming on Git operations
2022-05-18 03:08:41 +00:00
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `audit_event_streaming_git_operations`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.0.
2022-05-23 18:08:14 +00:00
> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.1 by default.
2022-03-09 03:09:18 +00:00
FLAG:
2022-05-23 18:08:14 +00:00
On self-managed GitLab, by default this feature is available. To hide the
feature, ask an administrator to [disable the feature flag ](feature_flags.md ) named `audit_event_streaming_git_operations` .
2022-03-09 03:09:18 +00:00
Streaming audit events can be sent when signed-in users push or pull a project's remote Git repositories:
2022-03-29 06:08:52 +00:00
- [Using SSH ](../user/ssh.md ).
2022-03-09 03:09:18 +00:00
- Using HTTP or HTTPS.
- Using the **Download** button (**{download}**) in GitLab UI.
Audit events are not captured for users that are not signed in. For example, when downloading a public project.
To configure streaming audit events for Git operations, see [Add a new event streaming destination ](#add-a-new-event-streaming-destination ).
2022-04-12 12:08:50 +00:00
### Headers
2022-03-09 03:09:18 +00:00
2022-05-31 06:09:17 +00:00
> `X-Gitlab-Audit-Event-Type` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86881) in GitLab 15.0.
2022-04-12 12:08:50 +00:00
Headers are formatted as follows:
2022-03-09 03:09:18 +00:00
```plaintext
POST /logs HTTP/1.1
Host: < DESTINATION_HOST >
Content-Type: application/x-www-form-urlencoded
X-Gitlab-Event-Streaming-Token: < DESTINATION_TOKEN >
2022-05-31 06:09:17 +00:00
X-Gitlab-Audit-Event-Type: repository_git_operation
2022-03-09 03:09:18 +00:00
```
2022-04-12 12:08:50 +00:00
### Example payloads for SSH events
2022-03-09 03:09:18 +00:00
Fetch:
```json
{
"id": 1,
"author_id": 1,
"entity_id": 29,
"entity_type": "Project",
"details": {
"author_name": "Administrator",
"target_id": 29,
"target_type": "Project",
"target_details": "example-project",
"custom_message": {
"protocol": "ssh",
"action": "git-upload-pack"
},
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "Administrator",
"entity_path": "example-group/example-project",
"target_details": "example-project",
"created_at": "2022-02-23T06:21:05.283Z",
"target_type": "Project",
2022-05-31 06:09:17 +00:00
"target_id": 29,
"event_type": "repository_git_operation"
2022-03-09 03:09:18 +00:00
}
```
Push:
```json
{
"id": 1,
"author_id": 1,
"entity_id": 29,
"entity_type": "Project",
"details": {
"author_name": "Administrator",
"target_id": 29,
"target_type": "Project",
"target_details": "example-project",
"custom_message": {
"protocol": "ssh",
"action": "git-receive-pack"
},
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "Administrator",
"entity_path": "example-group/example-project",
"target_details": "example-project",
"created_at": "2022-02-23T06:23:08.746Z",
"target_type": "Project",
2022-05-31 06:09:17 +00:00
"target_id": 29,
"event_type": "repository_git_operation"
2022-03-09 03:09:18 +00:00
}
```
2022-04-12 12:08:50 +00:00
### Example payloads for HTTP and HTTPS events
2022-03-09 03:09:18 +00:00
Fetch:
```json
{
"id": 1,
"author_id": 1,
"entity_id": 29,
"entity_type": "Project",
"details": {
"author_name": "Administrator",
"target_id": 29,
"target_type": "Project",
"target_details": "example-project",
"custom_message": {
"protocol": "http",
"action": "git-upload-pack"
},
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "Administrator",
"entity_path": "example-group/example-project",
"target_details": "example-project",
"created_at": "2022-02-23T06:25:43.938Z",
"target_type": "Project",
2022-05-31 06:09:17 +00:00
"target_id": 29,
"event_type": "repository_git_operation"
2022-03-09 03:09:18 +00:00
}
```
Push:
```json
{
"id": 1,
"author_id": 1,
"entity_id": 29,
"entity_type": "Project",
"details": {
"author_name": "Administrator",
"target_id": 29,
"target_type": "Project",
"target_details": "example-project",
"custom_message": {
"protocol": "http",
"action": "git-receive-pack"
},
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "Administrator",
"entity_path": "example-group/example-project",
"target_details": "example-project",
"created_at": "2022-02-23T06:26:29.294Z",
"target_type": "Project",
2022-05-31 06:09:17 +00:00
"target_id": 29,
"event_type": "repository_git_operation"
2022-03-09 03:09:18 +00:00
}
```
2022-04-12 12:08:50 +00:00
### Example payloads for events from GitLab UI download button
2022-03-09 03:09:18 +00:00
Fetch:
```json
{
"id": 1,
"author_id": 99,
"entity_id": 29,
"entity_type": "Project",
"details": {
"custom_message": "Repository Download Started",
"author_name": "example_username",
"target_id": 29,
"target_type": "Project",
"target_details": "example-group/example-project",
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "example_username",
"entity_path": "example-group/example-project",
"target_details": "example-group/example-project",
"created_at": "2022-02-23T06:27:17.873Z",
"target_type": "Project",
2022-05-31 06:09:17 +00:00
"target_id": 29,
"event_type": "repository_git_operation"
2022-03-09 03:09:18 +00:00
}
```
2022-03-15 12:07:44 +00:00
## Audit event streaming on merge request approval actions
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271162) in GitLab 14.9.
Stream audit events that relate to merge approval actions performed within a project.
2022-04-12 12:08:50 +00:00
### Headers
2022-03-15 12:07:44 +00:00
2022-04-12 12:08:50 +00:00
Headers are formatted as follows:
2022-03-15 12:07:44 +00:00
```plaintext
POST /logs HTTP/1.1
Host: < DESTINATION_HOST >
Content-Type: application/x-www-form-urlencoded
X-Gitlab-Event-Streaming-Token: < DESTINATION_TOKEN >
2022-05-31 06:09:17 +00:00
X-Gitlab-Audit-Event-Type: audit_operation
2022-03-15 12:07:44 +00:00
```
2022-04-12 12:08:50 +00:00
### Example payload
2022-03-15 12:07:44 +00:00
```json
{
"id": 1,
"author_id": 1,
"entity_id": 6,
"entity_type": "Project",
"details": {
"author_name": "example_username",
"target_id": 20,
"target_type": "MergeRequest",
"target_details": "merge request title",
"custom_message": "Approved merge request",
"ip_address": "127.0.0.1",
"entity_path": "example-group/example-project"
},
"ip_address": "127.0.0.1",
"author_name": "example_username",
"entity_path": "example-group/example-project",
"target_details": "merge request title",
"created_at": "2022-03-09T06:53:11.181Z",
"target_type": "MergeRequest",
2022-05-31 06:09:17 +00:00
"target_id": 20,
"event_type": "audit_operation"
2022-03-15 12:07:44 +00:00
}
```