--- stage: Manage group: Compliance info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Audit event streaming **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. Disabled by default. > - [Enabled on GitLab.com and by default on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338939) in GitLab 14.7. > - [Feature flag `ff_external_audit_events_namespace`](https://gitlab.com/gitlab-org/gitlab/-/issues/349588) removed in GitLab 14.8. Users can set an HTTP endpoint for a top-level group to receive all audit events about the group, its subgroups, and projects as structured JSON. Top-level group owners can manage their audit logs in third-party systems. Any service that can receive structured JSON data can be used as the endpoint. NOTE: GitLab can stream a single event more than once to the same destination. Use the `id` key in the payload to deduplicate incoming data. ## Add a new event streaming destination WARNING: Event streaming destinations receive **all** audit event data, which could include sensitive information. Make sure you trust the destination endpoint. ### Use the GitLab UI Users with at least the Owner role for a group can add event streaming destinations for it: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events** 1. On the main area, select **Streams** tab. - When the destination list is empty, select **Add stream** activate edit mode and add a new destination. - When the destination list is not empty, select **{plus}** under the **Streams** tab to activate edit mode. 1. Enter the endpoint you wish to add and select **Add**. Event streaming is enabled if: - No warning is shown. - The added endpoint is displayed in the UI. ### Use the API To enable event streaming and add a destination, users with at least the Owner role for a group must use the `externalAuditEventDestinationCreate` mutation in the GraphQL API. ```graphql mutation { externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", groupPath: "my-group" } ) { errors externalAuditEventDestination { destinationUrl verificationToken group { name } } } } ``` Event streaming is enabled if: - The returned `errors` object is empty. - The API responds with `200 OK`. ## List streaming destinations Users with at least the Owner role for a group can list event streaming destinations. ### Use the GitLab UI > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336411) in GitLab 14.9. Users with at least the Owner role for a group can list event streaming destinations: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events** 1. On the main area, select **Streams** tab. ### Use the API Users with at least the Owner role for a group can view a list of event streaming destinations at any time using the `externalAuditEventDestinations` query type. ```graphql query { group(fullPath: "my-group") { id externalAuditEventDestinations { nodes { destinationUrl verificationToken id } } } } ``` If the resulting list is empty, then audit event streaming is not enabled for that group. ## Delete streaming destinations Users with at least the Owner role for a group can delete event streaming destinations using the `deleteAuditEventDestinations` mutation type. When the last destination is successfully deleted, event streaming is disabled for the group. ### Use the GitLab UI > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336411) in GitLab 14.9. Users with at least the Owner role for a group can delete event streaming destinations. 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Security & Compliance > Audit events** 1. On the main area, select **Streams** tab. 1. Select **{remove}** at the right side of each item. The external streaming destination is deleted when: - No warning is shown. - The deleted endpoint is not displayed in the UI. ### Use the API Delete an event streaming destination by specifying an ID. Get the required ID by [listing the details](audit_event_streaming.md#use-the-api-1) of event streaming destinations. ```graphql mutation { externalAuditEventDestinationDestroy(input: { id: destination }) { errors } } ``` Destination is deleted if: - The returned `errors` object is empty. - The API responds with `200 OK`. ## Custom HTTP header values > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361216) in GitLab 15.1 [with a flag](feature_flags.md) named `streaming_audit_event_headers`. Disabled by default. FLAG: On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `streaming_audit_event_headers`. On GitLab.com, this feature is not available. The feature is not ready for production use. Each streaming destination can have up to 20 custom HTTP headers included with each streamed event. ### Add with the API Group owners can add a HTTP header using the GraphQL `auditEventsStreamingHeadersCreate` mutation. ```graphql mutation { auditEventsStreamingHeadersCreate(input: { destinationId: "gid://gitlab/AuditEvents::ExternalAuditEventDestination/24601", key: "foo", value: "bar" }) { errors } } ``` The header is created if the returned `errors` object is empty. ## Verify event authenticity > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8. Each streaming destination has a unique verification token (`verificationToken`) that can be used to verify the authenticity of the event. This token is generated when the event destination is created and cannot be changed. Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against the destination's value when [listing streaming destinations](#list-streaming-destinations). ## Audit event streaming on Git operations > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `audit_event_streaming_git_operations`. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.0. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.1 by default. FLAG: On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](feature_flags.md) named `audit_event_streaming_git_operations`. Streaming audit events can be sent when signed-in users push or pull a project's remote Git repositories: - [Using SSH](../user/ssh.md). - Using HTTP or HTTPS. - Using the **Download** button (**{download}**) in GitLab UI. Audit events are not captured for users that are not signed in. For example, when downloading a public project. To configure streaming audit events for Git operations, see [Add a new event streaming destination](#add-a-new-event-streaming-destination). ### Headers > `X-Gitlab-Audit-Event-Type` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86881) in GitLab 15.0. Headers are formatted as follows: ```plaintext POST /logs HTTP/1.1 Host: Content-Type: application/x-www-form-urlencoded X-Gitlab-Event-Streaming-Token: X-Gitlab-Audit-Event-Type: repository_git_operation ``` ### Example payloads for SSH events Fetch: ```json { "id": 1, "author_id": 1, "entity_id": 29, "entity_type": "Project", "details": { "author_name": "Administrator", "target_id": 29, "target_type": "Project", "target_details": "example-project", "custom_message": { "protocol": "ssh", "action": "git-upload-pack" }, "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "Administrator", "entity_path": "example-group/example-project", "target_details": "example-project", "created_at": "2022-02-23T06:21:05.283Z", "target_type": "Project", "target_id": 29, "event_type": "repository_git_operation" } ``` Push: ```json { "id": 1, "author_id": 1, "entity_id": 29, "entity_type": "Project", "details": { "author_name": "Administrator", "target_id": 29, "target_type": "Project", "target_details": "example-project", "custom_message": { "protocol": "ssh", "action": "git-receive-pack" }, "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "Administrator", "entity_path": "example-group/example-project", "target_details": "example-project", "created_at": "2022-02-23T06:23:08.746Z", "target_type": "Project", "target_id": 29, "event_type": "repository_git_operation" } ``` ### Example payloads for HTTP and HTTPS events Fetch: ```json { "id": 1, "author_id": 1, "entity_id": 29, "entity_type": "Project", "details": { "author_name": "Administrator", "target_id": 29, "target_type": "Project", "target_details": "example-project", "custom_message": { "protocol": "http", "action": "git-upload-pack" }, "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "Administrator", "entity_path": "example-group/example-project", "target_details": "example-project", "created_at": "2022-02-23T06:25:43.938Z", "target_type": "Project", "target_id": 29, "event_type": "repository_git_operation" } ``` Push: ```json { "id": 1, "author_id": 1, "entity_id": 29, "entity_type": "Project", "details": { "author_name": "Administrator", "target_id": 29, "target_type": "Project", "target_details": "example-project", "custom_message": { "protocol": "http", "action": "git-receive-pack" }, "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "Administrator", "entity_path": "example-group/example-project", "target_details": "example-project", "created_at": "2022-02-23T06:26:29.294Z", "target_type": "Project", "target_id": 29, "event_type": "repository_git_operation" } ``` ### Example payloads for events from GitLab UI download button Fetch: ```json { "id": 1, "author_id": 99, "entity_id": 29, "entity_type": "Project", "details": { "custom_message": "Repository Download Started", "author_name": "example_username", "target_id": 29, "target_type": "Project", "target_details": "example-group/example-project", "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "example_username", "entity_path": "example-group/example-project", "target_details": "example-group/example-project", "created_at": "2022-02-23T06:27:17.873Z", "target_type": "Project", "target_id": 29, "event_type": "repository_git_operation" } ``` ## Audit event streaming on merge request approval actions > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271162) in GitLab 14.9. Stream audit events that relate to merge approval actions performed within a project. ### Headers Headers are formatted as follows: ```plaintext POST /logs HTTP/1.1 Host: Content-Type: application/x-www-form-urlencoded X-Gitlab-Event-Streaming-Token: X-Gitlab-Audit-Event-Type: audit_operation ``` ### Example payload ```json { "id": 1, "author_id": 1, "entity_id": 6, "entity_type": "Project", "details": { "author_name": "example_username", "target_id": 20, "target_type": "MergeRequest", "target_details": "merge request title", "custom_message": "Approved merge request", "ip_address": "127.0.0.1", "entity_path": "example-group/example-project" }, "ip_address": "127.0.0.1", "author_name": "example_username", "entity_path": "example-group/example-project", "target_details": "merge request title", "created_at": "2022-03-09T06:53:11.181Z", "target_type": "MergeRequest", "target_id": 20, "event_type": "audit_operation" } ```