2020-07-30 12:09:33 +00:00
---
stage: Create
group: Source Code
2021-12-28 21:16:11 +00:00
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
2020-07-30 12:09:33 +00:00
---
2021-02-09 18:09:59 +00:00
# Project import/export API **(FREE)**
2018-02-13 14:04:19 +00:00
2020-01-31 09:08:53 +00:00
See also:
- [Project import/export documentation ](../user/project/settings/import_export.md ).
2021-01-28 06:08:59 +00:00
- [Project import/export administration Rake tasks ](../administration/raketasks/project_import_export.md ). ** (FREE SELF)**
2018-02-13 14:04:19 +00:00
2018-02-25 03:12:11 +00:00
## Schedule an export
2017-12-06 04:27:23 +00:00
Start a new export.
2021-05-25 18:10:42 +00:00
The endpoint also accepts an `upload` parameter. This parameter is a hash. It contains
2018-03-30 15:45:59 +00:00
all the necessary information to upload the exported project to a web server or
to any S3-compatible platform. At the moment we only support binary
data file uploads to the final server.
2021-12-28 21:16:11 +00:00
The `upload[url]` parameter is required if the `upload` parameter is present.
2018-03-30 15:45:59 +00:00
2020-05-20 03:08:04 +00:00
```plaintext
2017-12-06 04:27:23 +00:00
POST /projects/:id/export
```
| Attribute | Type | Required | Description |
| --------- | -------------- | -------- | ---------------------------------------- |
2021-06-28 15:08:03 +00:00
| `id` | integer/string | yes | The ID or [URL-encoded path of the project ](index.md#namespaced-path-encoding ) owned by the authenticated user |
2018-03-14 14:17:35 +00:00
| `description` | string | no | Overrides the project description |
2018-03-30 15:45:59 +00:00
| `upload` | hash | no | Hash that contains the information to upload the exported project to a web server |
| `upload[url]` | string | yes | The URL to upload the project |
| `upload[http_method]` | string | no | The HTTP method to upload the exported project. Only `PUT` and `POST` methods allowed. Default is `PUT` |
2017-12-06 04:27:23 +00:00
2020-01-30 15:09:15 +00:00
```shell
2020-05-28 00:08:37 +00:00
curl --request POST --header "PRIVATE-TOKEN: < your_access_token > " "https://gitlab.example.com/api/v4/projects/1/export" \
2018-07-10 13:55:56 +00:00
--data "upload[http_method]=PUT" \
--data-urlencode "upload[url]=https://example-bucket.s3.eu-west-3.amazonaws.com/backup?X-Amz-Algorithm=AWS4-HMAC-SHA256& X-Amz-Credential=AKIAIMBJHN2O62W8IELQ%2F20180312%2Feu-west-3%2Fs3%2Faws4_request& X-Amz-Date=20180312T110328Z& X-Amz-Expires=900& X-Amz-SignedHeaders=host& X-Amz-Signature=8413facb20ff33a49a147a0b4abcff4c8487cc33ee1f7e450c46e8f695569dbd"
```
2017-12-06 04:27:23 +00:00
```json
{
"message": "202 Accepted"
}
```
2020-12-04 21:09:29 +00:00
NOTE:
2021-03-08 18:09:12 +00:00
The upload request is sent with `Content-Type: application/gzip` header. Ensure that your pre-signed URL includes this as part of the signature.
2020-05-10 12:10:05 +00:00
2017-12-06 04:27:23 +00:00
## Export status
Get the status of export.
2020-05-20 03:08:04 +00:00
```plaintext
2017-12-06 04:27:23 +00:00
GET /projects/:id/export
```
| Attribute | Type | Required | Description |
| --------- | -------------- | -------- | ---------------------------------------- |
2021-06-28 15:08:03 +00:00
| `id` | integer/string | yes | The ID or [URL-encoded path of the project ](index.md#namespaced-path-encoding ) owned by the authenticated user |
2017-12-06 04:27:23 +00:00
2020-01-30 15:09:15 +00:00
```shell
2020-05-28 00:08:37 +00:00
curl --header "PRIVATE-TOKEN: < your_access_token > " "https://gitlab.example.com/api/v4/projects/1/export"
2017-12-06 04:27:23 +00:00
```
2020-01-31 09:08:53 +00:00
Status can be one of:
2021-05-25 18:10:42 +00:00
- `none` : No exports _queued_ , _started_ , _finished_ , or _being regenerated_ .
- `queued` : The request for export is received, and is in the queue to be processed.
- `started` : The export process has started and is in progress. It includes:
- The process of exporting.
- Actions performed on the resulting file, such as sending an email notifying
the user to download the file, or uploading the exported file to a web server.
- `finished` : After the export process has completed and the user has been notified.
- `regeneration_in_progress` : An export file is available to download, and a request to generate a new export is in process.
2020-09-14 09:09:34 +00:00
2018-03-01 03:30:47 +00:00
`_links` are only present when export has finished.
2020-08-20 18:10:16 +00:00
`created_at` is the project create timestamp, not the export start time.
2017-12-06 04:27:23 +00:00
```json
{
"id": 1,
"description": "Itaque perspiciatis minima aspernatur corporis consequatur.",
"name": "Gitlab Test",
"name_with_namespace": "Gitlab Org / Gitlab Test",
"path": "gitlab-test",
"path_with_namespace": "gitlab-org/gitlab-test",
"created_at": "2017-08-29T04:36:44.383Z",
"export_status": "finished",
"_links": {
"api_url": "https://gitlab.example.com/api/v4/projects/1/export/download",
2021-04-29 12:09:58 +00:00
"web_url": "https://gitlab.example.com/gitlab-org/gitlab-test/download_export"
2017-12-06 04:27:23 +00:00
}
}
```
## Export download
Download the finished export.
2020-05-20 03:08:04 +00:00
```plaintext
2018-02-25 03:12:11 +00:00
GET /projects/:id/export/download
2017-12-06 04:27:23 +00:00
```
| Attribute | Type | Required | Description |
| --------- | -------------- | -------- | ---------------------------------------- |
2021-06-28 15:08:03 +00:00
| `id` | integer/string | yes | The ID or [URL-encoded path of the project ](index.md#namespaced-path-encoding ) owned by the authenticated user |
2017-12-06 04:27:23 +00:00
2020-01-30 15:09:15 +00:00
```shell
2021-06-02 15:09:59 +00:00
curl --header "PRIVATE-TOKEN: < your_access_token > " --remote-header-name \
--remote-name "https://gitlab.example.com/api/v4/projects/5/export/download"
2017-12-06 04:27:23 +00:00
```
2020-01-30 15:09:15 +00:00
```shell
2017-12-06 04:27:23 +00:00
ls *export.tar.gz
2017-12-05_22-11-148_namespace_project_export.tar.gz
```
2018-02-13 14:04:19 +00:00
## Import a file
2020-05-20 03:08:04 +00:00
```plaintext
2018-02-13 14:04:19 +00:00
POST /projects/import
```
| Attribute | Type | Required | Description |
| --------- | -------------- | -------- | ---------------------------------------- |
2021-03-08 18:09:12 +00:00
| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace |
2019-09-20 00:05:59 +00:00
| `name` | string | no | The name of the project to be imported. Defaults to the path of the project if not provided |
2018-02-13 14:04:19 +00:00
| `file` | string | yes | The file to be uploaded |
| `path` | string | yes | Name and path for new project |
2021-03-08 18:09:12 +00:00
| `overwrite` | boolean | no | If there is a project with the same path the import overwrites it. Default to false |
2018-04-23 10:29:53 +00:00
| `override_params` | Hash | no | Supports all fields defined in the [Project API ](projects.md ) |
2018-03-29 13:08:31 +00:00
2021-03-08 18:09:12 +00:00
The override parameters passed take precedence over all values defined inside the export file.
2018-02-13 14:04:19 +00:00
2018-04-23 10:29:53 +00:00
To upload a file from your file system, use the `--form` argument. This causes
2018-02-13 14:04:19 +00:00
cURL to post data using the header `Content-Type: multipart/form-data` .
2018-04-23 10:29:53 +00:00
The `file=` parameter must point to a file on your file system and be preceded
2018-02-13 14:04:19 +00:00
by `@` . For example:
2020-01-30 15:09:15 +00:00
```shell
2021-06-02 15:09:59 +00:00
curl --request POST --header "PRIVATE-TOKEN: < your_access_token > " --form "path=api-project" \
--form "file=@/path/to/file" "https://gitlab.example.com/api/v4/projects/import"
2018-02-13 14:04:19 +00:00
```
2018-07-10 13:55:56 +00:00
cURL doesn't support posting a file from a remote server. Importing a project from a remote server can be accomplished through something like the following:
```python
import requests
2019-09-20 00:05:59 +00:00
from io import BytesIO
2018-07-10 13:55:56 +00:00
2019-09-20 00:05:59 +00:00
s3_file = requests.get(presigned_url)
2018-07-10 13:55:56 +00:00
url = 'https://gitlab.example.com/api/v4/projects/import'
2019-09-20 06:06:19 +00:00
files = {'file': ('file.tar.gz', BytesIO(s3_file.content))}
2018-07-10 13:55:56 +00:00
data = {
"path": "example-project",
"namespace": "example-group"
}
headers = {
2018-12-27 09:03:08 +00:00
'Private-Token': "< your_access_token > "
2018-07-10 13:55:56 +00:00
}
requests.post(url, headers=headers, data=data, files=files)
```
2018-02-13 14:04:19 +00:00
```json
{
"id": 1,
"description": null,
"name": "api-project",
"name_with_namespace": "Administrator / api-project",
"path": "api-project",
"path_with_namespace": "root/api-project",
"created_at": "2018-02-13T09:05:58.023Z",
2020-04-08 18:09:16 +00:00
"import_status": "scheduled",
2020-04-21 15:21:10 +00:00
"correlation_id": "mezklWso3Za",
"failed_relations": []
2018-02-13 14:04:19 +00:00
}
```
2020-12-04 21:09:29 +00:00
NOTE:
2021-01-13 21:11:00 +00:00
The maximum import file size can be set by the Administrator, default is `0` (unlimited)..
2021-01-20 00:10:39 +00:00
As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API ](settings.md#change-application-settings ) or the [Admin UI ](../user/admin_area/settings/account_and_limit_settings.md ). Default [modified ](https://gitlab.com/gitlab-org/gitlab/-/issues/251106 ) from 50MB to 0 in GitLab 13.8.
2020-06-09 15:08:05 +00:00
2021-05-14 18:10:34 +00:00
## Import a file from a remote object storage
2022-02-03 15:12:41 +00:00
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282503) in GitLab 13.12 in [Beta](../policy/alpha-beta-support.md#beta-features).
2021-05-14 18:10:34 +00:00
2022-02-02 21:16:54 +00:00
This endpoint is behind a feature flag that is enabled by default.
2021-05-14 18:10:34 +00:00
To enable this endpoint:
```ruby
Feature.enable(:import_project_from_remote_file)
```
To disable this endpoint:
```ruby
Feature.disable(:import_project_from_remote_file)
```
```plaintext
POST /projects/remote-import
```
| Attribute | Type | Required | Description |
| ----------------- | -------------- | -------- | ---------------------------------------- |
| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. |
| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. |
| `url` | string | yes | URL for the file to import. |
| `path` | string | yes | Name and path for the new project. |
| `overwrite` | boolean | no | Whether to overwrite a project with the same path when importing. Defaults to `false` . |
| `override_params` | Hash | no | Supports all fields defined in the [Project API ](projects.md ). |
The passed override parameters take precedence over all values defined in the export file.
```shell
curl --request POST \
--header "PRIVATE-TOKEN: < your_access_token > " \
2021-11-30 15:14:19 +00:00
--header "Content-Type: application/json" \
2021-05-14 18:10:34 +00:00
--url "https://gitlab.example.com/api/v4/projects/remote-import" \
--data '{"url":"https://remoteobject/file?token=123123","path":"remote-project"}'
```
```json
{
"id": 1,
"description": null,
"name": "remote-project",
"name_with_namespace": "Administrator / remote-project",
"path": "remote-project",
"path_with_namespace": "root/remote-project",
"created_at": "2018-02-13T09:05:58.023Z",
"import_status": "scheduled",
"correlation_id": "mezklWso3Za",
"failed_relations": [],
"import_error": null
}
```
2022-02-07 15:15:53 +00:00
The `Content-Length` header must return a valid number. The maximum file size is 10 gigabytes.
The `Content-Type` header must be `application/gzip` .
2021-05-14 18:10:34 +00:00
2022-03-07 06:15:33 +00:00
## Import a file from AWS S3
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348874) in GitLab 14.9 in [Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta), [with a flag](../administration/feature_flags.md) named `import_project_from_remote_file_s3`. Disabled by default.
FLAG:
On self-managed GitLab and GitLab.com, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag ](../administration/feature_flags.md ) named `import_project_from_remote_file_s3` . This feature is not ready for production use.
```plaintext
POST /projects/remote-import-s3
```
| Attribute | Type | Required | Description |
| ------------------- | -------------- | -------- | ---------------------------------------- |
| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. |
| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. |
| `region` | string | yes | [AWS S3 region name where the file is stored. ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html#Regions ) |
| `bucket_name` | string | yes | [AWS S3 bucket name where the file is stored. ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html ) |
| `file_key` | string | yes | [AWS S3 file key to identify the file. ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingObjects.html ) |
| `access_key_id` | string | yes | [AWS S3 access key ID. ](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys ). |
| `secret_access_key` | string | yes | [AWS S3 secret access key. ](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys ) |
The passed override parameters take precedence over all values defined in the export file.
```shell
curl --request POST \
--url "http://localhost:3000/api/v4/projects/remote-import-s3" \
--header "PRIVATE-TOKEN: < your gitlab access key > " \
--header 'Content-Type: application/json' \
--data '{
"name": "Sample Project",
"path": "sample-project",
"region": "< Your S3 region name > ",
"bucket_name": "< Your S3 bucket name > ",
"file_key": "< Your S3 file key > ",
"access_key_id": "< Your AWS access key id > ",
"secret_access_key": "< Your AWS secret access key > "
}'
```
```json
{
"id": 1,
"description": null,
"name": "Sample project",
"name_with_namespace": "Administrator / sample-project",
"path": "sample-project",
"path_with_namespace": "root/sample-project",
"created_at": "2018-02-13T09:05:58.023Z",
"import_status": "scheduled",
"correlation_id": "mezklWso3Za",
"failed_relations": [],
"import_error": null
}
```
2018-02-13 14:04:19 +00:00
## Import status
2018-02-13 14:12:18 +00:00
Get the status of an import.
2018-02-13 14:04:19 +00:00
2020-05-20 03:08:04 +00:00
```plaintext
2018-02-13 14:04:19 +00:00
GET /projects/:id/import
```
| Attribute | Type | Required | Description |
| --------- | -------------- | -------- | ---------------------------------------- |
2021-06-28 15:08:03 +00:00
| `id` | integer/string | yes | The ID or [URL-encoded path of the project ](index.md#namespaced-path-encoding ) owned by the authenticated user |
2018-02-13 14:04:19 +00:00
2020-01-30 15:09:15 +00:00
```shell
2020-05-28 00:08:37 +00:00
curl --header "PRIVATE-TOKEN: < your_access_token > " "https://gitlab.example.com/api/v4/projects/1/import"
2018-02-13 14:04:19 +00:00
```
2020-01-31 09:08:53 +00:00
Status can be one of:
- `none`
- `scheduled`
- `failed`
- `started`
- `finished`
2018-02-13 14:12:18 +00:00
2021-03-08 18:09:12 +00:00
If the status is `failed` , it includes the import error message under `import_error` .
2020-04-21 15:21:10 +00:00
If the status is `failed` , `started` or `finished` , the `failed_relations` array might
be populated with any occurrences of relations that failed to import either due to
unrecoverable errors or because retries were exhausted (a typical example are query timeouts.)
2020-12-04 21:09:29 +00:00
NOTE:
2020-04-21 15:21:10 +00:00
An element's `id` field in `failed_relations` references the failure record, not the relation.
2020-12-04 21:09:29 +00:00
NOTE:
2021-05-25 18:10:42 +00:00
The `failed_relations` array is capped to 100 items.
2018-02-13 14:04:19 +00:00
```json
{
"id": 1,
"description": "Itaque perspiciatis minima aspernatur corporis consequatur.",
"name": "Gitlab Test",
"name_with_namespace": "Gitlab Org / Gitlab Test",
"path": "gitlab-test",
"path_with_namespace": "gitlab-org/gitlab-test",
"created_at": "2017-08-29T04:36:44.383Z",
2020-04-08 18:09:16 +00:00
"import_status": "started",
2021-12-10 21:11:20 +00:00
"import_type": "github",
2020-04-21 15:21:10 +00:00
"correlation_id": "mezklWso3Za",
"failed_relations": [
{
"id": 42,
"created_at": "2020-04-02T14:48:59.526Z",
"exception_class": "RuntimeError",
"exception_message": "A failure occurred",
"source": "custom error context",
2021-12-10 21:11:20 +00:00
"relation_name": "merge_requests",
"line_number": 0
2020-04-21 15:21:10 +00:00
}
]
2018-02-13 14:04:19 +00:00
}
```
2021-12-10 21:11:20 +00:00
When importing from GitHub, the a `stats` field lists how many objects were already fetched from
GitHub and how many were already imported:
```json
{
"id": 1,
"description": "Itaque perspiciatis minima aspernatur corporis consequatur.",
"name": "Gitlab Test",
"name_with_namespace": "Gitlab Org / Gitlab Test",
"path": "gitlab-test",
"path_with_namespace": "gitlab-org/gitlab-test",
"created_at": "2017-08-29T04:36:44.383Z",
"import_status": "started",
"import_type": "github",
"correlation_id": "mezklWso3Za",
"failed_relations": [
{
"id": 42,
"created_at": "2020-04-02T14:48:59.526Z",
"exception_class": "RuntimeError",
"exception_message": "A failure occurred",
"source": "custom error context",
"relation_name": "merge_requests",
"line_number": 0
}
],
"stats": {
"fetched": {
"diff_note": 19,
"issue": 3,
"label": 1,
"note": 3,
"pull_request": 2,
"pull_request_merged_by": 1,
"pull_request_review": 16
},
"imported": {
"diff_note": 19,
"issue": 3,
"label": 1,
"note": 3,
"pull_request": 2,
"pull_request_merged_by": 1,
"pull_request_review": 16
}
}
}
```