1
0
Fork 0

Add Debian package registry (#22854)

Co-authored-by: @awkwardbunny

This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This commit is contained in:
KN4CK3R 2023-04-28 23:51:36 +02:00 committed by GitHub
parent bc4e06109d
commit bf77e2163b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
57 changed files with 1995 additions and 96 deletions

View file

@ -114,6 +114,11 @@
"path": "github.com/bits-and-blooms/bitset/LICENSE",
"licenseText": "Copyright (c) 2014 Will Fitzgerald. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are\nmet:\n\n * Redistributions of source code must retain the above copyright\nnotice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above\ncopyright notice, this list of conditions and the following disclaimer\nin the documentation and/or other materials provided with the\ndistribution.\n * Neither the name of Google Inc. nor the names of its\ncontributors may be used to endorse or promote products derived from\nthis software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n\"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\nA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\nOWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\nLIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\nDATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\nTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n"
},
{
"name": "github.com/blakesmith/ar",
"path": "github.com/blakesmith/ar/COPYING",
"licenseText": "Copyright (c) 2013 Blake Smith \u003cblakesmith0@gmail.com\u003e\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n\n"
},
{
"name": "github.com/blevesearch/bleve/v2",
"path": "github.com/blevesearch/bleve/v2/LICENSE",

View file

@ -25,7 +25,7 @@ func TestMigratePackages(t *testing.T) {
creator := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 1})
content := "package main\n\nfunc main() {\nfmt.Println(\"hi\")\n}\n"
buf, err := packages_module.CreateHashedBufferFromReader(strings.NewReader(content), 1024)
buf, err := packages_module.CreateHashedBufferFromReaderWithSize(strings.NewReader(content), 1024)
assert.NoError(t, err)
defer buf.Close()

View file

@ -2501,6 +2501,8 @@ ROUTER = console
;LIMIT_SIZE_CONDA = -1
;; Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_CONTAINER = -1
;; Maximum size of a Debian upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_DEBIAN = -1
;; Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
;LIMIT_SIZE_GENERIC = -1
;; Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)

View file

@ -1252,6 +1252,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
- `LIMIT_SIZE_CONAN`: **-1**: Maximum size of a Conan upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_CONDA`: **-1**: Maximum size of a Conda upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_CONTAINER`: **-1**: Maximum size of a Container upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_DEBIAN`: **-1**: Maximum size of a Debian upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_GENERIC`: **-1**: Maximum size of a Generic upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_HELM`: **-1**: Maximum size of a Helm upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)
- `LIMIT_SIZE_MAVEN`: **-1**: Maximum size of a Maven upload (`-1` means no limits, format `1000`, `1 MB`, `1 GiB`)

View file

@ -0,0 +1,134 @@
---
date: "2023-01-07T00:00:00+00:00"
title: "Debian Packages Repository"
slug: "packages/debian"
draft: false
toc: false
menu:
sidebar:
parent: "packages"
name: "Debian"
weight: 35
identifier: "debian"
---
# Debian Packages Repository
Publish [Debian](https://www.debian.org/distrib/packages) packages for your user or organization.
**Table of Contents**
{{< toc >}}
## Requirements
To work with the Debian registry, you need to use a HTTP client like `curl` to upload and a package manager like `apt` to consume packages.
The following examples use `apt`.
## Configuring the package registry
To register the Debian registry add the url to the list of known apt sources:
```shell
echo "deb https://gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list
```
| Placeholder | Description |
| -------------- | ----------- |
| `owner` | The owner of the package. |
| `distribution` | The distribution to use. |
| `component` | The component to use. |
If the registry is private, provide credentials in the url. You can use a password or a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}):
```shell
echo "deb https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list
```
The Debian registry files are signed with a PGP key which must be known to apt:
```shell
sudo curl https://gitea.example.com/api/packages/{owner}/debian/repository.key -o /etc/apt/trusted.gpg.d/gitea-{owner}.asc
```
Afterwards update the local package index:
```shell
apt update
```
## Publish a package
To publish a Debian package (`*.deb`), perform a HTTP PUT operation with the package content in the request body.
```
PUT https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/upload
```
| Parameter | Description |
| -------------- | ----------- |
| `owner` | The owner of the package. |
| `distribution` | The distribution may match the release name of the OS, ex: `bionic`. |
| `component` | The component can be used to group packages or just `main` or similar. |
Example request using HTTP Basic authentication:
```shell
curl --user your_username:your_password_or_token \
--upload-file path/to/file.deb \
https://gitea.example.com/api/packages/testuser/debian/pool/bionic/main/upload
```
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password.
You cannot publish a file with the same name twice to a package. You must delete the existing package version first.
The server reponds with the following HTTP Status codes.
| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `201 Created` | The package has been published. |
| `400 Bad Request` | The package name, version, distribution, component or architecture are invalid. |
| `409 Conflict` | A package file with the same combination of parameters exist already in the package. |
## Delete a package
To delete a Debian package perform a HTTP DELETE operation. This will delete the package version too if there is no file left.
```
DELETE https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/{package_name}/{package_version}/{architecture}
```
| Parameter | Description |
| ----------------- | ----------- |
| `owner` | The owner of the package. |
| `package_name` | The package name. |
| `package_version` | The package version. |
| `distribution` | The package distribution. |
| `component` | The package component. |
| `architecture` | The package architecture. |
Example request using HTTP Basic authentication:
```shell
curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/debian/pools/bionic/main/test-package/1.0.0/amd64
```
The server reponds with the following HTTP Status codes.
| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `204 No Content` | Success |
| `404 Not Found` | The package or file was not found. |
## Install a package
To install a package from the Debian registry, execute the following commands:
```shell
# use latest version
apt install {package_name}
# use specific version
apt install {package_name}={package_version}
```

View file

@ -33,6 +33,7 @@ The following package managers are currently supported:
| [Conan]({{< relref "doc/usage/packages/conan.en-us.md" >}}) | C++ | `conan` |
| [Conda]({{< relref "doc/usage/packages/conda.en-us.md" >}}) | - | `conda` |
| [Container]({{< relref "doc/usage/packages/container.en-us.md" >}}) | - | any OCI compliant client |
| [Debian]({{< relref "doc/usage/packages/debian.en-us.md" >}}) | - | `apt` |
| [Generic]({{< relref "doc/usage/packages/generic.en-us.md" >}}) | - | any HTTP client |
| [Helm]({{< relref "doc/usage/packages/helm.en-us.md" >}}) | - | any HTTP client, `cm-push` |
| [Maven]({{< relref "doc/usage/packages/maven.en-us.md" >}}) | Java | `mvn`, `gradle` |

3
go.mod
View file

@ -17,6 +17,7 @@ require (
github.com/NYTimes/gziphandler v1.1.1
github.com/PuerkitoBio/goquery v1.8.0
github.com/alecthomas/chroma/v2 v2.5.0
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb
github.com/blevesearch/bleve/v2 v2.3.6
github.com/bufbuild/connect-go v1.3.1
github.com/buildkite/terminal-to-html/v3 v3.7.0
@ -96,6 +97,7 @@ require (
github.com/stretchr/testify v1.8.1
github.com/syndtr/goleveldb v1.0.0
github.com/tstranex/u2f v1.0.0
github.com/ulikunitz/xz v0.5.11
github.com/urfave/cli v1.22.12
github.com/xanzy/go-gitlab v0.80.2
github.com/xeipuuv/gojsonschema v1.2.0
@ -260,7 +262,6 @@ require (
github.com/ssor/bom v0.0.0-20170718123548-6386211fdfcf // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/toqueteos/webbrowser v1.2.0 // indirect
github.com/ulikunitz/xz v0.5.11 // indirect
github.com/unknwon/com v1.0.1 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.44.0 // indirect

2
go.sum
View file

@ -162,6 +162,8 @@ github.com/bits-and-blooms/bitset v1.1.10/go.mod h1:w0XsmFg8qg6cmpTtJ0z3pKgjTDBM
github.com/bits-and-blooms/bitset v1.2.0/go.mod h1:gIdJ4wp64HaoK2YrL1Q5/N7Y16edYb8uY+O0FJTyyDA=
github.com/bits-and-blooms/bitset v1.5.0 h1:NpE8frKRLGHIcEzkR+gZhiioW1+WbYV6fKwD6ZIpQT8=
github.com/bits-and-blooms/bitset v1.5.0/go.mod h1:gIdJ4wp64HaoK2YrL1Q5/N7Y16edYb8uY+O0FJTyyDA=
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb h1:m935MPodAbYS46DG4pJSv7WO+VECIWUQ7OJYSoTrMh4=
github.com/blakesmith/ar v0.0.0-20190502131153-809d4375e1fb/go.mod h1:PkYb9DJNAwrSvRx5DYA+gUcOIgTGVMNkfSCbZM8cWpI=
github.com/blevesearch/bleve/v2 v2.0.5/go.mod h1:ZjWibgnbRX33c+vBRgla9QhPb4QOjD6fdVJ+R1Bk8LM=
github.com/blevesearch/bleve/v2 v2.3.6 h1:NlntUHcV5CSWIhpugx4d/BRMGCiaoI8ZZXrXlahzNq4=
github.com/blevesearch/bleve/v2 v2.3.6/go.mod h1:JM2legf1cKVkdV8Ehu7msKIOKC0McSw0Q16Fmv9vsW4=

View file

@ -489,6 +489,8 @@ var migrations = []Migration{
NewMigration("Add ActionTaskOutput table", v1_20.AddActionTaskOutputTable),
// v255 -> v256
NewMigration("Add ArchivedUnix Column", v1_20.AddArchivedUnixToRepository),
// v256 -> v257
NewMigration("Add is_internal column to package", v1_20.AddIsInternalColumnToPackage),
}
// GetCurrentDBVersion returns the current db version

View file

@ -0,0 +1,23 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package v1_20 //nolint
import (
"xorm.io/xorm"
)
func AddIsInternalColumnToPackage(x *xorm.Engine) error {
type Package struct {
ID int64 `xorm:"pk autoincr"`
OwnerID int64 `xorm:"UNIQUE(s) INDEX NOT NULL"`
RepoID int64 `xorm:"INDEX"`
Type string `xorm:"UNIQUE(s) INDEX NOT NULL"`
Name string `xorm:"NOT NULL"`
LowerName string `xorm:"UNIQUE(s) INDEX NOT NULL"`
SemverCompatible bool `xorm:"NOT NULL DEFAULT false"`
IsInternal bool `xorm:"INDEX NOT NULL DEFAULT false"`
}
return x.Sync2(new(Package))
}

View file

@ -101,16 +101,7 @@ func getContainerBlobsLimit(ctx context.Context, opts *BlobSearchOptions, limit
return nil, err
}
pfds := make([]*packages.PackageFileDescriptor, 0, len(pfs))
for _, pf := range pfs {
pfd, err := packages.GetPackageFileDescriptor(ctx, pf)
if err != nil {
return nil, err
}
pfds = append(pfds, pfd)
}
return pfds, nil
return packages.GetPackageFileDescriptors(ctx, pfs)
}
// GetManifestVersions gets all package versions representing the matching manifest

View file

@ -0,0 +1,131 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package debian
import (
"context"
"strconv"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/models/packages"
debian_module "code.gitea.io/gitea/modules/packages/debian"
"xorm.io/builder"
)
type PackageSearchOptions struct {
OwnerID int64
Distribution string
Component string
Architecture string
}
// SearchLatestPackages gets the latest packages matching the search options
func SearchLatestPackages(ctx context.Context, opts *PackageSearchOptions) ([]*packages.PackageFileDescriptor, error) {
var cond builder.Cond = builder.Eq{
"package_file.is_lead": true,
"package.type": packages.TypeDebian,
"package.owner_id": opts.OwnerID,
"package.is_internal": false,
"package_version.is_internal": false,
}
props := make(map[string]string)
if opts.Distribution != "" {
props[debian_module.PropertyDistribution] = opts.Distribution
}
if opts.Component != "" {
props[debian_module.PropertyComponent] = opts.Component
}
if opts.Architecture != "" {
props[debian_module.PropertyArchitecture] = opts.Architecture
}
if len(props) > 0 {
var propsCond builder.Cond = builder.Eq{
"package_property.ref_type": packages.PropertyTypeFile,
}
propsCond = propsCond.And(builder.Expr("package_property.ref_id = package_file.id"))
propsCondBlock := builder.NewCond()
for name, value := range props {
propsCondBlock = propsCondBlock.Or(builder.Eq{
"package_property.name": name,
"package_property.value": value,
})
}
propsCond = propsCond.And(propsCondBlock)
cond = cond.And(builder.Eq{
strconv.Itoa(len(props)): builder.Select("COUNT(*)").Where(propsCond).From("package_property"),
})
}
cond = cond.
And(builder.Expr("pv2.id IS NULL"))
joinCond := builder.
Expr("package_version.package_id = pv2.package_id AND (package_version.created_unix < pv2.created_unix OR (package_version.created_unix = pv2.created_unix AND package_version.id < pv2.id))").
And(builder.Eq{"pv2.is_internal": false})
pfs := make([]*packages.PackageFile, 0, 10)
err := db.GetEngine(ctx).
Table("package_file").
Select("package_file.*").
Join("INNER", "package_version", "package_version.id = package_file.version_id").
Join("LEFT", "package_version pv2", joinCond).
Join("INNER", "package", "package.id = package_version.package_id").
Where(cond).
Desc("package_version.created_unix").
Find(&pfs)
if err != nil {
return nil, err
}
return packages.GetPackageFileDescriptors(ctx, pfs)
}
// GetDistributions gets all available distributions
func GetDistributions(ctx context.Context, ownerID int64) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, "", debian_module.PropertyDistribution)
}
// GetComponents gets all available components for the given distribution
func GetComponents(ctx context.Context, ownerID int64, distribution string) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyComponent)
}
// GetArchitectures gets all available architectures for the given distribution
func GetArchitectures(ctx context.Context, ownerID int64, distribution string) ([]string, error) {
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyArchitecture)
}
func getDistinctPropertyValues(ctx context.Context, ownerID int64, distribution, propName string) ([]string, error) {
var cond builder.Cond = builder.Eq{
"package_property.ref_type": packages.PropertyTypeFile,
"package_property.name": propName,
"package.type": packages.TypeDebian,
"package.owner_id": ownerID,
}
if distribution != "" {
innerCond := builder.
Expr("pp.ref_id = package_property.ref_id").
And(builder.Eq{
"pp.ref_type": packages.PropertyTypeFile,
"pp.name": debian_module.PropertyDistribution,
"pp.value": distribution,
})
cond = cond.And(builder.Exists(builder.Select("pp.ref_id").From("package_property pp").Where(innerCond)))
}
values := make([]string, 0, 5)
return values, db.GetEngine(ctx).
Table("package_property").
Distinct("package_property.value").
Join("INNER", "package_file", "package_file.id = package_property.ref_id").
Join("INNER", "package_version", "package_version.id = package_file.version_id").
Join("INNER", "package", "package.id = package_version.package_id").
Where(cond).
Find(&values)
}

View file

@ -18,6 +18,7 @@ import (
"code.gitea.io/gitea/modules/packages/conan"
"code.gitea.io/gitea/modules/packages/conda"
"code.gitea.io/gitea/modules/packages/container"
"code.gitea.io/gitea/modules/packages/debian"
"code.gitea.io/gitea/modules/packages/helm"
"code.gitea.io/gitea/modules/packages/maven"
"code.gitea.io/gitea/modules/packages/npm"
@ -127,13 +128,9 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc
return nil, err
}
pfds := make([]*PackageFileDescriptor, 0, len(pfs))
for _, pf := range pfs {
pfd, err := GetPackageFileDescriptor(ctx, pf)
if err != nil {
return nil, err
}
pfds = append(pfds, pfd)
pfds, err := GetPackageFileDescriptors(ctx, pfs)
if err != nil {
return nil, err
}
var metadata interface{}
@ -150,6 +147,8 @@ func GetPackageDescriptor(ctx context.Context, pv *PackageVersion) (*PackageDesc
metadata = &conda.VersionMetadata{}
case TypeContainer:
metadata = &container.Metadata{}
case TypeDebian:
metadata = &debian.Metadata{}
case TypeGeneric:
// generic packages have no metadata
case TypeHelm:
@ -210,6 +209,19 @@ func GetPackageFileDescriptor(ctx context.Context, pf *PackageFile) (*PackageFil
}, nil
}
// GetPackageFileDescriptors gets the package file descriptors for the package files
func GetPackageFileDescriptors(ctx context.Context, pfs []*PackageFile) ([]*PackageFileDescriptor, error) {
pfds := make([]*PackageFileDescriptor, 0, len(pfs))
for _, pf := range pfs {
pfd, err := GetPackageFileDescriptor(ctx, pf)
if err != nil {
return nil, err
}
pfds = append(pfds, pfd)
}
return pfds, nil
}
// GetPackageDescriptors gets the package descriptions for the versions
func GetPackageDescriptors(ctx context.Context, pvs []*PackageVersion) ([]*PackageDescriptor, error) {
pds := make([]*PackageDescriptor, 0, len(pvs))

View file

@ -36,6 +36,7 @@ const (
TypeConan Type = "conan"
TypeConda Type = "conda"
TypeContainer Type = "container"
TypeDebian Type = "debian"
TypeGeneric Type = "generic"
TypeHelm Type = "helm"
TypeMaven Type = "maven"
@ -55,6 +56,7 @@ var TypeList = []Type{
TypeConan,
TypeConda,
TypeContainer,
TypeDebian,
TypeGeneric,
TypeHelm,
TypeMaven,
@ -82,6 +84,8 @@ func (pt Type) Name() string {
return "Conda"
case TypeContainer:
return "Container"
case TypeDebian:
return "Debian"
case TypeGeneric:
return "Generic"
case TypeHelm:
@ -121,6 +125,8 @@ func (pt Type) SVGName() string {
return "gitea-conda"
case TypeContainer:
return "octicon-container"
case TypeDebian:
return "gitea-debian"
case TypeGeneric:
return "octicon-package"
case TypeHelm:
@ -154,6 +160,7 @@ type Package struct {
Name string `xorm:"NOT NULL"`
LowerName string `xorm:"UNIQUE(s) INDEX NOT NULL"`
SemverCompatible bool `xorm:"NOT NULL DEFAULT false"`
IsInternal bool `xorm:"INDEX NOT NULL DEFAULT false"`
}
// TryInsertPackage inserts a package. If a package exists already, ErrDuplicatePackage is returned
@ -214,9 +221,10 @@ func GetPackageByID(ctx context.Context, packageID int64) (*Package, error) {
// GetPackageByName gets a package by name
func GetPackageByName(ctx context.Context, ownerID int64, packageType Type, name string) (*Package, error) {
var cond builder.Cond = builder.Eq{
"package.owner_id": ownerID,
"package.type": packageType,
"package.lower_name": strings.ToLower(name),
"package.owner_id": ownerID,
"package.type": packageType,
"package.lower_name": strings.ToLower(name),
"package.is_internal": false,
}
p := &Package{}
@ -236,8 +244,9 @@ func GetPackageByName(ctx context.Context, ownerID int64, packageType Type, name
// GetPackagesByType gets all packages of a specific type
func GetPackagesByType(ctx context.Context, ownerID int64, packageType Type) ([]*Package, error) {
var cond builder.Cond = builder.Eq{
"package.owner_id": ownerID,
"package.type": packageType,
"package.owner_id": ownerID,
"package.type": packageType,
"package.is_internal": false,
}
ps := make([]*Package, 0, 10)

View file

@ -117,13 +117,15 @@ func DeleteFileByID(ctx context.Context, fileID int64) error {
// PackageFileSearchOptions are options for SearchXXX methods
type PackageFileSearchOptions struct {
OwnerID int64
PackageType string
VersionID int64
Query string
CompositeKey string
Properties map[string]string
OlderThan time.Duration
OwnerID int64
PackageType string
VersionID int64
Query string
CompositeKey string
Properties map[string]string
OlderThan time.Duration
HashAlgorithmn string
Hash string
db.Paginator
}
@ -182,6 +184,15 @@ func (opts *PackageFileSearchOptions) toConds() builder.Cond {
cond = cond.And(builder.Lt{"package_file.created_unix": time.Now().Add(-opts.OlderThan).Unix()})
}
if opts.Hash != "" && (opts.HashAlgorithmn == "md5" || opts.HashAlgorithmn == "sha1" || opts.HashAlgorithmn == "sha256" || opts.HashAlgorithmn == "sha512") {
innerCond := builder.
Expr("package_blob.id = package_file.blob_id").
And(builder.Eq{
"package_blob.hash_" + opts.HashAlgorithmn: opts.Hash,
})
cond = cond.And(builder.Exists(builder.Select("package_blob.id").From("package_blob").Where(innerCond)))
}
return cond
}

View file

@ -173,7 +173,7 @@ const (
)
// PackageSearchOptions are options for SearchXXX methods
// Besides IsInternal are all fields optional and are not used if they have their default value (nil, "", 0)
// All fields optional and are not used if they have their default value (nil, "", 0)
type PackageSearchOptions struct {
OwnerID int64
RepoID int64
@ -192,7 +192,9 @@ type PackageSearchOptions struct {
func (opts *PackageSearchOptions) toConds() builder.Cond {
cond := builder.NewCond()
if !opts.IsInternal.IsNone() {
cond = builder.Eq{"package_version.is_internal": opts.IsInternal.IsTrue()}
cond = builder.Eq{
"package_version.is_internal": opts.IsInternal.IsTrue(),
}
}
if opts.OwnerID != 0 {

View file

@ -11,6 +11,7 @@ import (
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/modules/cache"
setting_module "code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
"xorm.io/builder"
)
@ -42,6 +43,10 @@ func (err ErrUserSettingIsNotExist) Error() string {
return fmt.Sprintf("Setting[%s] is not exist", err.Key)
}
func (err ErrUserSettingIsNotExist) Unwrap() error {
return util.ErrNotExist
}
// IsErrUserSettingIsNotExist return true if err is ErrSettingIsNotExist
func IsErrUserSettingIsNotExist(err error) bool {
_, ok := err.(ErrUserSettingIsNotExist)

View file

@ -0,0 +1,216 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package debian
import (
"archive/tar"
"bufio"
"compress/gzip"
"io"
"net/mail"
"regexp"
"strings"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/modules/validation"
"github.com/blakesmith/ar"
"github.com/klauspost/compress/zstd"
"github.com/ulikunitz/xz"
)
const (
PropertyDistribution = "debian.distribution"
PropertyComponent = "debian.component"
PropertyArchitecture = "debian.architecture"
PropertyControl = "debian.control"
PropertyRepositoryIncludeInRelease = "debian.repository.include_in_release"
SettingKeyPrivate = "debian.key.private"
SettingKeyPublic = "debian.key.public"
RepositoryPackage = "_debian"
RepositoryVersion = "_repository"
)
var (
ErrMissingControlFile = util.NewInvalidArgumentErrorf("control file is missing")
ErrUnsupportedCompression = util.NewInvalidArgumentErrorf("unsupported compression algorithmn")
ErrInvalidName = util.NewInvalidArgumentErrorf("package name is invalid")
ErrInvalidVersion = util.NewInvalidArgumentErrorf("package version is invalid")
ErrInvalidArchitecture = util.NewInvalidArgumentErrorf("package architecture is invalid")
// https://www.debian.org/doc/debian-policy/ch-controlfields.html#source
namePattern = regexp.MustCompile(`\A[a-z0-9][a-z0-9+-.]+\z`)
// https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
versionPattern = regexp.MustCompile(`\A(?:[0-9]:)?[a-zA-Z0-9.+~]+(?:-[a-zA-Z0-9.+-~]+)?\z`)
)
type Package struct {
Name string
Version string
Architecture string
Control string
Metadata *Metadata
}
type Metadata struct {
Maintainer string `json:"maintainer,omitempty"`
ProjectURL string `json:"project_url,omitempty"`
Description string `json:"description,omitempty"`
Dependencies []string `json:"dependencies,omitempty"`
}
// ParsePackage parses the Debian package file
// https://manpages.debian.org/bullseye/dpkg-dev/deb.5.en.html
func ParsePackage(r io.Reader) (*Package, error) {
arr := ar.NewReader(r)
for {
hd, err := arr.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
if strings.HasPrefix(hd.Name, "control.tar") {
var inner io.Reader
switch hd.Name[11:] {
case "":
inner = arr
case ".gz":
gzr, err := gzip.NewReader(arr)
if err != nil {
return nil, err
}
defer gzr.Close()
inner = gzr
case ".xz":
xzr, err := xz.NewReader(arr)
if err != nil {
return nil, err
}
inner = xzr
case ".zst":
zr, err := zstd.NewReader(arr)
if err != nil {
return nil, err
}
defer zr.Close()
inner = zr
default:
return nil, ErrUnsupportedCompression
}
tr := tar.NewReader(inner)
for {
hd, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
if hd.Typeflag != tar.TypeReg {
continue
}
if hd.FileInfo().Name() == "control" {
return ParseControlFile(tr)
}
}
}
}
return nil, ErrMissingControlFile
}
// ParseControlFile parses a Debian control file to retrieve the metadata
func ParseControlFile(r io.Reader) (*Package, error) {
p := &Package{
Metadata: &Metadata{},
}
key := ""
var depends strings.Builder
var control strings.Builder
s := bufio.NewScanner(io.TeeReader(r, &control))
for s.Scan() {
line := s.Text()
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
if line[0] == ' ' || line[0] == '\t' {
switch key {
case "Description":
p.Metadata.Description += line
case "Depends":
depends.WriteString(trimmed)
}
} else {
parts := strings.SplitN(trimmed, ":", 2)
if len(parts) < 2 {
continue
}
key = parts[0]
value := strings.TrimSpace(parts[1])
switch key {
case "Package":
if !namePattern.MatchString(value) {
return nil, ErrInvalidName
}
p.Name = value
case "Version":
if !versionPattern.MatchString(value) {
return nil, ErrInvalidVersion
}
p.Version = value
case "Architecture":
if value == "" {
return nil, ErrInvalidArchitecture
}
p.Architecture = value
case "Maintainer":
a, err := mail.ParseAddress(value)
if err != nil || a.Name == "" {
p.Metadata.Maintainer = value
} else {
p.Metadata.Maintainer = a.Name
}
case "Description":
p.Metadata.Description = value
case "Depends":
depends.WriteString(value)
case "Homepage":
if validation.IsValidURL(value) {
p.Metadata.ProjectURL = value
}
}
}
}
if err := s.Err(); err != nil {
return nil, err
}
dependencies := strings.Split(depends.String(), ",")
for i := range dependencies {
dependencies[i] = strings.TrimSpace(dependencies[i])
}
p.Metadata.Dependencies = dependencies
p.Control = control.String()
return p, nil
}

View file

@ -0,0 +1,171 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package debian
import (
"archive/tar"
"bytes"
"compress/gzip"
"io"
"testing"
"github.com/blakesmith/ar"
"github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/assert"
"github.com/ulikunitz/xz"
)
const (
packageName = "gitea"
packageVersion = "0:1.0.1-te~st"
packageArchitecture = "amd64"
packageAuthor = "KN4CK3R"
description = "Description with multiple lines."
projectURL = "https://gitea.io"
)
func TestParsePackage(t *testing.T) {
createArchive := func(files map[string][]byte) io.Reader {
var buf bytes.Buffer
aw := ar.NewWriter(&buf)
aw.WriteGlobalHeader()
for filename, content := range files {
hdr := &ar.Header{
Name: filename,
Mode: 0o600,
Size: int64(len(content)),
}
aw.WriteHeader(hdr)
aw.Write(content)
}
return &buf
}
t.Run("MissingControlFile", func(t *testing.T) {
data := createArchive(map[string][]byte{"dummy.txt": {}})
p, err := ParsePackage(data)
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrMissingControlFile)
})
t.Run("Compression", func(t *testing.T) {
t.Run("Unsupported", func(t *testing.T) {
data := createArchive(map[string][]byte{"control.tar.foo": {}})
p, err := ParsePackage(data)
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrUnsupportedCompression)
})
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
tw.WriteHeader(&tar.Header{
Name: "control",
Mode: 0o600,
Size: 50,
})
tw.Write([]byte("Package: gitea\nVersion: 1.0.0\nArchitecture: amd64\n"))
tw.Close()
t.Run("None", func(t *testing.T) {
data := createArchive(map[string][]byte{"control.tar": buf.Bytes()})
p, err := ParsePackage(data)
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, "gitea", p.Name)
})
t.Run("gz", func(t *testing.T) {
var zbuf bytes.Buffer
zw := gzip.NewWriter(&zbuf)
zw.Write(buf.Bytes())
zw.Close()
data := createArchive(map[string][]byte{"control.tar.gz": zbuf.Bytes()})
p, err := ParsePackage(data)
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, "gitea", p.Name)
})
t.Run("xz", func(t *testing.T) {
var xbuf bytes.Buffer
xw, _ := xz.NewWriter(&xbuf)
xw.Write(buf.Bytes())
xw.Close()
data := createArchive(map[string][]byte{"control.tar.xz": xbuf.Bytes()})
p, err := ParsePackage(data)
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, "gitea", p.Name)
})
t.Run("zst", func(t *testing.T) {
var zbuf bytes.Buffer
zw, _ := zstd.NewWriter(&zbuf)
zw.Write(buf.Bytes())
zw.Close()
data := createArchive(map[string][]byte{"control.tar.zst": zbuf.Bytes()})
p, err := ParsePackage(data)
assert.NotNil(t, p)
assert.NoError(t, err)
assert.Equal(t, "gitea", p.Name)
})
})
}
func TestParseControlFile(t *testing.T) {
buildContent := func(name, version, architecture string) *bytes.Buffer {
var buf bytes.Buffer
buf.WriteString("Package: " + name + "\nVersion: " + version + "\nArchitecture: " + architecture + "\nMaintainer: " + packageAuthor + " <kn4ck3r@gitea.io>\nHomepage: " + projectURL + "\nDepends: a,\n b\nDescription: Description\n with multiple\n lines.")
return &buf
}
t.Run("InvalidName", func(t *testing.T) {
for _, name := range []string{"", "-cd"} {
p, err := ParseControlFile(buildContent(name, packageVersion, packageArchitecture))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidName)
}
})
t.Run("InvalidVersion", func(t *testing.T) {
for _, version := range []string{"", "1-", ":1.0", "1_0"} {
p, err := ParseControlFile(buildContent(packageName, version, packageArchitecture))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidVersion)
}
})
t.Run("InvalidArchitecture", func(t *testing.T) {
p, err := ParseControlFile(buildContent(packageName, packageVersion, ""))
assert.Nil(t, p)
assert.ErrorIs(t, err, ErrInvalidArchitecture)
})
t.Run("Valid", func(t *testing.T) {
content := buildContent(packageName, packageVersion, packageArchitecture)
full := content.String()
p, err := ParseControlFile(content)
assert.NoError(t, err)
assert.NotNil(t, p)
assert.Equal(t, packageName, p.Name)
assert.Equal(t, packageVersion, p.Version)
assert.Equal(t, packageArchitecture, p.Architecture)
assert.Equal(t, description, p.Metadata.Description)
assert.Equal(t, projectURL, p.Metadata.ProjectURL)
assert.Equal(t, packageAuthor, p.Metadata.Maintainer)
assert.Equal(t, []string{"a", "b"}, p.Metadata.Dependencies)
assert.Equal(t, full, p.Control)
})
}

View file

@ -25,8 +25,15 @@ type HashedBuffer struct {
combinedWriter io.Writer
}
// NewHashedBuffer creates a hashed buffer with a specific maximum memory size
func NewHashedBuffer(maxMemorySize int) (*HashedBuffer, error) {
const DefaultMemorySize = 32 * 1024 * 1024
// NewHashedBuffer creates a hashed buffer with the default memory size
func NewHashedBuffer() (*HashedBuffer, error) {
return NewHashedBufferWithSize(DefaultMemorySize)
}
// NewHashedBuffer creates a hashed buffer with a specific memory size
func NewHashedBufferWithSize(maxMemorySize int) (*HashedBuffer, error) {
b, err := filebuffer.New(maxMemorySize)
if err != nil {
return nil, err
@ -43,9 +50,14 @@ func NewHashedBuffer(maxMemorySize int) (*HashedBuffer, error) {
}, nil
}
// CreateHashedBufferFromReader creates a hashed buffer and copies the provided reader data into it.
func CreateHashedBufferFromReader(r io.Reader, maxMemorySize int) (*HashedBuffer, error) {
b, err := NewHashedBuffer(maxMemorySize)
// CreateHashedBufferFromReader creates a hashed buffer with the default memory size and copies the provided reader data into it.
func CreateHashedBufferFromReader(r io.Reader) (*HashedBuffer, error) {
return CreateHashedBufferFromReaderWithSize(r, DefaultMemorySize)
}
// CreateHashedBufferFromReaderWithSize creates a hashed buffer and copies the provided reader data into it.
func CreateHashedBufferFromReaderWithSize(r io.Reader, maxMemorySize int) (*HashedBuffer, error) {
b, err := NewHashedBufferWithSize(maxMemorySize)
if err != nil {
return nil, err
}

View file

@ -26,7 +26,7 @@ func TestHashedBuffer(t *testing.T) {
}
for _, c := range cases {
buf, err := CreateHashedBufferFromReader(strings.NewReader(c.Data), c.MaxMemorySize)
buf, err := CreateHashedBufferFromReaderWithSize(strings.NewReader(c.Data), c.MaxMemorySize)
assert.NoError(t, err)
assert.EqualValues(t, len(c.Data), buf.Size())

View file

@ -63,7 +63,7 @@ func ExtractPortablePdb(r io.ReaderAt, size int64) (PortablePdbList, error) {
return err
}
buf, err := packages.CreateHashedBufferFromReader(f, 32*1024*1024)
buf, err := packages.CreateHashedBufferFromReader(f)
f.Close()

View file

@ -30,6 +30,7 @@ var (
LimitSizeConan int64
LimitSizeConda int64
LimitSizeContainer int64
LimitSizeDebian int64
LimitSizeGeneric int64
LimitSizeHelm int64
LimitSizeMaven int64
@ -73,6 +74,7 @@ func loadPackagesFrom(rootCfg ConfigProvider) {
Packages.LimitSizeConan = mustBytes(sec, "LIMIT_SIZE_CONAN")
Packages.LimitSizeConda = mustBytes(sec, "LIMIT_SIZE_CONDA")
Packages.LimitSizeContainer = mustBytes(sec, "LIMIT_SIZE_CONTAINER")
Packages.LimitSizeDebian = mustBytes(sec, "LIMIT_SIZE_DEBIAN")
Packages.LimitSizeGeneric = mustBytes(sec, "LIMIT_SIZE_GENERIC")
Packages.LimitSizeHelm = mustBytes(sec, "LIMIT_SIZE_HELM")
Packages.LimitSizeMaven = mustBytes(sec, "LIMIT_SIZE_MAVEN")

View file

@ -7,11 +7,10 @@ import (
"bytes"
"errors"
"io"
"math"
"os"
)
const maxInt = int(^uint(0) >> 1) // taken from bytes.Buffer
var (
// ErrInvalidMemorySize occurs if the memory size is not in a valid range
ErrInvalidMemorySize = errors.New("Memory size must be greater 0 and lower math.MaxInt32")
@ -37,7 +36,7 @@ type FileBackedBuffer struct {
// New creates a file backed buffer with a specific maximum memory size
func New(maxMemorySize int) (*FileBackedBuffer, error) {
if maxMemorySize < 0 || maxMemorySize > maxInt {
if maxMemorySize < 0 || maxMemorySize > math.MaxInt32 {
return nil, ErrInvalidMemorySize
}

View file

@ -3258,6 +3258,14 @@ container.layers = Image Layers
container.labels = Labels
container.labels.key = Key
container.labels.value = Value
debian.registry = Setup this registry from the command line:
debian.registry.info = Choose <code>&lt;distribution&gt;</code> and <code>&lt;component&gt;</code> from the list below.
debian.install = To install the package, run the following command:
debian.documentation = For more information on the Debian registry, see <a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/en-us/packages/debian/">the documentation</a>.
debian.repository = Repository Info
debian.repository.distributions = Distributions
debian.repository.components = Components
debian.repository.architectures = Architectures
generic.download = Download package from the command line:
generic.documentation = For more information on the generic registry, see <a target="_blank" rel="noopener noreferrer" href="https://docs.gitea.io/en-us/packages/generic">the documentation</a>.
helm.registry = Setup this registry from the command line:

1
public/img/svg/gitea-debian.svg generated Normal file
View file

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 210 260" class="svg gitea-debian" width="16" height="16" aria-hidden="true"><g fill="#D70751"><path d="M124.525 137.053c-4.125.058.78 2.125 6.165 2.954a54.75 54.75 0 0 0 4.04-3.479c-3.354.821-6.765.838-10.205.525m22.14-5.52c2.457-3.389 4.246-7.102 4.878-10.939-.551 2.736-2.035 5.099-3.435 7.592-7.711 4.854-.726-2.883-.004-5.824-8.29 10.436-1.138 6.257-1.439 9.171m8.174-21.265c.497-7.428-1.462-5.08-2.121-2.245.766.4 1.377 5.237 2.121 2.245M108.883 8.736c2.201.395 4.757.698 4.398 1.224 2.407-.528 2.954-1.015-4.398-1.224M113.281 9.96l-1.556.32 1.448-.127.108-.193"/><path d="M181.93 113.085c.247 6.671-1.95 9.907-3.932 15.637l-3.564 1.781c-2.919 5.666.282 3.598-1.807 8.105-4.556 4.049-13.823 12.67-16.789 13.457-2.163-.047 1.469-2.554 1.943-3.537-6.097 4.188-4.894 6.285-14.217 8.83l-.273-.607c-23.001 10.818-54.947-10.622-54.526-39.876-.246 1.857-.698 1.393-1.208 2.144-1.186-15.052 6.952-30.17 20.675-36.343 13.427-6.646 29.163-3.918 38.78 5.044-5.282-6.92-15.795-14.254-28.255-13.568-12.208.193-23.625 7.95-27.436 16.369-6.253 3.938-6.979 15.177-9.704 17.233-3.665 26.943 6.896 38.583 24.762 52.275 2.812 1.896.792 2.184 1.173 3.627-5.936-2.779-11.372-6.976-15.841-12.114 2.372 3.473 4.931 6.847 8.239 9.499-5.596-1.897-13.074-13.563-15.256-14.038 9.647 17.274 39.142 30.295 54.587 23.836-7.146.263-16.226.146-24.256-2.822-3.371-1.734-7.958-5.331-7.14-6.003 21.079 7.875 42.854 5.965 61.09-8.655 4.641-3.614 9.709-9.761 11.173-9.846-2.206 3.317.377 1.596-1.318 4.523 4.625-7.456-2.008-3.035 4.779-12.877l2.507 3.453c-.931-6.188 7.687-13.704 6.813-23.492 1.975-2.994 2.206 3.22.107 10.107 2.912-7.64.767-8.867 1.516-15.171.81 2.118 1.867 4.37 2.412 6.606-1.895-7.382 1.948-12.433 2.898-16.724-.937-.415-2.928 3.264-3.383-5.457.065-3.788 1.054-1.985 1.435-2.917-.744-.427-2.694-3.33-3.88-8.9.86-1.308 2.3 3.393 3.47 3.586-.753-4.429-2.049-7.805-2.103-11.202-3.421-7.149-1.211.953-3.985-3.069-3.641-11.357 3.021-2.637 3.47-7.796 5.52 7.995 8.667 20.387 10.11 25.519-1.103-6.258-2.883-12.32-5.058-18.185 1.677.705-2.699-12.875 2.18-3.882-5.21-19.172-22.302-37.087-38.025-45.493 1.924 1.76 4.354 3.971 3.481 4.317-7.819-4.656-6.444-5.018-7.565-6.985-6.369-2.591-6.788.208-11.007.004-12.005-6.368-14.318-5.69-25.368-9.681l.502 2.349c-7.953-2.649-9.265 1.005-17.862.009-.523-.409 2.753-1.479 5.452-1.871-7.69 1.015-7.329-1.515-14.854.279 1.855-1.301 3.815-2.162 5.793-3.269-6.271.381-14.971 3.649-12.286.677-10.235 4.569-28.403 10.976-38.597 20.535l-.321-2.142c-4.672 5.608-20.371 16.748-21.622 24.011l-1.249.291c-2.431 4.116-4.004 8.781-5.932 13.016-3.18 5.417-4.661 2.085-4.208 2.934-6.253 12.679-9.359 23.332-12.043 32.069 1.912 2.858.046 17.206.769 28.688-3.141 56.709 39.8 111.77 86.737 124.48 6.88 2.459 17.11 2.364 25.813 2.618-10.268-2.937-11.595-1.556-21.595-5.044-7.215-3.398-8.797-7.277-13.907-11.711l2.022 3.573c-10.021-3.547-5.829-4.39-13.982-6.972l2.16-2.82c-3.249-.246-8.604-5.475-10.069-8.371l-3.553.14c-4.27-5.269-6.545-9.063-6.379-12.005l-1.148 2.047c-1.301-2.235-15.709-19.759-8.234-15.679-1.389-1.271-3.235-2.067-5.237-5.703l1.522-1.739c-3.597-4.627-6.621-10.562-6.391-12.536 1.919 2.592 3.25 3.075 4.568 3.52-9.083-22.539-9.593-1.242-16.474-22.942l1.456-.116c-1.116-1.682-1.793-3.506-2.69-5.298l.633-6.313c-6.541-7.562-1.829-32.151-.887-45.637.655-5.485 5.459-11.322 9.114-20.477l-2.227-.384c4.256-7.423 24.301-29.814 33.583-28.662 4.499-5.649-.892-.02-1.772-1.443 9.878-10.223 12.984-7.222 19.65-9.061 7.19-4.268-6.17 1.664-2.761-1.628 12.427-3.174 8.808-7.216 25.021-8.828 1.71.973-3.969 1.503-5.395 2.766 10.354-5.066 32.769-3.914 47.326 2.811 16.895 7.896 35.873 31.232 36.622 53.189l.852.229c-.431 8.729 1.336 18.822-1.727 28.094l2.1-4.385"/><path d="m79.5 142.715-.578 2.893c2.71 3.683 4.861 7.673 8.323 10.552-2.49-4.863-4.341-6.872-7.745-13.445m6.409-.251c-1.435-1.587-2.284-3.497-3.235-5.4.909 3.345 2.771 6.219 4.504 9.143l-1.269-3.743m113.411-24.65-.605 1.52c-1.111 7.892-3.511 15.701-7.189 22.941a72.098 72.098 0 0 0 7.79-24.461M109.698 6.757c2.789-1.022 6.855-.56 9.814-1.233-3.855.324-7.693.517-11.484 1.005l1.67.228M11.781 58.824c.642 5.951-4.477 8.26 1.134 4.337 3.007-6.773-1.175-1.87-1.134-4.337M5.188 86.362c1.292-3.967 1.526-6.349 2.02-8.645-3.571 4.566-1.643 5.539-2.02 8.645"/></g></svg>

After

Width:  |  Height:  |  Size: 4.2 KiB

View file

@ -21,6 +21,7 @@ import (
"code.gitea.io/gitea/routers/api/packages/conan"
"code.gitea.io/gitea/routers/api/packages/conda"
"code.gitea.io/gitea/routers/api/packages/container"
"code.gitea.io/gitea/routers/api/packages/debian"
"code.gitea.io/gitea/routers/api/packages/generic"
"code.gitea.io/gitea/routers/api/packages/helm"
"code.gitea.io/gitea/routers/api/packages/maven"
@ -272,6 +273,24 @@ func CommonRoutes(ctx gocontext.Context) *web.Route {
conda.UploadPackageFile(ctx)
})
}, reqPackageAccess(perm.AccessModeRead))
r.Group("/debian", func() {
r.Get("/repository.key", debian.GetRepositoryKey)
r.Group("/dists/{distribution}", func() {
r.Get("/{filename}", debian.GetRepositoryFile)
r.Get("/by-hash/{algorithmn}/{hash}", debian.GetRepositoryFileByHash)
r.Group("/{component}/{architecture}", func() {
r.Get("/{filename}", debian.GetRepositoryFile)
r.Get("/by-hash/{algorithmn}/{hash}", debian.GetRepositoryFileByHash)
})
})
r.Group("/pool/{distribution}/{component}", func() {
r.Get("/{name}_{version}_{architecture}.deb", debian.DownloadPackageFile)
r.Group("", func() {
r.Put("/upload", debian.UploadPackageFile)
r.Delete("/{name}/{version}/{architecture}", debian.DeletePackageFile)
}, reqPackageAccess(perm.AccessModeWrite))
})
}, reqPackageAccess(perm.AccessModeRead))
r.Group("/generic", func() {
r.Group("/{packagename}/{packageversion}", func() {
r.Delete("", reqPackageAccess(perm.AccessModeWrite), generic.DeletePackage)

View file

@ -173,7 +173,7 @@ func UploadPackage(ctx *context.Context) {
return
}
buf, err := packages_module.CreateHashedBufferFromReader(cp.Content, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(cp.Content)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -263,7 +263,7 @@ func UploadPackage(ctx *context.Context) {
}
defer file.Close()
buf, err := packages_module.CreateHashedBufferFromReader(file, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(file)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -192,7 +192,7 @@ func DownloadPackageFile(ctx *context.Context) {
// UploadPackage creates a new package
func UploadPackage(ctx *context.Context) {
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -318,7 +318,7 @@ func uploadFile(ctx *context.Context, fileFilter container.Set[string], fileKey
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
@ -648,10 +648,7 @@ func deleteRecipeOrPackage(apictx *context.Context, rref *conan_module.RecipeRef
}
for _, pf := range pfs {
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil {
return err
}
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil {
if err := packages_service.DeletePackageFile(ctx, pf); err != nil {
return err
}
}
@ -664,11 +661,7 @@ func deleteRecipeOrPackage(apictx *context.Context, rref *conan_module.RecipeRef
if !has {
versionDeleted = true
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeVersion, pv.ID); err != nil {
return err
}
if err := packages_model.DeleteVersionByID(ctx, pv.ID); err != nil {
if err := packages_service.DeletePackageVersionAndReferences(ctx, pv); err != nil {
return err
}
}

View file

@ -183,7 +183,7 @@ func UploadPackageFile(ctx *context.Context) {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -27,10 +27,6 @@ var uploadVersionMutex sync.Mutex
// saveAsPackageBlob creates a package blob from an upload
// The uploaded blob gets stored in a special upload version to link them to the package/image
func saveAsPackageBlob(hsr packages_module.HashedSizeReader, pci *packages_service.PackageCreationInfo) (*packages_model.PackageBlob, error) {
if err := packages_service.CheckSizeQuotaExceeded(db.DefaultContext, pci.Creator, pci.Owner, packages_model.TypeContainer, hsr.Size()); err != nil {
return nil, err
}
pb := packages_service.NewPackageBlob(hsr)
exists := false
@ -43,6 +39,10 @@ func saveAsPackageBlob(hsr packages_module.HashedSizeReader, pci *packages_servi
}
err = db.WithTx(db.DefaultContext, func(ctx context.Context) error {
if err := packages_service.CheckSizeQuotaExceeded(ctx, pci.Creator, pci.Owner, packages_model.TypeContainer, hsr.Size()); err != nil {
return err
}
pb, exists, err = packages_model.GetOrInsertBlob(ctx, pb)
if err != nil {
log.Error("Error inserting package blob: %v", err)

View file

@ -219,7 +219,7 @@ func InitiateUploadBlob(ctx *context.Context) {
digest := ctx.FormTrim("digest")
if digest != "" {
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
@ -538,7 +538,7 @@ func UploadManifest(ctx *context.Context) {
}
maxSize := maxManifestSize + 1
buf, err := packages_module.CreateHashedBufferFromReader(&io.LimitedReader{R: ctx.Req.Body, N: int64(maxSize)}, maxSize)
buf, err := packages_module.CreateHashedBufferFromReaderWithSize(&io.LimitedReader{R: ctx.Req.Body, N: int64(maxSize)}, maxSize)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -0,0 +1,317 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package debian
import (
stdctx "context"
"errors"
"fmt"
"io"
"net/http"
"strings"
"code.gitea.io/gitea/models/db"
packages_model "code.gitea.io/gitea/models/packages"
"code.gitea.io/gitea/modules/context"
"code.gitea.io/gitea/modules/notification"
packages_module "code.gitea.io/gitea/modules/packages"
debian_module "code.gitea.io/gitea/modules/packages/debian"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/routers/api/packages/helper"
packages_service "code.gitea.io/gitea/services/packages"
debian_service "code.gitea.io/gitea/services/packages/debian"
)
func apiError(ctx *context.Context, status int, obj interface{}) {
helper.LogAndProcessError(ctx, status, obj, func(message string) {
ctx.PlainText(status, message)
})
}
func GetRepositoryKey(ctx *context.Context) {
_, pub, err := debian_service.GetOrCreateKeyPair(ctx.Package.Owner.ID)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
ctx.ServeContent(strings.NewReader(pub), &context.ServeHeaderOptions{
ContentType: "application/pgp-keys",
Filename: "repository.key",
})
}
// https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files
// https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices
func GetRepositoryFile(ctx *context.Context) {
pv, err := debian_service.GetOrCreateRepositoryVersion(ctx.Package.Owner.ID)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
key := ctx.Params("distribution")
component := ctx.Params("component")
architecture := strings.TrimPrefix(ctx.Params("architecture"), "binary-")
if component != "" && architecture != "" {
key += "|" + component + "|" + architecture
}
s, pf, err := packages_service.GetFileStreamByPackageVersion(
ctx,
pv,
&packages_service.PackageFileInfo{
Filename: ctx.Params("filename"),
CompositeKey: key,
},
)
if err != nil {
if err == packages_model.ErrPackageNotExist || err == packages_model.ErrPackageFileNotExist {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
defer s.Close()
ctx.ServeContent(s, &context.ServeHeaderOptions{
Filename: pf.Name,
LastModified: pf.CreatedUnix.AsLocalTime(),
})
}
// https://wiki.debian.org/DebianRepository/Format#indices_acquisition_via_hashsums_.28by-hash.29
func GetRepositoryFileByHash(ctx *context.Context) {
pv, err := debian_service.GetOrCreateRepositoryVersion(ctx.Package.Owner.ID)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
algorithmn := strings.ToLower(ctx.Params("algorithmn"))
if algorithmn == "md5sum" {
algorithmn = "md5"
}
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{
VersionID: pv.ID,
Hash: strings.ToLower(ctx.Params("hash")),
HashAlgorithmn: algorithmn,
})
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
if len(pfs) != 1 {
apiError(ctx, http.StatusNotFound, nil)
return
}
s, pf, err := packages_service.GetPackageFileStream(ctx, pfs[0])
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
defer s.Close()
ctx.ServeContent(s, &context.ServeHeaderOptions{
Filename: pf.Name,
LastModified: pf.CreatedUnix.AsLocalTime(),
})
}
func UploadPackageFile(ctx *context.Context) {
distribution := strings.TrimSpace(ctx.Params("distribution"))
component := strings.TrimSpace(ctx.Params("component"))
if distribution == "" || component == "" {
apiError(ctx, http.StatusBadRequest, "invalid distribution or component")
return
}
upload, close, err := ctx.UploadStream()
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
if close {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
defer buf.Close()
pck, err := debian_module.ParsePackage(buf)
if err != nil {
if errors.Is(err, util.ErrInvalidArgument) {
apiError(ctx, http.StatusBadRequest, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
if _, err := buf.Seek(0, io.SeekStart); err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
_, _, err = packages_service.CreatePackageOrAddFileToExisting(
&packages_service.PackageCreationInfo{
PackageInfo: packages_service.PackageInfo{
Owner: ctx.Package.Owner,
PackageType: packages_model.TypeDebian,
Name: pck.Name,
Version: pck.Version,
},
Creator: ctx.Doer,
Metadata: pck.Metadata,
},
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s_%s_%s.deb", pck.Name, pck.Version, pck.Architecture),
CompositeKey: fmt.Sprintf("%s|%s", distribution, component),
},
Creator: ctx.Doer,
Data: buf,
IsLead: true,
Properties: map[string]string{
debian_module.PropertyDistribution: distribution,
debian_module.PropertyComponent: component,
debian_module.PropertyArchitecture: pck.Architecture,
debian_module.PropertyControl: pck.Control,
},
},
)
if err != nil {
switch err {
case packages_model.ErrDuplicatePackageVersion:
apiError(ctx, http.StatusBadRequest, err)
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize:
apiError(ctx, http.StatusForbidden, err)
default:
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
if err := debian_service.BuildSpecificRepositoryFiles(ctx, ctx.Package.Owner.ID, distribution, component, pck.Architecture); err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
ctx.Status(http.StatusCreated)
}
func DownloadPackageFile(ctx *context.Context) {
name := ctx.Params("name")
version := ctx.Params("version")
s, pf, err := packages_service.GetFileStreamByPackageNameAndVersion(
ctx,
&packages_service.PackageInfo{
Owner: ctx.Package.Owner,
PackageType: packages_model.TypeDebian,
Name: name,
Version: version,
},
&packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s_%s_%s.deb", name, version, ctx.Params("architecture")),
CompositeKey: fmt.Sprintf("%s|%s", ctx.Params("distribution"), ctx.Params("component")),
},
)
if err != nil {
if err == packages_model.ErrPackageNotExist || err == packages_model.ErrPackageFileNotExist {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
defer s.Close()
ctx.ServeContent(s, &context.ServeHeaderOptions{
ContentType: "application/vnd.debian.binary-package",
Filename: pf.Name,
LastModified: pf.CreatedUnix.AsLocalTime(),
})
}
func DeletePackageFile(ctx *context.Context) {
distribution := ctx.Params("distribution")
component := ctx.Params("component")
name := ctx.Params("name")
version := ctx.Params("version")
architecture := ctx.Params("architecture")
owner := ctx.Package.Owner
var pd *packages_model.PackageDescriptor
err := db.WithTx(ctx, func(ctx stdctx.Context) error {
pv, err := packages_model.GetVersionByNameAndVersion(ctx, owner.ID, packages_model.TypeDebian, name, version)
if err != nil {
return err
}
pf, err := packages_model.GetFileForVersionByName(
ctx,
pv.ID,
fmt.Sprintf("%s_%s_%s.deb", name, version, architecture),
fmt.Sprintf("%s|%s", distribution, component),
)
if err != nil {
return err
}
if err := packages_service.DeletePackageFile(ctx, pf); err != nil {
return err
}
has, err := packages_model.HasVersionFileReferences(ctx, pv.ID)
if err != nil {
return err
}
if !has {
pd, err = packages_model.GetPackageDescriptor(ctx, pv)
if err != nil {
return err
}
if err := packages_service.DeletePackageVersionAndReferences(ctx, pv); err != nil {
return err
}
}
return nil
})
if err != nil {
if errors.Is(err, util.ErrNotExist) {
apiError(ctx, http.StatusNotFound, err)
} else {
apiError(ctx, http.StatusInternalServerError, err)
}
return
}
if pd != nil {
notification.NotifyPackageDelete(ctx, ctx.Doer, pd)
}
if err := debian_service.BuildSpecificRepositoryFiles(ctx, ctx.Package.Owner.ID, distribution, component, architecture); err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return
}
ctx.Status(http.StatusNoContent)
}

View file

@ -84,7 +84,7 @@ func UploadPackage(ctx *context.Context) {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
log.Error("Error creating hashed buffer: %v", err)
apiError(ctx, http.StatusInternalServerError, err)

View file

@ -155,7 +155,7 @@ func UploadPackage(ctx *context.Context) {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -245,7 +245,7 @@ func UploadPackageFile(ctx *context.Context) {
packageName := params.GroupID + "-" + params.ArtifactID
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(ctx.Req.Body)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -189,7 +189,7 @@ func UploadPackage(ctx *context.Context) {
}
}
buf, err := packages_module.CreateHashedBufferFromReader(bytes.NewReader(npmPackage.Data), 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(bytes.NewReader(npmPackage.Data))
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -475,7 +475,7 @@ func UploadSymbolPackage(ctx *context.Context) {
Version: np.Version,
}
_, _, err = packages_service.AddFileToExistingPackage(
_, err = packages_service.AddFileToExistingPackage(
pi,
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
@ -501,7 +501,7 @@ func UploadSymbolPackage(ctx *context.Context) {
}
for _, pdb := range pdbs {
_, _, err := packages_service.AddFileToExistingPackage(
_, err := packages_service.AddFileToExistingPackage(
pi,
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
@ -545,7 +545,7 @@ func processUploadedFile(ctx *context.Context, expectedType nuget_module.Package
closables = append(closables, upload)
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return nil, nil, closables

View file

@ -166,7 +166,7 @@ func UploadPackageFile(ctx *context.Context) {
}
defer file.Close()
buf, err := packages_module.CreateHashedBufferFromReader(file, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(file)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -117,7 +117,7 @@ func UploadPackageFile(ctx *context.Context) {
}
defer file.Close()
buf, err := packages_module.CreateHashedBufferFromReader(file, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(file)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -209,7 +209,7 @@ func UploadPackageFile(ctx *context.Context) {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -300,7 +300,7 @@ func UploadPackageFile(ctx *context.Context) {
}
defer file.Close()
buf, err := packages_module.CreateHashedBufferFromReader(file, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(file)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -158,7 +158,7 @@ func UploadPackageFile(ctx *context.Context) {
defer upload.Close()
}
buf, err := packages_module.CreateHashedBufferFromReader(upload, 32*1024*1024)
buf, err := packages_module.CreateHashedBufferFromReader(upload)
if err != nil {
apiError(ctx, http.StatusInternalServerError, err)
return

View file

@ -40,7 +40,7 @@ func ListPackages(ctx *context.APIContext) {
// in: query
// description: package type filter
// type: string
// enum: [cargo, chef, composer, conan, conda, container, generic, helm, maven, npm, nuget, pub, pypi, rubygems, swift, vagrant]
// enum: [cargo, chef, composer, conan, conda, container, debian, generic, helm, maven, npm, nuget, pub, pypi, rubygems, swift, vagrant]
// - name: q
// in: query
// description: name filter

View file

@ -14,8 +14,10 @@ import (
access_model "code.gitea.io/gitea/models/perm/access"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/container"
"code.gitea.io/gitea/modules/context"
"code.gitea.io/gitea/modules/log"
debian_module "code.gitea.io/gitea/modules/packages/debian"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/modules/web"
@ -163,6 +165,32 @@ func ViewPackageVersion(ctx *context.Context) {
ctx.Data["IsPackagesPage"] = true
ctx.Data["PackageDescriptor"] = pd
switch pd.Package.Type {
case packages_model.TypeContainer:
ctx.Data["RegistryHost"] = setting.Packages.RegistryHost
case packages_model.TypeDebian:
distributions := make(container.Set[string])
components := make(container.Set[string])
architectures := make(container.Set[string])
for _, f := range pd.Files {
for _, pp := range f.Properties {
switch pp.Name {
case debian_module.PropertyDistribution:
distributions.Add(pp.Value)
case debian_module.PropertyComponent:
components.Add(pp.Value)
case debian_module.PropertyArchitecture:
architectures.Add(pp.Value)
}
}
}
ctx.Data["Distributions"] = distributions.Values()
ctx.Data["Components"] = components.Values()
ctx.Data["Architectures"] = architectures.Values()
}
var (
total int64
pvs []*packages_model.PackageVersion
@ -170,8 +198,6 @@ func ViewPackageVersion(ctx *context.Context) {
)
switch pd.Package.Type {
case packages_model.TypeContainer:
ctx.Data["RegistryHost"] = setting.Packages.RegistryHost
pvs, total, err = container_model.SearchImageTags(ctx, &container_model.ImageTagsSearchOptions{
Paginator: db.NewAbsoluteListOptions(0, 5),
PackageID: pd.Package.ID,
@ -183,10 +209,6 @@ func ViewPackageVersion(ctx *context.Context) {
PackageID: pd.Package.ID,
IsInternal: util.OptionalBoolFalse,
})
if err != nil {
ctx.ServerError("SearchVersions", err)
return
}
}
if err != nil {
ctx.ServerError("", err)

View file

@ -15,7 +15,7 @@ import (
type PackageCleanupRuleForm struct {
ID int64
Enabled bool
Type string `binding:"Required;In(cargo,chef,composer,conan,conda,container,generic,helm,maven,npm,nuget,pub,pypi,rubygems,swift,vagrant)"`
Type string `binding:"Required;In(cargo,chef,composer,conan,conda,container,debian,generic,helm,maven,npm,nuget,pub,pypi,rubygems,swift,vagrant)"`
KeepCount int `binding:"In(0,1,5,10,25,50,100)"`
KeepPattern string `binding:"RegexPattern"`
RemoveDays int `binding:"In(0,7,14,30,60,90,180)"`

View file

@ -17,6 +17,7 @@ import (
packages_service "code.gitea.io/gitea/services/packages"
cargo_service "code.gitea.io/gitea/services/packages/cargo"
container_service "code.gitea.io/gitea/services/packages/container"
debian_service "code.gitea.io/gitea/services/packages/debian"
)
// Cleanup removes expired package data
@ -45,6 +46,7 @@ func Cleanup(taskCtx context.Context, olderThan time.Duration) error {
return fmt.Errorf("CleanupRule [%d]: GetPackagesByType failed: %w", pcr.ID, err)
}
anyVersionDeleted := false
for _, p := range packages {
pvs, _, err := packages_model.SearchVersions(ctx, &packages_model.PackageSearchOptions{
PackageID: p.ID,
@ -91,6 +93,7 @@ func Cleanup(taskCtx context.Context, olderThan time.Duration) error {
}
versionDeleted = true
anyVersionDeleted = true
}
if versionDeleted {
@ -105,6 +108,14 @@ func Cleanup(taskCtx context.Context, olderThan time.Duration) error {
}
}
}
if anyVersionDeleted {
if pcr.Type == packages_model.TypeDebian {
if err := debian_service.BuildAllRepositoryFiles(ctx, pcr.OwnerID); err != nil {
return fmt.Errorf("CleanupRule [%d]: debian.BuildAllRepositoryFiles failed: %w", pcr.ID, err)
}
}
}
return nil
})
if err != nil {

View file

@ -0,0 +1,443 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package debian
import (
"bytes"
"compress/gzip"
"context"
"errors"
"fmt"
"io"
"sort"
"strings"
"time"
"code.gitea.io/gitea/models/db"
packages_model "code.gitea.io/gitea/models/packages"
debian_model "code.gitea.io/gitea/models/packages/debian"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/log"
packages_module "code.gitea.io/gitea/modules/packages"
debian_module "code.gitea.io/gitea/modules/packages/debian"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
packages_service "code.gitea.io/gitea/services/packages"
"github.com/keybase/go-crypto/openpgp"
"github.com/keybase/go-crypto/openpgp/armor"
"github.com/keybase/go-crypto/openpgp/clearsign"
"github.com/keybase/go-crypto/openpgp/packet"
"github.com/ulikunitz/xz"
)
// GetOrCreateRepositoryVersion gets or creates the internal repository package
// The Debian registry needs multiple index files which are stored in this package.
func GetOrCreateRepositoryVersion(ownerID int64) (*packages_model.PackageVersion, error) {
var repositoryVersion *packages_model.PackageVersion
return repositoryVersion, db.WithTx(db.DefaultContext, func(ctx context.Context) error {
p := &packages_model.Package{
OwnerID: ownerID,
Type: packages_model.TypeDebian,
Name: debian_module.RepositoryPackage,
LowerName: debian_module.RepositoryPackage,
IsInternal: true,
}
var err error
if p, err = packages_model.TryInsertPackage(ctx, p); err != nil {
if err != packages_model.ErrDuplicatePackage {
log.Error("Error inserting package: %v", err)
return err
}
}
pv := &packages_model.PackageVersion{
PackageID: p.ID,
CreatorID: ownerID,
Version: debian_module.RepositoryVersion,
LowerVersion: debian_module.RepositoryVersion,
IsInternal: true,
MetadataJSON: "null",
}
if pv, err = packages_model.GetOrInsertVersion(ctx, pv); err != nil {
if err != packages_model.ErrDuplicatePackageVersion {
log.Error("Error inserting package version: %v", err)
return err
}
}
repositoryVersion = pv
return nil
})
}
// GetOrCreateKeyPair gets or creates the PGP keys used to sign repository files
func GetOrCreateKeyPair(ownerID int64) (string, string, error) {
priv, err := user_model.GetSetting(ownerID, debian_module.SettingKeyPrivate)
if err != nil && !errors.Is(err, util.ErrNotExist) {
return "", "", err
}
pub, err := user_model.GetSetting(ownerID, debian_module.SettingKeyPublic)
if err != nil && !errors.Is(err, util.ErrNotExist) {
return "", "", err
}
if priv == "" || pub == "" {
priv, pub, err = generateKeypair()
if err != nil {
return "", "", err
}
if err := user_model.SetUserSetting(ownerID, debian_module.SettingKeyPrivate, priv); err != nil {
return "", "", err
}
if err := user_model.SetUserSetting(ownerID, debian_module.SettingKeyPublic, pub); err != nil {
return "", "", err
}
}
return priv, pub, nil
}
func generateKeypair() (string, string, error) {
e, err := openpgp.NewEntity(setting.AppName, "Debian Registry", "", nil)
if err != nil {
return "", "", err
}
var priv strings.Builder
var pub strings.Builder
w, err := armor.Encode(&priv, openpgp.PrivateKeyType, nil)
if err != nil {
return "", "", err
}
if err := e.SerializePrivate(w, nil); err != nil {
return "", "", err
}
w.Close()
w, err = armor.Encode(&pub, openpgp.PublicKeyType, nil)
if err != nil {
return "", "", err
}
if err := e.Serialize(w); err != nil {
return "", "", err
}
w.Close()
return priv.String(), pub.String(), nil
}
// BuildAllRepositoryFiles (re)builds all repository files for every available distributions, components and architectures
func BuildAllRepositoryFiles(ctx context.Context, ownerID int64) error {
pv, err := GetOrCreateRepositoryVersion(ownerID)
if err != nil {
return err
}
// 1. Delete all existing repository files
pfs, err := packages_model.GetFilesByVersionID(ctx, pv.ID)
if err != nil {
return err
}
for _, pf := range pfs {
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil {
return err
}
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil {
return err
}
}
// 2. (Re)Build repository files for existing packages
distributions, err := debian_model.GetDistributions(ctx, ownerID)
if err != nil {
return err
}
for _, distribution := range distributions {
components, err := debian_model.GetComponents(ctx, ownerID, distribution)
if err != nil {
return err
}
architectures, err := debian_model.GetArchitectures(ctx, ownerID, distribution)
if err != nil {
return err
}
for _, component := range components {
for _, architecture := range architectures {
if err := buildRepositoryFiles(ctx, ownerID, pv, distribution, component, architecture); err != nil {
return fmt.Errorf("failed to build repository files [%s/%s/%s]: %w", distribution, component, architecture, err)
}
}
}
}
return nil
}
// BuildSpecificRepositoryFiles builds index files for the repository
func BuildSpecificRepositoryFiles(ctx context.Context, ownerID int64, distribution, component, architecture string) error {
pv, err := GetOrCreateRepositoryVersion(ownerID)
if err != nil {
return err
}
return buildRepositoryFiles(ctx, ownerID, pv, distribution, component, architecture)
}
func buildRepositoryFiles(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution, component, architecture string) error {
if err := buildPackagesIndices(ctx, ownerID, repoVersion, distribution, component, architecture); err != nil {
return err
}
return buildReleaseFiles(ctx, ownerID, repoVersion, distribution)
}
// https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices
func buildPackagesIndices(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution, component, architecture string) error {
pfds, err := debian_model.SearchLatestPackages(ctx, &debian_model.PackageSearchOptions{
OwnerID: ownerID,
Distribution: distribution,
Component: component,
Architecture: architecture,
})
if err != nil {
return err
}
// Delete the package indices if there are no packages
if len(pfds) == 0 {
key := fmt.Sprintf("%s|%s|%s", distribution, component, architecture)
for _, filename := range []string{"Packages", "Packages.gz", "Packages.xz"} {
pf, err := packages_model.GetFileForVersionByName(ctx, repoVersion.ID, filename, key)
if err != nil && !errors.Is(err, util.ErrNotExist) {
return err
}
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil {
return err
}
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil {
return err
}
}
return nil
}
packagesContent, _ := packages_module.NewHashedBuffer()
packagesGzipContent, _ := packages_module.NewHashedBuffer()
gzw := gzip.NewWriter(packagesGzipContent)
packagesXzContent, _ := packages_module.NewHashedBuffer()
xzw, _ := xz.NewWriter(packagesXzContent)
w := io.MultiWriter(packagesContent, gzw, xzw)
addSeperator := false
for _, pfd := range pfds {
if addSeperator {
fmt.Fprintln(w)
}
addSeperator = true
fmt.Fprint(w, pfd.Properties.GetByName(debian_module.PropertyControl))
fmt.Fprintf(w, "Filename: pool/%s/%s/%s\n", distribution, component, pfd.File.Name)
fmt.Fprintf(w, "Size: %d\n", pfd.Blob.Size)
fmt.Fprintf(w, "MD5sum: %s\n", pfd.Blob.HashMD5)
fmt.Fprintf(w, "SHA1: %s\n", pfd.Blob.HashSHA1)
fmt.Fprintf(w, "SHA256: %s\n", pfd.Blob.HashSHA256)
fmt.Fprintf(w, "SHA512: %s\n", pfd.Blob.HashSHA512)
}
gzw.Close()
xzw.Close()
for _, file := range []struct {
Name string
Data packages_module.HashedSizeReader
}{
{"Packages", packagesContent},
{"Packages.gz", packagesGzipContent},
{"Packages.xz", packagesXzContent},
} {
_, err = packages_service.AddFileToPackageVersionInternal(
repoVersion,
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
Filename: file.Name,
CompositeKey: fmt.Sprintf("%s|%s|%s", distribution, component, architecture),
},
Creator: user_model.NewGhostUser(),
Data: file.Data,
IsLead: false,
OverwriteExisting: true,
Properties: map[string]string{
debian_module.PropertyRepositoryIncludeInRelease: "",
debian_module.PropertyDistribution: distribution,
debian_module.PropertyComponent: component,
debian_module.PropertyArchitecture: architecture,
},
},
)
if err != nil {
return err
}
}
return nil
}
// https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files
func buildReleaseFiles(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution string) error {
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{
VersionID: repoVersion.ID,
Properties: map[string]string{
debian_module.PropertyRepositoryIncludeInRelease: "",
debian_module.PropertyDistribution: distribution,
},
})
if err != nil {
return err
}
// Delete the release files if there are no packages
if len(pfs) == 0 {
for _, filename := range []string{"Release", "Release.gpg", "InRelease"} {
pf, err := packages_model.GetFileForVersionByName(ctx, repoVersion.ID, filename, distribution)
if err != nil && !errors.Is(err, util.ErrNotExist) {
return err
}
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil {
return err
}
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil {
return err
}
}
return nil
}
components, err := debian_model.GetComponents(ctx, ownerID, distribution)
if err != nil {
return err
}
sort.Strings(components)
architectures, err := debian_model.GetArchitectures(ctx, ownerID, distribution)
if err != nil {
return err
}
sort.Strings(architectures)
priv, _, err := GetOrCreateKeyPair(ownerID)
if err != nil {
return err
}
block, err := armor.Decode(strings.NewReader(priv))
if err != nil {
return err
}
e, err := openpgp.ReadEntity(packet.NewReader(block.Body))
if err != nil {
return err
}
inReleaseContent, _ := packages_module.NewHashedBuffer()
sw, err := clearsign.Encode(inReleaseContent, e.PrivateKey, nil)
if err != nil {
return err
}
var buf bytes.Buffer
w := io.MultiWriter(sw, &buf)
fmt.Fprintf(w, "Origin: %s\n", setting.AppName)
fmt.Fprintf(w, "Label: %s\n", setting.AppName)
fmt.Fprintf(w, "Suite: %s\n", distribution)
fmt.Fprintf(w, "Codename: %s\n", distribution)
fmt.Fprintf(w, "Components: %s\n", strings.Join(components, " "))
fmt.Fprintf(w, "Architectures: %s\n", strings.Join(architectures, " "))
fmt.Fprintf(w, "Date: %s\n", time.Now().UTC().Format(time.RFC1123))
fmt.Fprint(w, "Acquire-By-Hash: yes")
pfds, err := packages_model.GetPackageFileDescriptors(ctx, pfs)
if err != nil {
return err
}
var md5, sha1, sha256, sha512 strings.Builder
for _, pfd := range pfds {
path := fmt.Sprintf("%s/binary-%s/%s", pfd.Properties.GetByName(debian_module.PropertyComponent), pfd.Properties.GetByName(debian_module.PropertyArchitecture), pfd.File.Name)
fmt.Fprintf(&md5, " %s %d %s\n", pfd.Blob.HashMD5, pfd.Blob.Size, path)
fmt.Fprintf(&sha1, " %s %d %s\n", pfd.Blob.HashSHA1, pfd.Blob.Size, path)
fmt.Fprintf(&sha256, " %s %d %s\n", pfd.Blob.HashSHA256, pfd.Blob.Size, path)
fmt.Fprintf(&sha512, " %s %d %s\n", pfd.Blob.HashSHA512, pfd.Blob.Size, path)
}
fmt.Fprintln(w, "MD5Sum:")
fmt.Fprint(w, md5.String())
fmt.Fprintln(w, "SHA1:")
fmt.Fprint(w, sha1.String())
fmt.Fprintln(w, "SHA256:")
fmt.Fprint(w, sha256.String())
fmt.Fprintln(w, "SHA512:")
fmt.Fprint(w, sha512.String())
sw.Close()
releaseGpgContent, _ := packages_module.NewHashedBuffer()
if err := openpgp.ArmoredDetachSign(releaseGpgContent, e, bytes.NewReader(buf.Bytes()), nil); err != nil {
return err
}
releaseContent, _ := packages_module.CreateHashedBufferFromReader(&buf)
for _, file := range []struct {
Name string
Data packages_module.HashedSizeReader
}{
{"Release", releaseContent},
{"Release.gpg", releaseGpgContent},
{"InRelease", inReleaseContent},
} {
_, err = packages_service.AddFileToPackageVersionInternal(
repoVersion,
&packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{
Filename: file.Name,
CompositeKey: distribution,
},
Creator: user_model.NewGhostUser(),
Data: file.Data,
IsLead: false,
OverwriteExisting: true,
Properties: map[string]string{
debian_module.PropertyDistribution: distribution,
},
},
)
if err != nil {
return err
}
}
return nil
}

View file

@ -187,19 +187,33 @@ func createPackageAndVersion(ctx context.Context, pvci *PackageCreationInfo, all
}
// AddFileToExistingPackage adds a file to an existing package. If the package does not exist, ErrPackageNotExist is returned
func AddFileToExistingPackage(pvi *PackageInfo, pfci *PackageFileCreationInfo) (*packages_model.PackageVersion, *packages_model.PackageFile, error) {
func AddFileToExistingPackage(pvi *PackageInfo, pfci *PackageFileCreationInfo) (*packages_model.PackageFile, error) {
return addFileToPackageWrapper(func(ctx context.Context) (*packages_model.PackageFile, *packages_model.PackageBlob, bool, error) {
pv, err := packages_model.GetVersionByNameAndVersion(ctx, pvi.Owner.ID, pvi.PackageType, pvi.Name, pvi.Version)
if err != nil {
return nil, nil, false, err
}
return addFileToPackageVersion(ctx, pv, pvi, pfci)
})
}
// AddFileToPackageVersionInternal adds a file to the package
// This method skips quota checks and should only be used for system-managed packages.
func AddFileToPackageVersionInternal(pv *packages_model.PackageVersion, pfci *PackageFileCreationInfo) (*packages_model.PackageFile, error) {
return addFileToPackageWrapper(func(ctx context.Context) (*packages_model.PackageFile, *packages_model.PackageBlob, bool, error) {
return addFileToPackageVersionUnchecked(ctx, pv, pfci)
})
}
func addFileToPackageWrapper(fn func(ctx context.Context) (*packages_model.PackageFile, *packages_model.PackageBlob, bool, error)) (*packages_model.PackageFile, error) {
ctx, committer, err := db.TxContext(db.DefaultContext)
if err != nil {
return nil, nil, err
return nil, err
}
defer committer.Close()
pv, err := packages_model.GetVersionByNameAndVersion(ctx, pvi.Owner.ID, pvi.PackageType, pvi.Name, pvi.Version)
if err != nil {
return nil, nil, err
}
pf, pb, blobCreated, err := addFileToPackageVersion(ctx, pv, pvi, pfci)
pf, pb, blobCreated, err := fn(ctx)
removeBlob := false
defer func() {
if removeBlob {
@ -211,15 +225,15 @@ func AddFileToExistingPackage(pvi *PackageInfo, pfci *PackageFileCreationInfo) (
}()
if err != nil {
removeBlob = blobCreated
return nil, nil, err
return nil, err
}
if err := committer.Commit(); err != nil {
removeBlob = blobCreated
return nil, nil, err
return nil, err
}
return pv, pf, nil
return pf, nil
}
// NewPackageBlob creates a package blob instance
@ -236,12 +250,16 @@ func NewPackageBlob(hsr packages_module.HashedSizeReader) *packages_model.Packag
}
func addFileToPackageVersion(ctx context.Context, pv *packages_model.PackageVersion, pvi *PackageInfo, pfci *PackageFileCreationInfo) (*packages_model.PackageFile, *packages_model.PackageBlob, bool, error) {
log.Trace("Adding package file: %v, %s", pv.ID, pfci.Filename)
if err := CheckSizeQuotaExceeded(ctx, pfci.Creator, pvi.Owner, pvi.PackageType, pfci.Data.Size()); err != nil {
return nil, nil, false, err
}
return addFileToPackageVersionUnchecked(ctx, pv, pfci)
}
func addFileToPackageVersionUnchecked(ctx context.Context, pv *packages_model.PackageVersion, pfci *PackageFileCreationInfo) (*packages_model.PackageFile, *packages_model.PackageBlob, bool, error) {
log.Trace("Adding package file: %v, %s", pv.ID, pfci.Filename)
pb, exists, err := packages_model.GetOrInsertBlob(ctx, NewPackageBlob(pfci.Data))
if err != nil {
log.Error("Error inserting package blob: %v", err)
@ -345,6 +363,8 @@ func CheckSizeQuotaExceeded(ctx context.Context, doer, owner *user_model.User, p
typeSpecificSize = setting.Packages.LimitSizeConda
case packages_model.TypeContainer:
typeSpecificSize = setting.Packages.LimitSizeContainer
case packages_model.TypeDebian:
typeSpecificSize = setting.Packages.LimitSizeDebian
case packages_model.TypeGeneric:
typeSpecificSize = setting.Packages.LimitSizeGeneric
case packages_model.TypeHelm:

View file

@ -0,0 +1,65 @@
{{if eq .PackageDescriptor.Package.Type "debian"}}
<h4 class="ui top attached header">{{.locale.Tr "packages.installation"}}</h4>
<div class="ui attached segment">
<div class="ui form">
<div class="field">
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.debian.registry"}}</label>
<div class="markup"><pre class="code-block"><code>sudo curl <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/debian/repository.key"></gitea-origin-url> -o /etc/apt/trusted.gpg.d/gitea-{{$.PackageDescriptor.Owner.Name}}.asc
echo "deb <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/debian"></gitea-origin-url> &lt;distribution&gt; &lt;component&gt;" | sudo tee -a /etc/apt/sources.list.d/gitea.list
sudo apt update</code></pre></div>
<p>{{.locale.Tr "packages.debian.registry.info" | Safe}}</p>
</div>
<div class="field">
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.debian.install"}}</label>
<div class="markup">
<pre class="code-block"><code>sudo apt install {{$.PackageDescriptor.Package.Name}}={{$.PackageDescriptor.Version.Version}}</code></pre>
</div>
</div>
<div class="field">
<label>{{.locale.Tr "packages.debian.documentation" | Safe}}</label>
</div>
</div>
</div>
<h4 class="ui top attached header">{{.locale.Tr "packages.debian.repository"}}</h4>
<div class="ui attached segment">
<table class="ui single line very basic table">
<tbody>
<tr>
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.distributions"}}</h5></td>
<td>{{Join .Distributions ", "}}</td>
</tr>
<tr>
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.components"}}</h5></td>
<td>{{Join .Components ", "}}</td>
</tr>
<tr>
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.architectures"}}</h5></td>
<td>{{Join .Architectures ", "}}</td>
</tr>
</tbody>
</table>
</div>
{{if .PackageDescriptor.Metadata.Description}}
<h4 class="ui top attached header">{{.locale.Tr "packages.about"}}</h4>
<div class="ui attached segment">
{{.PackageDescriptor.Metadata.Description}}
</div>
{{end}}
{{if .PackageDescriptor.Metadata.Dependencies}}
<h4 class="ui top attached header">{{.locale.Tr "packages.dependencies"}}</h4>
<div class="ui attached segment">
<table class="ui single line very basic table">
<tbody>
{{range .PackageDescriptor.Metadata.Dependencies}}
<tr>
<td>{{.}}</td>
</tr>
{{end}}
</tbody>
</table>
</div>
{{end}}
{{end}}

View file

@ -0,0 +1,4 @@
{{if eq .PackageDescriptor.Package.Type "debian"}}
{{if .PackageDescriptor.Metadata.Maintainer}}<div class="item" title="{{.locale.Tr "packages.details.author"}}">{{svg "octicon-person" 16 "mr-3"}} {{.PackageDescriptor.Metadata.Maintainer}}</div>{{end}}
{{if .PackageDescriptor.Metadata.ProjectURL}}<div class="item">{{svg "octicon-link-external" 16 "mr-3"}} <a href="{{.PackageDescriptor.Metadata.ProjectURL}}" target="_blank" rel="noopener noreferrer me">{{.locale.Tr "packages.details.project_site"}}</a></div>{{end}}
{{end}}

View file

@ -25,6 +25,7 @@
{{template "package/content/conan" .}}
{{template "package/content/conda" .}}
{{template "package/content/container" .}}
{{template "package/content/debian" .}}
{{template "package/content/generic" .}}
{{template "package/content/helm" .}}
{{template "package/content/maven" .}}
@ -52,6 +53,7 @@
{{template "package/metadata/conan" .}}
{{template "package/metadata/conda" .}}
{{template "package/metadata/container" .}}
{{template "package/metadata/debian" .}}
{{template "package/metadata/generic" .}}
{{template "package/metadata/helm" .}}
{{template "package/metadata/maven" .}}

View file

@ -2415,6 +2415,7 @@
"conan",
"conda",
"container",
"debian",
"generic",
"helm",
"maven",

View file

@ -0,0 +1,252 @@
// Copyright 2023 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package integration
import (
"archive/tar"
"bytes"
"compress/gzip"
"fmt"
"io"
"net/http"
"strings"
"testing"
"code.gitea.io/gitea/models/db"
"code.gitea.io/gitea/models/packages"
"code.gitea.io/gitea/models/unittest"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/base"
debian_module "code.gitea.io/gitea/modules/packages/debian"
"code.gitea.io/gitea/tests"
"github.com/blakesmith/ar"
"github.com/stretchr/testify/assert"
)
func TestPackageDebian(t *testing.T) {
defer tests.PrepareTestEnv(t)()
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
packageName := "gitea"
packageVersion := "1.0.3"
packageDescription := "Package Description"
createArchive := func(name, version, architecture string) io.Reader {
var cbuf bytes.Buffer
zw := gzip.NewWriter(&cbuf)
tw := tar.NewWriter(zw)
tw.WriteHeader(&tar.Header{
Name: "control",
Mode: 0o600,
Size: 50,
})
fmt.Fprintf(tw, "Package: %s\nVersion: %s\nArchitecture: %s\nDescription: %s\n", name, version, architecture, packageDescription)
tw.Close()
zw.Close()
var buf bytes.Buffer
aw := ar.NewWriter(&buf)
aw.WriteGlobalHeader()
hdr := &ar.Header{
Name: "control.tar.gz",
Mode: 0o600,
Size: int64(cbuf.Len()),
}
aw.WriteHeader(hdr)
aw.Write(cbuf.Bytes())
return &buf
}
distributions := []string{"test", "gitea"}
components := []string{"main", "stable"}
architectures := []string{"all", "amd64"}
rootURL := fmt.Sprintf("/api/packages/%s/debian", user.Name)
t.Run("RepositoryKey", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", rootURL+"/repository.key")
resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, "application/pgp-keys", resp.Header().Get("Content-Type"))
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP PUBLIC KEY BLOCK-----")
})
for _, distribution := range distributions {
t.Run(fmt.Sprintf("[Distribution:%s]", distribution), func(t *testing.T) {
for _, component := range components {
for _, architecture := range architectures {
t.Run(fmt.Sprintf("[Component:%s,Architecture:%s]", component, architecture), func(t *testing.T) {
t.Run("Upload", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
uploadURL := fmt.Sprintf("%s/pool/%s/%s/upload", rootURL, distribution, component)
req := NewRequestWithBody(t, "PUT", uploadURL, bytes.NewReader([]byte{}))
MakeRequest(t, req, http.StatusUnauthorized)
req = NewRequestWithBody(t, "PUT", uploadURL, bytes.NewReader([]byte{}))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusBadRequest)
req = NewRequestWithBody(t, "PUT", uploadURL, createArchive("", "", ""))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusBadRequest)
req = NewRequestWithBody(t, "PUT", uploadURL, createArchive(packageName, packageVersion, architecture))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusCreated)
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeDebian)
assert.NoError(t, err)
assert.Len(t, pvs, 1)
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pvs[0])
assert.NoError(t, err)
assert.Nil(t, pd.SemVer)
assert.IsType(t, &debian_module.Metadata{}, pd.Metadata)
assert.Equal(t, packageName, pd.Package.Name)
assert.Equal(t, packageVersion, pd.Version.Version)
pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID)
assert.NoError(t, err)
assert.NotEmpty(t, pfs)
assert.Condition(t, func() bool {
seen := false
expectedFilename := fmt.Sprintf("%s_%s_%s.deb", packageName, packageVersion, architecture)
expectedCompositeKey := fmt.Sprintf("%s|%s", distribution, component)
for _, pf := range pfs {
if pf.Name == expectedFilename && pf.CompositeKey == expectedCompositeKey {
if seen {
return false
}
seen = true
assert.True(t, pf.IsLead)
pfps, err := packages.GetProperties(db.DefaultContext, packages.PropertyTypeFile, pf.ID)
assert.NoError(t, err)
for _, pfp := range pfps {
switch pfp.Name {
case debian_module.PropertyDistribution:
assert.Equal(t, distribution, pfp.Value)
case debian_module.PropertyComponent:
assert.Equal(t, component, pfp.Value)
case debian_module.PropertyArchitecture:
assert.Equal(t, architecture, pfp.Value)
}
}
}
}
return seen
})
})
t.Run("Download", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/pool/%s/%s/%s_%s_%s.deb", rootURL, distribution, component, packageName, packageVersion, architecture))
resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, "application/vnd.debian.binary-package", resp.Header().Get("Content-Type"))
})
t.Run("Packages", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
url := fmt.Sprintf("%s/dists/%s/%s/binary-%s/Packages", rootURL, distribution, component, architecture)
req := NewRequest(t, "GET", url)
resp := MakeRequest(t, req, http.StatusOK)
body := resp.Body.String()
assert.Contains(t, body, "Package: "+packageName)
assert.Contains(t, body, "Version: "+packageVersion)
assert.Contains(t, body, "Architecture: "+architecture)
assert.Contains(t, body, fmt.Sprintf("Filename: pool/%s/%s/%s_%s_%s.deb", distribution, component, packageName, packageVersion, architecture))
req = NewRequest(t, "GET", url+".gz")
MakeRequest(t, req, http.StatusOK)
req = NewRequest(t, "GET", url+".xz")
MakeRequest(t, req, http.StatusOK)
url = fmt.Sprintf("%s/dists/%s/%s/%s/by-hash/SHA256/%s", rootURL, distribution, component, architecture, base.EncodeSha256(body))
req = NewRequest(t, "GET", url)
resp = MakeRequest(t, req, http.StatusOK)
assert.Equal(t, body, resp.Body.String())
})
})
}
}
t.Run("Release", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release", rootURL, distribution))
resp := MakeRequest(t, req, http.StatusOK)
body := resp.Body.String()
assert.Contains(t, body, "Components: "+strings.Join(components, " "))
assert.Contains(t, body, "Architectures: "+strings.Join(architectures, " "))
for _, component := range components {
for _, architecture := range architectures {
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages", component, architecture))
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages.gz", component, architecture))
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages.xz", component, architecture))
}
}
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/by-hash/SHA256/%s", rootURL, distribution, base.EncodeSha256(body)))
resp = MakeRequest(t, req, http.StatusOK)
assert.Equal(t, body, resp.Body.String())
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release.gpg", rootURL, distribution))
resp = MakeRequest(t, req, http.StatusOK)
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP SIGNATURE-----")
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/InRelease", rootURL, distribution))
resp = MakeRequest(t, req, http.StatusOK)
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP SIGNED MESSAGE-----")
})
})
}
t.Run("Delete", func(t *testing.T) {
defer tests.PrintCurrentTest(t)()
distribution := distributions[0]
architecture := architectures[0]
for _, component := range components {
req := NewRequest(t, "DELETE", fmt.Sprintf("%s/pool/%s/%s/%s/%s/%s", rootURL, distribution, component, packageName, packageVersion, architecture))
MakeRequest(t, req, http.StatusUnauthorized)
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/pool/%s/%s/%s/%s/%s", rootURL, distribution, component, packageName, packageVersion, architecture))
AddBasicAuthHeader(req, user.Name)
MakeRequest(t, req, http.StatusNoContent)
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/%s/binary-%s/Packages", rootURL, distribution, component, architecture))
MakeRequest(t, req, http.StatusNotFound)
}
req := NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release", rootURL, distribution))
resp := MakeRequest(t, req, http.StatusOK)
body := resp.Body.String()
assert.Contains(t, body, "Components: "+strings.Join(components, " "))
assert.Contains(t, body, "Architectures: "+architectures[1])
})
}

View file

@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="16" height="16" viewBox="0 0 210 260" xmlns="http://www.w3.org/2000/svg">
<g transform="translate(60,75)" fill="#D70751">
<path d="M64.525 62.053c-4.125.058.78 2.125 6.165 2.954 1.488-1.161 2.838-2.336 4.04-3.479-3.354.821-6.765.838-10.205.525m22.14-5.52c2.457-3.389 4.246-7.102 4.878-10.939-.551 2.736-2.035 5.099-3.435 7.592-7.711 4.854-.726-2.883-.004-5.824-8.29 10.436-1.138 6.257-1.439 9.171m8.174-21.265c.497-7.428-1.462-5.08-2.121-2.245.766.4 1.377 5.237 2.121 2.245M48.883-66.264c2.201.395 4.757.698 4.398 1.224 2.407-.528 2.954-1.015-4.398-1.224"/>
<path d="m53.281-65.04-1.556.32 1.448-.127.108-.193"/>
<path d="M121.93 38.085c.247 6.671-1.95 9.907-3.932 15.637l-3.564 1.781c-2.919 5.666.282 3.598-1.807 8.105-4.556 4.049-13.823 12.67-16.789 13.457-2.163-.047 1.469-2.554 1.943-3.537-6.097 4.188-4.894 6.285-14.217 8.83l-.273-.607C60.29 92.569 28.344 71.129 28.765 41.875c-.246 1.857-.698 1.393-1.208 2.144-1.186-15.052 6.952-30.17 20.675-36.343 13.427-6.646 29.163-3.918 38.78 5.044C81.73 5.8 71.217-1.534 58.757-.848c-12.208.193-23.625 7.95-27.436 16.369-6.253 3.938-6.979 15.177-9.704 17.233-3.665 26.943 6.896 38.583 24.762 52.275 2.812 1.896.792 2.184 1.173 3.627-5.936-2.779-11.372-6.976-15.841-12.114 2.372 3.473 4.931 6.847 8.239 9.499-5.596-1.897-13.074-13.563-15.256-14.038 9.647 17.274 39.142 30.295 54.587 23.836-7.146.263-16.226.146-24.256-2.822-3.371-1.734-7.958-5.331-7.14-6.003 21.079 7.875 42.854 5.965 61.09-8.655 4.641-3.614 9.709-9.761 11.173-9.846-2.206 3.317.377 1.596-1.318 4.523 4.625-7.456-2.008-3.035 4.779-12.877l2.507 3.453c-.931-6.188 7.687-13.704 6.813-23.492 1.975-2.994 2.206 3.22.107 10.107 2.912-7.64.767-8.867 1.516-15.171.81 2.118 1.867 4.37 2.412 6.606-1.895-7.382 1.948-12.433 2.898-16.724-.937-.415-2.928 3.264-3.383-5.457.065-3.788 1.054-1.985 1.435-2.917-.744-.427-2.694-3.33-3.88-8.9.86-1.308 2.3 3.393 3.47 3.586-.753-4.429-2.049-7.805-2.103-11.202-3.421-7.149-1.211.953-3.985-3.069-3.641-11.357 3.021-2.637 3.47-7.796 5.52 7.995 8.667 20.387 10.11 25.519-1.103-6.258-2.883-12.32-5.058-18.185 1.677.705-2.699-12.875 2.18-3.882-5.21-19.172-22.302-37.087-38.025-45.493 1.924 1.76 4.354 3.971 3.481 4.317-7.819-4.656-6.444-5.018-7.565-6.985-6.369-2.591-6.788.208-11.007.004-12.005-6.368-14.318-5.69-25.368-9.681l.502 2.349c-7.953-2.649-9.265 1.005-17.862.009-.523-.409 2.753-1.479 5.452-1.871-7.69 1.015-7.329-1.515-14.854.279 1.855-1.301 3.815-2.162 5.793-3.269-6.271.381-14.971 3.649-12.286.677C20.144-62.46 1.976-56.053-8.218-46.494l-.321-2.142c-4.672 5.608-20.371 16.748-21.622 24.011l-1.249.291c-2.431 4.116-4.004 8.781-5.932 13.016-3.18 5.417-4.661 2.085-4.208 2.934-6.253 12.679-9.359 23.332-12.043 32.069 1.912 2.858.046 17.206.769 28.688-3.141 56.709 39.8 111.77 86.737 124.48 6.88 2.459 17.11 2.364 25.813 2.618-10.268-2.937-11.595-1.556-21.595-5.044-7.215-3.398-8.797-7.277-13.907-11.711l2.022 3.573c-10.021-3.547-5.829-4.39-13.982-6.972l2.16-2.82c-3.249-.246-8.604-5.475-10.069-8.371l-3.553.14c-4.27-5.269-6.545-9.063-6.379-12.005l-1.148 2.047c-1.301-2.235-15.709-19.759-8.234-15.679-1.389-1.271-3.235-2.067-5.237-5.703l1.522-1.739c-3.597-4.627-6.621-10.562-6.391-12.536 1.919 2.592 3.25 3.075 4.568 3.52-9.083-22.539-9.593-1.242-16.474-22.942l1.456-.116c-1.116-1.682-1.793-3.506-2.69-5.298l.633-6.313c-6.541-7.562-1.829-32.151-.887-45.637.655-5.485 5.459-11.322 9.114-20.477l-2.227-.384C-27.316-2.419-7.271-24.81 2.011-23.658c4.499-5.649-.892-.02-1.772-1.443 9.878-10.223 12.984-7.222 19.65-9.061 7.19-4.268-6.17 1.664-2.761-1.628 12.427-3.174 8.808-7.216 25.021-8.828 1.71.973-3.969 1.503-5.395 2.766 10.354-5.066 32.769-3.914 47.326 2.811 16.895 7.896 35.873 31.232 36.622 53.189l.852.229c-.431 8.729 1.336 18.822-1.727 28.094l2.1-4.385"/>
<path d="m19.5 67.715-.578 2.893c2.71 3.683 4.861 7.673 8.323 10.552-2.49-4.863-4.341-6.872-7.745-13.445m6.409-.251c-1.435-1.587-2.284-3.497-3.235-5.4.909 3.345 2.771 6.219 4.504 9.143l-1.269-3.743m113.411-24.65-.605 1.52c-1.111 7.892-3.511 15.701-7.189 22.941 4.06-7.639 6.69-15.995 7.79-24.461M49.698-68.243c2.789-1.022 6.855-.56 9.814-1.233-3.855.324-7.693.517-11.484 1.005l1.67.228m-97.917 52.067c.642 5.951-4.477 8.26 1.134 4.337 3.007-6.773-1.175-1.87-1.134-4.337m-6.593 27.538c1.292-3.967 1.526-6.349 2.02-8.645-3.571 4.566-1.643 5.539-2.02 8.645"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.2 KiB