Commit graph

17 commits

Author SHA1 Message Date
Jan Provaznik
239fdc78b1 Use FastDestroy for deleting uploads
It gathers list of file paths to delete before destroying
the parent object. Then after the parent_object is destroyed
these paths are scheduled for deletion asynchronously.

Carrierwave needed associated model for deleting upload file.
To avoid this requirement, simple Fog/File layer is used directly
for file deletion, this allows us to use just a simple list of paths.
2018-12-06 22:00:19 +01:00
Toon Claes
149b632722 Backport changes from EE
Now the files are identical again compared to EE.

These are backported from
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5050
2018-11-07 11:29:31 +01:00
Toon Claes
1c481b7aac Enhance performance of counting local Uploads
Add an index to the `store` column on `uploads`. This makes counting
local uploads faster.

Also, there is no longer need to check for objects with `store = NULL`.
See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18557

---

### Query plans

Query:

```sql
SELECT COUNT(*)
FROM "uploads"
WHERE ("uploads"."store" = ? OR "uploads"."store" IS NULL)
```

#### Without index

```
gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                  QUERY PLAN
---------------------------------------------------------------------------------------------------------------
 Seq Scan on uploads  (cost=0.00..601729.54 rows=578 width=272) (actual time=6.170..2308.256 rows=545 loops=1)
   Filter: ((store = 1) OR (store IS NULL))
   Rows Removed by Filter: 4411957
 Planning time: 38.652 ms
 Execution time: 2308.454 ms
(5 rows)
```

#### Add index

```
gitlabhq_production=# create index uploads_tmp1 on uploads (store);
CREATE INDEX
```

#### With index

```
gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL);
                                                          QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on uploads  (cost=11.46..1238.88 rows=574 width=272) (actual time=0.155..0.577 rows=545 loops=1)
   Recheck Cond: ((store = 1) OR (store IS NULL))
   Heap Blocks: exact=217
   ->  BitmapOr  (cost=11.46..11.46 rows=574 width=0) (actual time=0.116..0.116 rows=0 loops=1)
         ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..8.74 rows=574 width=0) (actual time=0.095..0.095 rows=545 loops=1)
               Index Cond: (store = 1)
         ->  Bitmap Index Scan on uploads_tmp1  (cost=0.00..2.44 rows=1 width=0) (actual time=0.020..0.020 rows=0 loops=1)
               Index Cond: (store IS NULL)
 Planning time: 0.274 ms
 Execution time: 0.637 ms
(10 rows)
```

Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6070
2018-11-07 11:29:31 +01:00
gfyoung
50abbd3e53 Enable frozen string in app/models/*.rb
Partially addresses #47424.
2018-07-26 16:55:41 -07:00
Micaël Bergeron
3b69e41018 remove EE-only code 2018-03-22 13:00:58 -04:00
Micaël Bergeron
44f37504fb Backport ee-40781-os-to-ce 2018-03-22 08:49:04 -04:00
Micaël Bergeron
0f1d348d68 port the object storage to CE 2018-03-01 10:34:30 -05:00
Kamil Trzciński
79a5e7fb53 Merge commit '7e424eb852716495073881710e8a8851b4a4cd5a' into object-storage-ee-to-ce-backport 2018-02-28 21:14:25 +01:00
Sean McGivern
a7dae52e9d Merge branch '4163-move-uploads-to-object-storage' into 'master'
Move uploads to object storage

Closes #4163

See merge request gitlab-org/gitlab-ee!3867
2018-02-28 20:58:15 +01:00
Micaël Bergeron
939391af7b improvements from feedback 2018-02-06 08:48:56 -05:00
Micaël Bergeron
d84650917a remove file after Upload#destroy
it will also automatically prune empty directories for
`FileUploader`-based uploaders.
2018-02-06 08:48:35 -05:00
Micaël Bergeron
ce84d18353 apply fixes from feedback 2018-02-02 09:32:20 -05:00
Micaël Bergeron
74ddc80590 add the uploader context to the upload model 2018-02-02 09:28:15 -05:00
Micaël Bergeron
e3d32e8a26 fix specs 2018-02-01 12:14:46 -05:00
Micaël Bergeron
2057a6acde port of 594e6a0a625^..f74c90f68c6 2018-02-01 12:14:46 -05:00
Yorick Peterse
5819ca1a24
Added Cop to blacklist polymorphic associations
One should really use a separate table instead of using polymorphic
associations.

See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11168 for
more information.
2017-06-07 17:36:55 +02:00
Robert Speicher
4c622b71fd Add Upload model and UploadChecksumWorker worker 2017-03-06 14:41:09 -05:00