Commit graph

15 commits

Author SHA1 Message Date
Jan Provaznik
41d52bbfe9 Add direct upload support for personal snippets 2019-08-23 11:19:14 +02:00
Heinrich Lee Yu
76b0518f33 Use #filename when generating upload URLs
We don't need to find the filename from the remote URL
2019-07-04 03:46:26 +08:00
Stan Hu
a1f44c1b49 Fix incorrect prefix used in new uploads for personal snippets
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24550 fixed the
case where the wrong path on disk was being searched, but it
inadvertently ommitted the `/uploads/-/system` prefix when rendering the
Markdown for personal snippet uploads when they were stored directly in
object storage.

A personal snippet path is stored using FileUploader#upload_path.
The format for the path:

Local storage: :random_hex/:filename.
Object storage: personal_snippet/:id/:random_hex/:filename.

upload_paths represent the possible paths for a given identifier,
which will vary depending on whether the file is stored in local or
object storage. upload_path should match an element in upload_paths.

base_dir represents the path seen by the user in Markdown, and it
should always be prefixed with uploads/-/system.

store_dirs represent the paths that are actually used on disk. For
object storage, this should omit the prefix /uploads/-/system.

For example, consider the requested path
/uploads/-/system/personal_snippet/172/ff4ad5c2/file.png.

For local storage:

base_dir: uploads/-/system/personal_snippet/172
upload_path: ff4ad5c2/file.png
upload_paths: ["ff4ad5c2/file.png", "personal_snippet/172/ff4ad5c2/file.png"].
store_dirs: {1=>"uploads/-/system/personal_snippet/172/ff4ad5c2",
             2=>"personal_snippet/172/ff4ad5c2"}

For object storage:

upload_path: personal_snippet/172/ff4ad5c2/file.png

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61671
2019-05-16 23:36:55 -07:00
Sean McGivern
79dd92c818 Optimise upload path calls
String#underscore isn't particularly slow, but it's possible for us to
call it many times in a users autocomplete request, with mostly-static
values ('User', 'Group', etc.). We can memoise this and save a
surprising amount of time (around 10% of the total request time in some
cases).
2019-05-14 11:01:48 -05:00
Stan Hu
940ad0c7a1 Fix 404s with snippet uploads in object storage
Previously, an HTTP request for
`/uploads/-/system/personal_snippet/:snippet_id/:hash/:filename` would
look for an uploader of `PersonalFileUploader` class and use
`PersonalFileUploader#upload_paths` to search the datbase for one of the
following paths:

1. `:hash/:filename`
2. `uploads/-/system/personal_snippet/:id/:hash/:filename`

However, when the upload were stored in object storage,
`PersonalFileUploader#store_dirs` stored the path as:

`personal_snippet/:snippet_id/:hash`

The extraneous `uploads/-/system` prefix prevented the path from being
matched, and uploads in object storage would return a 404 error. Uploads
in local storage would work fine.

To fix this, we set the `#base_dir` properly so that `#upload_paths`
generates the right value for object storage. Note that this also makes
`#store_dirs` do the right thing in `FileUploader`.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52595
2019-01-21 22:13:37 -08:00
gfyoung
d5ee813e3d Enable frozen string in apps/validators/*.rb
Partially addresses #47424.
2018-07-05 00:13:58 -07:00
Sean McGivern
2387ef2b4a Merge branch 'poc-upload-hashing-path' into 'master'
File uploads on objects storage should use hashed storage

Closes #4952

See merge request gitlab-org/gitlab-ee!4597
2018-03-08 09:25:56 -05:00
Kamil Trzciński
a2f375e8f7 Merge remote-tracking branch 'origin/master' into object-storage-ee-to-ce-backport 2018-02-28 21:28:43 +01:00
Sean McGivern
a7dae52e9d Merge branch '4163-move-uploads-to-object-storage' into 'master'
Move uploads to object storage

Closes #4163

See merge request gitlab-org/gitlab-ee!3867
2018-02-28 20:58:15 +01:00
Sean McGivern
028562a049 Fix 500 error when loading an invalid upload URL 2018-02-22 14:32:01 +00:00
Micaël Bergeron
2057a6acde port of 594e6a0a625^..f74c90f68c6 2018-02-01 12:14:46 -05:00
Bob Van Landuyt
180de2d201 Make sure uploads for personal snippets are correctly rendered 2017-08-11 18:15:02 +02:00
Sean McGivern
88df076fae Merge branch '33359-pers-snippet-files-location' into 'security-9-3'
Use uploads/system directory for personal snippets

See merge request !2123
2017-07-19 22:28:27 -05:00
Jarka Kadlecova
4464c22d6d Support descriptions for snippets 2017-05-31 07:17:03 +02:00
Jarka Kadlecova
43ff738641 Support uploaders for personal snippets comments 2017-05-02 15:22:24 +02:00