1
0
Fork 0
mirror of https://github.com/rails/rails.git synced 2022-11-09 12:12:34 -05:00
Commit graph

442 commits

Author SHA1 Message Date
Yuji Yaginuma
222757618e
Merge pull request from marceloperini/marceloperini/33795
add require 'database/setup' in activestorage/test/service/s3_service_test.rb
2018-12-03 08:29:57 +09:00
Richard Macklin
fb0c81d349 Remove obsolete yarn.lock files and check in root yarn.lock file 2018-12-02 10:02:14 -08:00
Marcelo Perini Veloso
ce40e85194
add require 'database/setup' in activestorage/test/service/s3_service_test.rb 2018-12-01 15:47:54 -02:00
yuuji.yaginuma
6ca2f2acb2 metadata is not passed to service
Ref: 604fac6d71/activestorage/app/models/active_storage/blob.rb (L256-L264)

This fixes broken `GCSServiceTest`.
https://travis-ci.org/rails/rails/jobs/461868394#L6624-L6626

Follow up to  .
2018-12-01 09:12:55 +09:00
Yannick Schutz
7bd21e8575
Add a test with extra keys to active_storage Service#upload 2018-11-30 15:54:38 +01:00
Hiroki Sanpei
4deb88ca26 Fix ArgumentError when uploading to amazon s3 2018-11-28 17:06:29 +09:00
yuuji.yaginuma
ac721c8552 Fix "warning: ambiguous first argument; put parentheses or a space even after `/' operator" 2018-11-28 11:29:03 +09:00
yuuji.yaginuma
b7ec129de5 text is treated as attachment 2018-11-28 10:35:57 +09:00
yuuji.yaginuma
c2ef8bbf52 Fix broken ActiveStorage::BlobTest
`ActiveStorage::Filename#parameters` was removed by .
2018-11-28 10:16:07 +09:00
yuuji.yaginuma
7e7a60bd3f Remove duplicated test
Since 06ab7b27ea,
`GCSServiceTest#test_signed_URL_response_headers` is broken.
https://travis-ci.org/rails/rails/jobs/460454477#L7084-L7087

This seems to be due to lack of `content_type` at upload.
This is solved by specifying `conten_type`.

However, since the same content is also tested with `test_upload_with_content_type`,
it will be duplicated content, so I think that can remove `test_signed_URL_response_headers`.
2018-11-28 10:01:03 +09:00
Adrian Hawrylak
2c279aa980 Fix minor Active Storage docs typo [ci skip] 2018-11-28 00:27:58 +01:00
Rosa Gutierrez
06ab7b27ea Prevent content type and disposition bypass in storage service URLs
* Force content-type to binary on service urls for relevant content types

We have a list of content types that must be forcibly served as binary,
but in practice this only means to serve them as attachment always. We
should also set the Content-Type to the configured binary type.

As a bonus: add text/cache-manifest to the list of content types to be
served as binary by default.

* Store content-disposition and content-type in GCS

Forcing these in the service_url when serving the file works fine for S3
and Azure, since these services include params in the signature.
However, GCS specifically excludes response-content-disposition and
response-content-type from the signature, which means an attacker can
modify these and have files that should be served as text/plain attachments
served as inline HTML for example. This makes our attempt to force
specific files to be served as binary and as attachment can be easily
bypassed.

The only way this can be forced in GCS is by storing
content-disposition and content-type in the object metadata.

* Update GCS object metadata after identifying blob

In some cases we create the blob and upload the data before identifying
the content-type, which means we can't store that in GCS right when
uploading. In these, after creating the attachment, we enqueue a job to
identify the blob, and set the content-type.

In other cases, files are uploaded to the storage service via direct
upload link. We create the blob before the direct upload, which happens
independently from the blob creation itself. We then mark the blob as
identified, but we have already the content-type we need without having
put it in the service.

In these two cases, then, we need to update the metadata in the GCS
service.

* Include content-type and disposition in the verified key for disk service

This prevents an attacker from modifying these params in the service
signed URL, which is particularly important when we want to force them
to have specific values for security reasons.

* Allow only a list of specific content types to be served inline

This is different from the content types that must be served as binary
in the sense that any content type not in this list will be always
served as attachment but with its original content type. Only types in
this list are allowed to be served either inline or as attachment.

Apart from forcing this in the service URL, for GCS we need to store the
disposition in the metadata.

Fix CVE-2018-16477.
2018-11-27 15:36:27 -05:00
Tasos Maschalidis
d6fcc419fe ASt direct uploads: account for <button type="submit"> elements 2018-11-24 14:54:05 -05:00
Maurice Kühlborn
455a122ef0 Add progressive JPG to default variable content types 2018-11-21 23:21:01 +01:00
Gannon McGibbon
e74fdbe00c Amend CVE note and security guide section wordings
Reword first sentence of dep management and CVE section of
security guide. Also, reword and move gemspec notes above deps.

[ci skip]
2018-11-06 18:06:57 -05:00
Gannon McGibbon
1c11688b56 Add CVE note to security guide and gemspecs
[ci skip]
2018-11-06 14:25:36 -05:00
Kyle Keesling
79e086efc4
update activestorage attachment model documentation reflect recent behavior changes 2018-10-24 09:08:45 -04:00
Donnie Propst
b4e827d80a Point to requiring the ASt engine in the installation instructions [ci skip] 2018-10-08 11:26:51 -04:00
Graham Conzett
bba5ecc923 Use content_mime_type 2018-10-08 09:50:51 -04:00
Graham Conzett
1986048d27 Fix issue ActiveStorage direct upload disk
Fix an issue in ActiveStorage where a direct upload to disk storage
would fail due to a content type mismatch if the file was uploaded using
a mime-type synonym.
2018-10-07 16:07:09 -04:00
David Heinemeier Hansson
4838c1716a
Make Webpacker the default JavaScript compiler for Rails 6 ()
* Use Webpacker by default on new apps

* Stop including coffee-rails by default

* Drop using a js_compressor by default

* Drop extra test for coffeescript inclusion by default

* Stick with skip_javascript to signify skipping webpack

* Don't install a JS runtime by default any more

* app/javascript will be the new default directory for JS

* Make it clear that this is just for configuring the default Webpack framework setup now

* Start using the Webpack tag in the default layout

* Irrelevant test

* jQuery is long gone

* Stop having asset pipeline compile default application.js

* Add rails-ujs by default to the Webpack setup

* Add Active Storage JavaScript to application.js pack by default

* Consistent quoting

* Add Turbolinks to default pack

* Add Action Cable to default pack

Need some work on how to set the global consumer that channels will
work with. @javan?

* Require all channels by default and use a separate consumer stub

* Channel generator now targets Webpack style

* Update task docs to match new generator style

* Use uniform import style

* Drop the JS assets generator

It was barely helpful as it was. It’s no longer helpful in a Webpacked
world. Sayonara!

* Add app/javascript to the stats directories

* Simpler import style

Which match the other imports.

* Address test failures from dropping JS compilation (and compression)

* webpacker-default: Modify `AssetsGeneratorTest`

Before:

```
$ bin/test test/generators/assets_generator_test.rb
Run options: --seed 46201

F

Failure:
AssetsGeneratorTest#test_assets [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/assets_generator_test.rb:12]:
Expected file "app/assets/javascripts/posts.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/assets_generator_test.rb:10

.

Finished in 0.031343s, 63.8101 runs/s, 95.7152 assertions/s.
2 runs, 3 assertions, 1 failures, 0 errors, 0 skips
```

After:

```
$ bin/test test/generators/assets_generator_test.rb
Run options: --seed 43571

..

Finished in 0.030370s, 65.8545 runs/s, 65.8545 assertions/s.
2 runs, 2 assertions, 0 failures, 0 errors, 0 skips
```

* webpacker-default: Modify `ChannelGeneratorTest`

Before:

```
$ bin/test test/generators/channel_generator_test.rb
Run options: --seed 8986

.F

Failure:
ChannelGeneratorTest#test_channel_with_multiple_actions_is_created [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:43]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:34

.F

Failure:
ChannelGeneratorTest#test_channel_is_created [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:29]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:22

E

Error:
ChannelGeneratorTest#test_cable_js_is_created_if_not_present_already:
Errno::ENOENT: No such file or directory @ apply2files - /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/fixtures/tmp/app/assets/javascripts/cable.js

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:60

F

Failure:
ChannelGeneratorTest#test_channel_suffix_is_not_duplicated [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:87]:
Expected file "app/assets/javascripts/channels/chat.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:80

F

Failure:
ChannelGeneratorTest#test_channel_on_revoke [/Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:77]:
Expected file "app/assets/javascripts/cable.js" to exist, but does not

bin/test /Users/ttanimichi/ghq/github.com/ttanimichi/rails/railties/test/generators/channel_generator_test.rb:68

Finished in 0.064384s, 108.7227 runs/s, 481.4861 assertions/s.
7 runs, 31 assertions, 4 failures, 1 errors, 0 skips
```

After:

```
$ bin/test test/generators/channel_generator_test.rb
Run options: --seed 44857

.......

Finished in 0.060243s, 116.1961 runs/s, 697.1764 assertions/s.
7 runs, 42 assertions, 0 failures, 0 errors, 0 skips
```

* Fix shared generator tests.

* webpacker-default: Modify `ControllerGeneratorTest`

The JS assets generator was dropped. ref. 46215b1794

* Revert "Simpler import style". It's currently failing with an error of "TypeError: undefined is not an object (evaluating '__WEBPACK_IMPORTED_MODULE_2_activestorage___default.a.start')". Waiting for @javan to have a look.

This reverts commit 5d3ebb7105.

* require webpacker in test app

* Add webpacker without making the build hang/timeout. ()

* use yarn workspaces to allow for installing unreleased packages and only generate js/bootsnap when required

* no longer need to have webpacker in env templates as webpacker moved this config to yml file

* Fix rubocop violation

* Got the test passing for the running scaffold

* update expected lines of code

* update middleware tests to account for webpacker

* disable js in plugins be default to get the tests passing ()

* clear codeclimate report issues

* Anything newer than currently released is good

* Use Webpacker development version during development of Rails

* Edge should get development webpacker as well

* Add changelog entry for Webpacker change
2018-09-30 22:31:21 -07:00
Rafael Mendonça França
f679933daa
Change the empty block style to have space inside of the block 2018-09-25 13:19:35 -04:00
Kasper Timm Hansen
ed56a03104
Merge pull request from mtsmfm/encode-filename
Encode Content-Disposition filenames on send_data and send_file
2018-09-23 19:43:06 +02:00
yuuji.yaginuma
1b86d90136 Enable Performance/UnfreezeString cop
In Ruby 2.3 or later, `String#+@` is available and `+@` is faster than `dup`.

```ruby
# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "benchmark-ips"
end

Benchmark.ips do |x|
  x.report('+@') { +"" }
  x.report('dup') { "".dup }
  x.compare!
end
```

```
$ ruby -v benchmark.rb
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
Warming up --------------------------------------
                  +@   282.289k i/100ms
                 dup   187.638k i/100ms
Calculating -------------------------------------
                  +@      6.775M (± 3.6%) i/s -     33.875M in   5.006253s
                 dup      3.320M (± 2.2%) i/s -     16.700M in   5.032125s

Comparison:
                  +@:  6775299.3 i/s
                 dup:  3320400.7 i/s - 2.04x  slower

```
2018-09-23 08:56:55 +09:00
bogdanvlviv
be0d22a8e1
Raise ActiveRecord::InvalidForeignKey in before_destroy for a blob if attachments exist
The issue  was fixed in  by adding foreign key constraint
to the `active_storage_attachments` table for blobs.
This commit implements fix on app-level in order to ensure that users
can't delete a blob with attachments even if they don't have the foreign key constraint.
See a related discussion in the Campfire:
https://3.basecamp.com/3076981/buckets/24956/chats/12416418@1236718899

Note that, we should backport it to `5-2-stable` too.

Related to 
2018-09-16 22:44:30 +03:00
Chris Bisnett
7dd9916c0d Configure Active Storage route prefix
Applications can configure the route prefix prepended to the Active
Storage routes. By default this maintains the previous prefix
`/rails/active_storage` but supports custom prefixes.

Before this change the route for serving blobs is fixed to
`/rails/active_storage/blobs/:signed_id/*filename`. After this change
it's possible to configure the route to something like
`/files/blobs/:signed_id/*filename`.
2018-09-14 10:40:18 -04:00
Fumiaki MATSUSHIMA
890485cfce Encode Content-Disposition filenames on send_data and send_file 2018-09-13 21:38:46 +09:00
Xavier Noria
f589e20b0a use "minitest" consistently
The name of the minitest library is spelled that way: regular font, and
lowercase. Lowercase is used even at the beginning of sentences, see

    http://docs.seattlerb.org/minitest/

I double-checked this with @zenspider too (thanks!).
2018-09-11 23:57:25 +02:00
Marcelo Perini Veloso
0d4ba40688 Fix zero-byte files upload 2018-09-06 20:04:21 -03:00
Cameron Bothner
aae56c3529 Handle only specifically relevant Azure HTTPErrors
The Azure gem uses `Azure::Core::Http::HTTPError` for everything:
checksum mismatch, missing object, network unavailable, and many more.
(https://www.rubydoc.info/github/yaxia/azure-storage-ruby/Azure/Core/Http/HTTPError).
Rescuing that class obscures all sorts of  configuration errors. We
should check the type of error in those rescue  blocks, and reraise when
needed.
2018-08-23 23:36:43 -04:00
George Claghorn
3868648cae Revert "Merge pull request from cbothner/azure-service-swallowing-all-errors"
This reverts commit b204d167c5, reversing
changes made to de6a200f82.
2018-08-23 19:07:29 -04:00
Cameron Bothner
6acf2fa363 Handle only specifically relevant Azure HTTPErrors
The Azure gem uses `Azure::Core::Http::HTTPError` for everything:
checksum mismatch, missing object, network unavailable, and many more.
(https://www.rubydoc.info/github/yaxia/azure-storage-ruby/Azure/Core/Http/HTTPError).
Rescuing that class obscures all sorts of  configuration errors. We
should check the type of error in those rescue  blocks, and reraise when
needed.
2018-08-23 17:54:35 -04:00
George Claghorn
dc001dbd58
Merge pull request from cbothner/fail-gracefully-from-activestorage-file-not-found
Fail more gracefully from ActiveStorage missing file exceptions
2018-08-22 22:56:10 -04:00
Cameron Bothner
22efb2ec49 Respond with 404 in ActiveStorage::DiskController#show when file missing
`ActiveStorage::DiskController#show` generates a 404 Not Found response when
the requested file is missing from the disk service. It previously raised
`Errno::ENOENT`.
2018-08-21 15:35:58 -04:00
Cameron Bothner
5cd2d07bdc Translate service-specific missing object exceptions into a generic one
`ActiveStorage::Blob#download` and `ActiveStorage::Blob#open` raise
`ActiveStorage::FileNotFoundError` when the corresponding file is missing
from the storage service. Services translate service-specific missing
object exceptions (e.g. `Google::Cloud::NotFoundError` for the GCS service
and `Errno::ENOENT` for the disk service) into
`ActiveStorage::FileNotFoundError`.
2018-08-21 15:31:14 -04:00
bogdanvlviv
0e77706dc5
activestorage: yarn build
Context https://github.com/rails/rails/pull/33413#issuecomment-414137587
2018-08-19 19:33:16 +03:00
George Claghorn
093d8d3953 Retry ActiveStorage::AnalyzeJobs on download integrity check failures 2018-08-19 01:19:51 -04:00
George Claghorn
116fae6ef9 Remove superfluous test
This test no longer covers the behavior of ActiveStorage::PurgeJob. Attached blobs are ignored by ActiveStorage::Blob#purge as of 934fccd, which includes an equivalent model test.
2018-08-19 01:10:50 -04:00
Chris Seelus
880f977925 Enable multiple submit buttons for Active Storage forms 2018-08-18 19:31:39 +02:00
George Claghorn
e33c3cd8cc Extract ActiveStorage::SetCurrent
Provide a handy concern for custom Active Storage controllers that can't inherit from ActiveStorage::BaseController.
2018-08-16 01:41:15 -04:00
George Claghorn
2d3a28b949 Document all Active Storage error classes [ci skip] 2018-08-10 18:38:28 -04:00
George Claghorn
18425b8371 Add a generic base class for Active Storage exceptions
Closes .

[Andrei Makarov & George Claghorn]
2018-08-10 18:35:49 -04:00
George Claghorn
697f4a93ad Extract transformers 2018-08-10 12:42:14 -04:00
George Claghorn
10129fbf57 DRY up web image checks in ActiveStorage::Variant 2018-08-09 19:53:12 -04:00
George Claghorn
79573b3aff Camelize instead of classifying
Avoid mangling service names that end in S:

    "GCS".classify # => "GC"
    "GCS".camelize # => "GCS"
2018-08-06 22:23:47 -04:00
Joel Taylor
3082786be6 Improve ActiveStorage service adapter error handling 2018-08-06 18:17:49 -07:00
George Claghorn
bd680dd59a Fix uploading Tempfiles to Azure Storage
Closes .
2018-08-03 22:41:19 -04:00
George Claghorn
bba4d2dd8c Remove another unused require 2018-08-02 14:38:33 -04:00
George Claghorn
bd01f9831c Remove unused require 2018-08-01 09:59:04 -04:00
George Claghorn
fee7fba893 Retry ActiveStorage::PurgeJobs on DB deadlock 2018-07-31 19:29:23 -04:00