1
0
Fork 0
mirror of https://github.com/rails/rails.git synced 2022-11-09 12:12:34 -05:00
rails--rails/activestorage
David Heinemeier Hansson 69f976b859 Add credentials using a generic EncryptedConfiguration class (#30067)
* WIP: Add credentials using a generic EncryptedConfiguration class

This is sketch code so far.

* Flesh out EncryptedConfiguration and test it

* Better name

* Add command and generator for credentials

* Use the Pathnames

* Extract EncryptedFile from EncryptedConfiguration and add serializers

* Test EncryptedFile

* Extract serializer validation

* Stress the point about losing comments

* Allow encrypted configuration to be read without parsing for display

* Use credentials by default and base them on the master key

* Derive secret_key_base in test/dev, source it from credentials in other envs

And document the usage.

* Document the new credentials setup

* Stop generating the secrets.yml file now that we have credentials

* Document what we should have instead

Still need to make it happen, tho.

* [ci skip] Keep wording to `key base`; prefer defaults.

Usually we say we change defaults, not "spec" out a release.

Can't use backticks in our sdoc generated documentation either.

* Abstract away OpenSSL; prefer MessageEncryptor.

* Spare needless new when raising.

* Encrypted file test shouldn't depend on subclass.

* [ci skip] Some woordings.

* Ditch serializer future coding.

* I said flip it. Flip it good.

* [ci skip] Move require_master_key to the real production.rb.

* Add require_master_key to abort the boot process.

In case the master key is required in a certain environment
we should inspect that the key is there and abort if it isn't.

* Print missing key message and exit immediately.

Spares us a lengthy backtrace and prevents further execution.

I've verified the behavior in a test app, but couldn't figure the
test out as loading the app just exits immediately with:

```
/Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `load': marshal data too short (ArgumentError)
	from /Users/kasperhansen/Documents/code/rails/activesupport/lib/active_support/testing/isolation.rb:23:in `run'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest.rb:830:in `run_one_method'
	from /Users/kasperhansen/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/minitest-5.10.2/lib/minitest/parallel.rb:32:in `block (2 levels) in start'
```

It's likely we need to capture and prevent the exit somehow.
Kernel.stub(:exit) didn't work. Leaving it for tomorrow.

* Fix require_master_key config test.

Loading the app would trigger the `exit 1` per require_master_key's
semantics, which then aborted the test.

Fork and wait for the child process to finish, then inspect the
exit status.

Also check we aborted because of a missing master key, so something
else didn't just abort the boot.

Much <3 to @tenderlove for the tip.

* Support reading/writing configs via methods.

* Skip needless deep symbolizing.

* Remove save; test config reader elsewhere.

* Move secret_key_base check to when we're reading it.

Otherwise we'll abort too soon since we don't assign the secret_key_base
to secrets anymore.

* Add missing string literal comments; require unneeded yaml require.

* ya ya ya, rubocop.

* Add master_key/credentials after bundle.

Then we can reuse the existing message on `rails new bc4`.

It'll look like:

```
Using web-console 3.5.1 from https://github.com/rails/web-console.git (at master@ce985eb)
Using rails 5.2.0.alpha from source at `/Users/kasperhansen/Documents/code/rails`
Using sass-rails 5.0.6
Bundle complete! 16 Gemfile dependencies, 72 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Adding config/master.key to store the master encryption key: 97070158c44b4675b876373a6bc9d5a0

Save this in a password manager your team can access.

If you lose the key, no one, including you, can access anything encrypted with it.

      create  config/master.key
```

And that'll be executed even if `--skip-bundle` was passed.

* Ensure test app has secret_key_base.

* Assign secret_key_base to app or omit.

* Merge noise

* Split options for dynamic delegation into its own method and use deep symbols to make it work

* Update error to point to credentials instead

* Appease Rubocop

* Validate secret_key_base when reading it.

Instead of relying on the validation in key_generator move that into
secret_key_base itself.

* Fix generator and secrets test.

Manually add config.read_encrypted_secrets since it's not there by default
anymore.

Move mentions of config/secrets.yml to config/credentials.yml.enc.

* Remove files I have no idea how they got here.

* [ci skip] swap secrets for credentials.

* [ci skip] And now, changelogs are coming.
2017-09-11 20:21:20 +02:00
..
app Have attachments touch their records 2017-08-31 20:10:56 -04:00
bin Add executable file activestorage/bin/test 2017-08-20 21:28:56 +03:00
config Use frozen string literal in Active Storage 2017-08-12 21:43:42 +09:00
db/migrate Widen blob size column 2017-08-24 09:37:04 -04:00
lib Add credentials using a generic EncryptedConfiguration class (#30067) 2017-09-11 20:21:20 +02:00
test Fix replacing a singular attachment 2017-08-29 15:34:50 -04:00
.babelrc
.eslintrc
.gitignore Fix gitignore to be relative 2017-07-31 15:59:04 -05:00
activestorage.gemspec Make Rubocop happier about ActiveStorage 2017-08-03 11:43:08 -07:00
CHANGELOG.md Initialize changelog 2017-07-31 15:52:39 -05:00
MIT-LICENSE
package.json Active Storage to 5.2.0.alpha inside activestorage/package.json 2017-08-06 17:23:07 +00:00
Rakefile Use frozen string literal in Active Storage 2017-08-12 21:43:42 +09:00
README.md minor tweaks in Active Storage after a walkthrough 2017-08-15 18:50:46 +02:00
webpack.config.js
yarn.lock

Active Storage

Active Storage makes it simple to upload and reference files in cloud services like Amazon S3, Google Cloud Storage, or Microsoft Azure Storage, and attach those files to Active Records. Supports having one main service and mirrors in other services for redundancy. It also provides a disk service for testing or local deployments, but the focus is on cloud storage.

Files can be uploaded from the server to the cloud or directly from the client to the cloud.

Image files can furthermore be transformed using on-demand variants for quality, aspect ratio, size, or any other MiniMagick supported transformation.

Compared to other storage solutions

A key difference to how Active Storage works compared to other attachment solutions in Rails is through the use of built-in Blob and Attachment models (backed by Active Record). This means existing application models do not need to be modified with additional columns to associate with files. Active Storage uses polymorphic associations via the Attachment join model, which then connects to the actual Blob.

Blob models store attachment metadata (filename, content-type, etc.), and their identifier key in the storage service. Blob models do not store the actual binary data. They are intended to be immutable in spirit. One file, one blob. You can associate the same blob with multiple application models as well. And if you want to do transformations of a given Blob, the idea is that you'll simply create a new one, rather than attempt to mutate the existing one (though of course you can delete the previous version later if you don't need it).

Examples

One attachment:

class User < ApplicationRecord
  # Associates an attachment and a blob. When the user is destroyed they are
  # purged by default (models destroyed, and resource files deleted).
  has_one_attached :avatar
end

# Attach an avatar to the user.
user.avatar.attach(io: File.open("~/face.jpg"), filename: "avatar.jpg", content_type: "image/jpg")

# Does the user have an avatar?
user.avatar.attached? # => true

# Synchronously destroy the avatar and actual resource files.
user.avatar.purge

# Destroy the associated models and actual resource files async, via Active Job.
user.avatar.purge_later

# Does the user have an avatar?
user.avatar.attached? # => false

# Generate a permanent URL for the blob that points to the application.
# Upon access, a redirect to the actual service endpoint is returned.
# This indirection decouples the public URL from the actual one, and
# allows for example mirroring attachments in different services for
# high-availability. The redirection has an HTTP expiration of 5 min.
url_for(user.avatar)

class AvatarsController < ApplicationController
  def update
    # params[:avatar] contains a ActionDispatch::Http::UploadedFile object
    Current.user.avatar.attach(params.require(:avatar))
    redirect_to Current.user
  end
end

Many attachments:

class Message < ApplicationRecord
  has_many_attached :images
end
<%= form_with model: @message do |form| %>
  <%= form.text_field :title, placeholder: "Title" %><br>
  <%= form.text_area :content %><br><br>

  <%= form.file_field :images, multiple: true %><br>
  <%= form.submit %>
<% end %>
class MessagesController < ApplicationController
  def index
    # Use the built-in with_attached_images scope to avoid N+1
    @messages = Message.all.with_attached_images
  end

  def create
    message = Message.create! params.require(:message).permit(:title, :content)
    message.images.attach(params[:message][:images])
    redirect_to message
  end

  def show
    @message = Message.find(params[:id])
  end
end

Variation of image attachment:

<%# Hitting the variant URL will lazy transform the original blob and then redirect to its new service location %>
<%= image_tag user.avatar.variant(resize: "100x100") %>

Direct uploads

Active Storage, with its included JavaScript library, supports uploading directly from the client to the cloud.

Direct upload installation

  1. Include activestorage.js in your application's JavaScript bundle.

    Using the asset pipeline:

    //= require activestorage
    

    Using the npm package:

    import * as ActiveStorage from "activestorage"
    ActiveStorage.start()
    
  2. Annotate file inputs with the direct upload URL.

    <%= form.file_field :attachments, multiple: true, direct_upload: true %>
    
  3. That's it! Uploads begin upon form submission.

Direct upload JavaScript events

Event name Event target Event data (event.detail) Description
direct-uploads:start <form> None A form containing files for direct upload fields was submitted.
direct-upload:initialize <input> {id, file} Dispatched for every file after form submission.
direct-upload:start <input> {id, file} A direct upload is starting.
direct-upload:before-blob-request <input> {id, file, xhr} Before making a request to your application for direct upload metadata.
direct-upload:before-storage-request <input> {id, file, xhr} Before making a request to store a file.
direct-upload:progress <input> {id, file, progress} As requests to store files progress.
direct-upload:error <input> {id, file, error} An error occurred. An alert will display unless this event is canceled.
direct-upload:end <input> {id, file} A direct upload has ended.
direct-uploads:end <form> None All direct uploads have ended.

License

Active Storage is released under the MIT License.