mirror of
https://github.com/rails/rails.git
synced 2022-11-09 12:12:34 -05:00
Basic API for connection switching
This PR adds the ability to 1) connect to multiple databases in a model,
and 2) switch between those connections using a block.
To connect a model to a set of databases for writing and reading use
the following API. This API supercedes `establish_connection`. The
`writing` and `reading` keys represent handler / role names and
`animals` and `animals_replica` represents the database key to look up
the configuration hash from.
```
class AnimalsBase < ApplicationRecord
connects_to database: { writing: :animals, reading: :animals_replica }
end
```
Inside the application - outside the model declaration - we can switch
connections with a block call to `connected_to`.
If we want to connect to a db that isn't default (ie readonly_slow) we
can connect like this:
Outside the model we may want to connect to a new database (one that is
not in the default writing/reading set) - for example a slow replica for
making slow queries. To do this we have the `connected_to` method that
takes a `database` hash that matches the signature of `connects_to`. The
`connected_to` method also takes a block.
```
AcitveRecord::Base.connected_to(database: { slow_readonly: :primary_replica_slow }) do
ModelInPrimary.do_something_thats_slow
end
```
For models that are already loaded and connections that are already
connected, `connected_to` doesn't need to pass in a `database` because
you may want to run queries against multiple databases using a specific
role/handler.
In this case `connected_to` can take a `role` and use that to swap on
the connection passed. This simplies queries - and matches how we do it
in GitHub. Once you're connected to the database you don't need to
re-connect, we assume the connection is in the pool and simply pass the
handler we'd like to swap on.
```
ActiveRecord::Base.connected_to(role: :reading) do
Dog.read_something_from_dog
ModelInPrimary.do_something_from_model_in_primary
end
```
This commit is contained in:
Eileen Uchitelle
parent
58999af99f
commit
31021a8c85
4 changed files with 328 additions and 1 deletions
|
|
@ -1,3 +1,39 @@
|
|||
* Add basic API for connection switching to support multiple databases.
|
||||
|
||||
1) Adds a `connects_to` method for models to connect to multiple databases. Example:
|
||||
|
||||
```
|
||||
class AnimalsModel < ApplicationRecord
|
||||
self.abstract_class = true
|
||||
|
||||
connects_to database: { writing: :animals_primary, reading: :animals_replica }
|
||||
end
|
||||
|
||||
class Dog < AnimalsModel
|
||||
# connected to both the animals_primary db for writing and the animals_replica for reading
|
||||
end
|
||||
```
|
||||
|
||||
2) Adds a `connected_to` block method for switching connection roles or connecting to
|
||||
a database that the model didn't connect to. Connecting to the database in this block is
|
||||
useful when you have another defined connection, for example `slow_replica` that you don't
|
||||
want to connect to by default but need in the console, or a specific code block.
|
||||
|
||||
```
|
||||
ActiveRecord::Base.connected_to(role: :reading) do
|
||||
Dog.first # finds dog from replica connected to AnimalsBase
|
||||
Book.first # doesn't have a reading connection, will raise an error
|
||||
end
|
||||
```
|
||||
|
||||
```
|
||||
ActiveRecord::Base.connected_to(database: :slow_replica) do
|
||||
SlowReplicaModel.first # if the db config has a slow_replica configuration this will be used to do the lookup, otherwise this will throw an exception
|
||||
end
|
||||
```
|
||||
|
||||
*Eileen M. Uchitelle*
|
||||
|
||||
* Enum raises on invalid definition values
|
||||
|
||||
When defining a Hash enum it can be easy to use [] instead of {}. This
|
||||
|
|
|
|||
|
|
@ -47,6 +47,92 @@ module ActiveRecord
|
|||
# The exceptions AdapterNotSpecified, AdapterNotFound and +ArgumentError+
|
||||
# may be returned on an error.
|
||||
def establish_connection(config_or_env = nil)
|
||||
config_hash = resolve_config_for_connection(config_or_env)
|
||||
connection_handler.establish_connection(config_hash)
|
||||
end
|
||||
|
||||
# Connects a model to the databases specified. The +database+ keyword
|
||||
# takes a hash consisting of a +role+ and a +database_key+.
|
||||
#
|
||||
# This will create a connection handler for switching between connections,
|
||||
# look up the config hash using the +database_key+ and finally
|
||||
# establishes a connection to that config.
|
||||
#
|
||||
# class AnimalsModel < ApplicationRecord
|
||||
# self.abstract_class = true
|
||||
#
|
||||
# connects_to database: { writing: :primary, reading: :primary_replica }
|
||||
# end
|
||||
#
|
||||
# Returns an array of established connections.
|
||||
def connects_to(database: {})
|
||||
connections = []
|
||||
|
||||
database.each do |role, database_key|
|
||||
config_hash = resolve_config_for_connection(database_key)
|
||||
handler = lookup_connection_handler(role.to_sym)
|
||||
|
||||
connections << handler.establish_connection(config_hash)
|
||||
end
|
||||
|
||||
connections
|
||||
end
|
||||
|
||||
# Connects to a database or role (ex writing, reading, or another
|
||||
# custom role) for the duration of the block.
|
||||
#
|
||||
# If a role is passed, Active Record will look up the connection
|
||||
# based on the requested role:
|
||||
#
|
||||
# ActiveRecord::Base.connected_to(role: :writing) do
|
||||
# Dog.create! # creates dog using dog connection
|
||||
# end
|
||||
#
|
||||
# ActiveRecord::Base.connected_to(role: :reading) do
|
||||
# Dog.create! # throws exception because we're on a replica
|
||||
# end
|
||||
#
|
||||
# ActiveRecord::Base.connected_to(role: :unknown_ode) do
|
||||
# # raises exception due to non-existent role
|
||||
# end
|
||||
#
|
||||
# For cases where you may want to connect to a database outside of the model,
|
||||
# you can use +connected_to+ with a +database+ argument. The +database+ argument
|
||||
# expects a symbol that corresponds to the database key in your config.
|
||||
#
|
||||
# This will connect to a new database for the queries inside the block.
|
||||
#
|
||||
# ActiveRecord::Base.connected_to(database: :animals_slow_replica) do
|
||||
# Dog.run_a_long_query # runs a long query while connected to the +animals_slow_replica+
|
||||
# end
|
||||
def connected_to(database: nil, role: nil, &blk)
|
||||
if database && role
|
||||
raise ArgumentError, "connected_to can only accept a database or role argument, but not both arguments."
|
||||
elsif database
|
||||
config_hash = resolve_config_for_connection(database)
|
||||
handler = lookup_connection_handler(database.to_sym)
|
||||
|
||||
with_handler(database.to_sym) do
|
||||
handler.establish_connection(config_hash)
|
||||
return yield
|
||||
end
|
||||
elsif role
|
||||
with_handler(role.to_sym, &blk)
|
||||
else
|
||||
raise ArgumentError, "must provide a `database` or a `role`."
|
||||
end
|
||||
end
|
||||
|
||||
def lookup_connection_handler(handler_key) # :nodoc:
|
||||
connection_handlers[handler_key] ||= ActiveRecord::ConnectionAdapters::ConnectionHandler.new
|
||||
end
|
||||
|
||||
def with_handler(handler_key, &blk) # :nodoc:
|
||||
handler = lookup_connection_handler(handler_key)
|
||||
swap_connection_handler(handler, &blk)
|
||||
end
|
||||
|
||||
def resolve_config_for_connection(config_or_env) # :nodoc:
|
||||
raise "Anonymous class is not allowed." unless name
|
||||
|
||||
config_or_env ||= DEFAULT_ENV.call.to_sym
|
||||
|
|
@ -57,7 +143,7 @@ module ActiveRecord
|
|||
config_hash = resolver.resolve(config_or_env, pool_name).symbolize_keys
|
||||
config_hash[:name] = pool_name
|
||||
|
||||
connection_handler.establish_connection(config_hash)
|
||||
config_hash
|
||||
end
|
||||
|
||||
# Returns the connection currently associated with the class. This can
|
||||
|
|
@ -118,5 +204,14 @@ module ActiveRecord
|
|||
|
||||
delegate :clear_active_connections!, :clear_reloadable_connections!,
|
||||
:clear_all_connections!, :flush_idle_connections!, to: :connection_handler
|
||||
|
||||
private
|
||||
|
||||
def swap_connection_handler(handler, &blk) # :nodoc:
|
||||
old_handler, ActiveRecord::Base.connection_handler = ActiveRecord::Base.connection_handler, handler
|
||||
yield
|
||||
ensure
|
||||
ActiveRecord::Base.connection_handler = old_handler
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
|
|||
|
|
@ -124,6 +124,8 @@ module ActiveRecord
|
|||
|
||||
mattr_accessor :belongs_to_required_by_default, instance_accessor: false
|
||||
|
||||
mattr_accessor :connection_handlers, instance_accessor: false, default: {}
|
||||
|
||||
class_attribute :default_connection_handler, instance_writer: false
|
||||
|
||||
self.filter_attributes = []
|
||||
|
|
@ -137,6 +139,7 @@ module ActiveRecord
|
|||
end
|
||||
|
||||
self.default_connection_handler = ConnectionAdapters::ConnectionHandler.new
|
||||
self.connection_handlers = { writing: ActiveRecord::Base.default_connection_handler }
|
||||
end
|
||||
|
||||
module ClassMethods
|
||||
|
|
|
|||
|
|
@ -0,0 +1,193 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
require "cases/helper"
|
||||
require "models/person"
|
||||
|
||||
module ActiveRecord
|
||||
module ConnectionAdapters
|
||||
class ConnectionHandlersMultiDbTest < ActiveRecord::TestCase
|
||||
self.use_transactional_tests = false
|
||||
|
||||
fixtures :people
|
||||
|
||||
def setup
|
||||
@handlers = { writing: ConnectionHandler.new, reading: ConnectionHandler.new }
|
||||
@rw_handler = @handlers[:writing]
|
||||
@ro_handler = @handlers[:reading]
|
||||
@spec_name = "primary"
|
||||
@rw_pool = @handlers[:writing].establish_connection(ActiveRecord::Base.configurations["arunit"])
|
||||
@ro_pool = @handlers[:reading].establish_connection(ActiveRecord::Base.configurations["arunit"])
|
||||
end
|
||||
|
||||
def teardown
|
||||
ActiveRecord::Base.connection_handlers = { writing: ActiveRecord::Base.default_connection_handler }
|
||||
end
|
||||
|
||||
class MultiConnectionTestModel < ActiveRecord::Base
|
||||
end
|
||||
|
||||
def test_multiple_connection_handlers_works_in_a_threaded_environment
|
||||
tf_writing = Tempfile.open "test_writing"
|
||||
tf_reading = Tempfile.open "test_reading"
|
||||
|
||||
MultiConnectionTestModel.connects_to database: { writing: { database: tf_writing.path, adapter: "sqlite3" }, reading: { database: tf_reading.path, adapter: "sqlite3" } }
|
||||
|
||||
MultiConnectionTestModel.connection.execute("CREATE TABLE `test_1` (connection_role VARCHAR (255))")
|
||||
MultiConnectionTestModel.connection.execute("INSERT INTO test_1 VALUES ('writing')")
|
||||
|
||||
ActiveRecord::Base.connected_to(role: :reading) do
|
||||
MultiConnectionTestModel.connection.execute("CREATE TABLE `test_1` (connection_role VARCHAR (255))")
|
||||
MultiConnectionTestModel.connection.execute("INSERT INTO test_1 VALUES ('reading')")
|
||||
end
|
||||
|
||||
read_latch = Concurrent::CountDownLatch.new
|
||||
write_latch = Concurrent::CountDownLatch.new
|
||||
|
||||
MultiConnectionTestModel.connection
|
||||
|
||||
thread = Thread.new do
|
||||
MultiConnectionTestModel.connection
|
||||
|
||||
write_latch.wait
|
||||
assert_equal "writing", MultiConnectionTestModel.connection.select_value("SELECT connection_role from test_1")
|
||||
read_latch.count_down
|
||||
end
|
||||
|
||||
ActiveRecord::Base.connected_to(role: :reading) do
|
||||
write_latch.count_down
|
||||
assert_equal "reading", MultiConnectionTestModel.connection.select_value("SELECT connection_role from test_1")
|
||||
read_latch.wait
|
||||
end
|
||||
|
||||
thread.join
|
||||
ensure
|
||||
tf_reading.close
|
||||
tf_reading.unlink
|
||||
tf_writing.close
|
||||
tf_writing.unlink
|
||||
end
|
||||
|
||||
unless in_memory_db?
|
||||
def test_establish_connection_using_3_levels_config
|
||||
previous_env, ENV["RAILS_ENV"] = ENV["RAILS_ENV"], "default_env"
|
||||
|
||||
config = {
|
||||
"default_env" => {
|
||||
"readonly" => { "adapter" => "sqlite3", "database" => "db/readonly.sqlite3" },
|
||||
"primary" => { "adapter" => "sqlite3", "database" => "db/primary.sqlite3" }
|
||||
}
|
||||
}
|
||||
@prev_configs, ActiveRecord::Base.configurations = ActiveRecord::Base.configurations, config
|
||||
|
||||
ActiveRecord::Base.connects_to(database: { writing: :primary, reading: :readonly })
|
||||
|
||||
assert_not_nil pool = ActiveRecord::Base.connection_handlers[:writing].retrieve_connection_pool("primary")
|
||||
assert_equal "db/primary.sqlite3", pool.spec.config[:database]
|
||||
|
||||
assert_not_nil pool = ActiveRecord::Base.connection_handlers[:reading].retrieve_connection_pool("primary")
|
||||
assert_equal "db/readonly.sqlite3", pool.spec.config[:database]
|
||||
ensure
|
||||
ActiveRecord::Base.configurations = @prev_configs
|
||||
ActiveRecord::Base.establish_connection(:arunit)
|
||||
ENV["RAILS_ENV"] = previous_env
|
||||
end
|
||||
|
||||
def test_switching_connections_via_handler
|
||||
previous_env, ENV["RAILS_ENV"] = ENV["RAILS_ENV"], "default_env"
|
||||
|
||||
config = {
|
||||
"default_env" => {
|
||||
"readonly" => { "adapter" => "sqlite3", "database" => "db/readonly.sqlite3" },
|
||||
"primary" => { "adapter" => "sqlite3", "database" => "db/primary.sqlite3" }
|
||||
}
|
||||
}
|
||||
@prev_configs, ActiveRecord::Base.configurations = ActiveRecord::Base.configurations, config
|
||||
|
||||
ActiveRecord::Base.connects_to(database: { writing: :primary, reading: :readonly })
|
||||
|
||||
ActiveRecord::Base.connected_to(role: :reading) do
|
||||
@ro_handler = ActiveRecord::Base.connection_handler
|
||||
assert_equal ActiveRecord::Base.connection_handler, ActiveRecord::Base.connection_handlers[:reading]
|
||||
end
|
||||
|
||||
ActiveRecord::Base.connected_to(role: :writing) do
|
||||
assert_equal ActiveRecord::Base.connection_handler, ActiveRecord::Base.connection_handlers[:writing]
|
||||
assert_not_equal @ro_handler, ActiveRecord::Base.connection_handler
|
||||
end
|
||||
ensure
|
||||
ActiveRecord::Base.configurations = @prev_configs
|
||||
ActiveRecord::Base.establish_connection(:arunit)
|
||||
ENV["RAILS_ENV"] = previous_env
|
||||
end
|
||||
|
||||
def test_connects_to_with_single_configuration
|
||||
config = {
|
||||
"development" => { "adapter" => "sqlite3", "database" => "db/primary.sqlite3" },
|
||||
}
|
||||
@prev_configs, ActiveRecord::Base.configurations = ActiveRecord::Base.configurations, config
|
||||
|
||||
ActiveRecord::Base.connects_to database: { writing: :development }
|
||||
|
||||
assert_equal 1, ActiveRecord::Base.connection_handlers.size
|
||||
assert_equal ActiveRecord::Base.connection_handler, ActiveRecord::Base.connection_handlers[:writing]
|
||||
ensure
|
||||
ActiveRecord::Base.configurations = @prev_configs
|
||||
ActiveRecord::Base.establish_connection(:arunit)
|
||||
end
|
||||
|
||||
def test_connects_to_using_top_level_key_in_two_level_config
|
||||
config = {
|
||||
"development" => { "adapter" => "sqlite3", "database" => "db/primary.sqlite3" },
|
||||
"development_readonly" => { "adapter" => "sqlite3", "database" => "db/readonly.sqlite3" }
|
||||
}
|
||||
@prev_configs, ActiveRecord::Base.configurations = ActiveRecord::Base.configurations, config
|
||||
|
||||
ActiveRecord::Base.connects_to database: { writing: :development, reading: :development_readonly }
|
||||
|
||||
assert_not_nil pool = ActiveRecord::Base.connection_handlers[:reading].retrieve_connection_pool("primary")
|
||||
assert_equal "db/readonly.sqlite3", pool.spec.config[:database]
|
||||
ensure
|
||||
ActiveRecord::Base.configurations = @prev_configs
|
||||
ActiveRecord::Base.establish_connection(:arunit)
|
||||
end
|
||||
end
|
||||
|
||||
def test_connection_pools
|
||||
assert_equal([@rw_pool], @handlers[:writing].connection_pools)
|
||||
assert_equal([@ro_pool], @handlers[:reading].connection_pools)
|
||||
end
|
||||
|
||||
def test_retrieve_connection
|
||||
assert @rw_handler.retrieve_connection(@spec_name)
|
||||
assert @ro_handler.retrieve_connection(@spec_name)
|
||||
end
|
||||
|
||||
def test_active_connections?
|
||||
assert_not_predicate @rw_handler, :active_connections?
|
||||
assert_not_predicate @ro_handler, :active_connections?
|
||||
|
||||
assert @rw_handler.retrieve_connection(@spec_name)
|
||||
assert @ro_handler.retrieve_connection(@spec_name)
|
||||
|
||||
assert_predicate @rw_handler, :active_connections?
|
||||
assert_predicate @ro_handler, :active_connections?
|
||||
|
||||
@rw_handler.clear_active_connections!
|
||||
assert_not_predicate @rw_handler, :active_connections?
|
||||
|
||||
@ro_handler.clear_active_connections!
|
||||
assert_not_predicate @ro_handler, :active_connections?
|
||||
end
|
||||
|
||||
def test_retrieve_connection_pool
|
||||
assert_not_nil @rw_handler.retrieve_connection_pool(@spec_name)
|
||||
assert_not_nil @ro_handler.retrieve_connection_pool(@spec_name)
|
||||
end
|
||||
|
||||
def test_retrieve_connection_pool_with_invalid_id
|
||||
assert_nil @rw_handler.retrieve_connection_pool("foo")
|
||||
assert_nil @ro_handler.retrieve_connection_pool("foo")
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
Loading…
Reference in a new issue