While we query the proper columns, we go through normal handling for
converting the value to a primitive which assumes it should use the
table's primary key. If the association specifies a different value (and
we know that we're working with an association), we should use the
custom primary key instead.
Fixes#18813.
It's finally finished!!!!!!! The reason the Attributes API was kept
private in 4.2 was due to some publicly visible implementation details.
It was previously implemented by overloading `columns` and
`columns_hash`, to make them return column objects which were modified
with the attribute information.
This meant that those methods LIED! We didn't change the database
schema. We changed the attribute information on the class. That is
wrong! It should be the other way around, where schema loading just
calls the attributes API for you. And now it does!
Yes, this means that there is nothing that happens in automatic schema
loading that you couldn't manually do yourself. (There's still some
funky cases where we hit the connection adapter that I need to handle,
before we can turn off automatic schema detection entirely.)
There were a few weird test failures caused by this that had to be
fixed. The main source came from the fact that the attribute methods are
now defined in terms of `attribute_names`, which has a clause like
`return [] unless table_exists?`. I don't *think* this is an issue,
since the only place this caused failures were in a fake adapter which
didn't override `table_exists?`.
Additionally, there were a few cases where tests were failing because a
migration was run, but the model was not reloaded. I'm not sure why
these started failing from this change, I might need to clear an
additional cache in `reload_schema_from_cache`. Again, since this is not
normal usage, and it's expected that `reset_column_information` will be
called after the table is modified, I don't think it's a problem.
Still, test failures that were unrelated to the change are worrying, and
I need to dig into them further.
Finally, I spent a lot of time debugging issues with the mutex used in
`define_attribute_methods`. I think we can just remove that method
entirely, and define the attribute methods *manually* in the call to
`define_attribute`, which would simplify the code *tremendously*.
Ok. now to make this damn thing public, and work on moving it up to
Active Model.
`bound_attributes` is now used universally across the board, removing
the need for the conversion layer. These changes are mostly mechanical,
with the exception of the log subscriber. Additional, we had to
implement `hash` on the attribute objects, so they could be used as a
key for query caching.
The column is primarily used for type casting, which we're trying to
separate from the idea of a column. Since what we really need is the
combination of a name, type, and value, let's use the object that we
already have to represent that concept, rather than this tuple. No
consumers of the bind values have been changed, only the producers
(outside of tests which care too much about internals). This is
*finally* possible since the bind values are now produced from a
reasonable number of lcoations.
The only place it was accessed was in tests. Many of them have another
way that they can test their behavior, that doesn't involve reaching
into internals as far as they did. `AssociationScopeTest` is testing a
situation where the where clause would have one bind param per
predicate, so it can just ignore the predicates entirely. The where
chain test was primarly duplicating the logic tested on `WhereClause`
directly, so I instead just make sure it calls the appropriate method
which is fully tested in isolation.
The bind values can come from four places. `having`, `where`, `joins`,
and `from` when selecting from a subquery that contains binds. These
need to be kept in a specific order, since the clauses will always
appear in that order. Up until recently, they were not.
Additionally, `joins` actually did keep its bind values in a separate
location (presumably because it's the only case that people noticed was
broken). However, this meant that anything accessing just `bind_values`
was broken (which most places were). This is no longer possible, there
is only a single way to access the bind values, and it includes joins in
the proper location. The setter was removed yesterday, so breaking `+=`
cases is not possible.
I'm still not happy that `joins` is putting it's bind values on the
Arel AST, and I'm planning on refactoring it further, but this removes a
ton of bug cases.
The last place that was assigning it was when `from` is called with a
relation to use as a subquery. The implementation was actually
completely broken, and would break if you called `from` more than once,
or if you called it on a relation, which also had its own join clause,
as the bind values would get completely scrambled. The simplest solution
was to just move it into its own array, since creating a `FromClause`
class for this would be overkill.
This fixed an issue where `having` can only be called after the last
call to `where`, because it messes with the same `bind_values` array.
With this change, the two can be called as many times as needed, in any
order, and the final query will be correct. However, once something
assigns `bind_values`, that stops. This is because we have to move all
of the bind values from the having clause over to the where clause since
we can't differentiate the two, and assignment was likely in the form
of:
`relation.bind_values += other.bind_values`
This will go away once we remove all places that are assigning
`bind_values`, which is next on the list.
While this fixes a bug that was present in at least 4.2 (more likely
present going back as far as 3.0, becoming more likely in 4.1 and later
as we switched to prepared statements in more cases), I don't think this
can be easily backported. The internal changes to `Relation` are
non-trivial, anything that involves modifying the `bind_values` array
would need to change, and I'm not confident that we have sufficient test
coverage of all of those locations (when `having` was called with a hash
that could generate bind values).
[Sean Griffin & anthonynavarre]
The code assumes that non-single-value methods mean multi value methods.
That is not the case. We need to change the accessor name, and only
assign an array for multi value methods
This object being a black box, it knows the details of how to merge
itself with another where clause. This removes all references to where
values or bind values in `Relation::Merger`
The way that bind values are currently stored on Relation is a mess.
They can come from `having`, `where`, or `join`. I'm almost certain that
`having` is actually broken, and calling `where` followed by `having`
followed by `where` will completely scramble the binds.
Joins don't actually add the bind parameters to the relation itself, but
instead add it onto an accessor on the arel AST which is undocumented,
and unused in Arel itself. This means that the bind values must always
be accessed as `relation.arel.bind_values + relation.bind_values`.
Anything that doesn't is likely broken (and tons of bugs have come up
for exactly that reason)
The result is that everything dealing with `Relation` instances has to
know far too much about the internals. The binds are split, combined,
and re-stored in non-obvious ways that makes it difficult to change
anything about the internal representation of `bind_values`, and is
extremely prone to bugs.
So the goal is to move a lot of logic off of `Relation`, and into
separate objects. This is not the same as what is currently done with
`JoinDependency`, as `Relation` knows far too much about its internals,
and vice versa. Instead these objects need to be black boxes that can
have their implementations swapped easily.
The end result will be two classes, `WhereClause` and `JoinClause`
(`having` will just re-use `WhereClause`), and there will be a single
method to access the bind values of a `Relation` which will be
implemented as
```
join_clause.binds + where_clause.binds + having_clause.binds
```
This is the first step towards that refactoring, with the internal
representation of where changed, and an intermediate representation of
`where_values` and `bind_values` to let the refactoring take small
steps. These will be removed shortly.
This will allow all types which require no additional handling to use
prepared statements. Specifically, this will allow for `true`, `false`,
`Date`, `Time`, and any custom PG type to use prepared statements. This
also revealed another source of nil columns in bind params, and an
inconsistency in their use.
The specific inconsistency comes from a nested query coming from a
through association, where one of the inversed associations is not
bi-directional.
The stop-gap is to simply construct the column at the site it is being
used. This should simply go away on its own once we use `Attribute` to
represent them instead, since we already have all of the information we
need.
This is to help facilitate future refactorings, as the internal
representation is changed. I'm planning on having `where_values` return
an array that's computed on call, which means that mutation will have no
affect. This is the only remaining place that was mutating (tested by
replacing the method with calling `dup`)
Specifically, the issue is relying on `where_unscoping` mutating the
where values. It does not, however, mutate the bind values, which could
cause an error under certain circumstances. This was not exposed by the
tests, since the only place which would have been affected is unscoping
a boolean, which doesn't go through prepared statements. I had a hard
time getting better test coverage to demonstrate the issue.
This in turn, caused `merge` to go through proper logic, and try to
clear out the binds associated with the unscoped relation, which then
exposed a source of `nil` for the columns, as binds weren't expanding
`{ "posts.id" => 1 }` to `{ "posts" => { "id" => 1 } }`. This has been
fixed.
The bulk of `create_binds` needed to be moved to a separate method,
since the dot notation should not be expanded recursively.
I'm pretty sure this removes a subtle quirk that a ton of code in
`Relation::Merger` is working around, and I suspect that code can be
greatly simplified. However, unraveling that rats nest is no small task.
With the old implementation, the bind values were created, and then we
search the attributes for `Relation` objects, and merge them. This
completely ignores the order that the actual `where` clause will use. If
all non-relation where parameters are before the relations, it will
work. However, if we query on both a relation and a value, with the
value coming second, it breaks. The order of the hash should not affect
the final query (especially since hashes being ordered is an
implementation detail)
I'm looking to introduce a `WhereClause` class to handle most of this
logic, and this method will eventually move over to there. However, this
intermediate refactoring should make that easier to do.
Looking through the blame, this logic used to be when we actually
created the bind tuple. My guess is that `nil` couldn't be handled there
at that time. It can, now.