470 lines
14 KiB
Markdown
470 lines
14 KiB
Markdown
|
# Navigating GitLab via Rails console
|
||
|
|
||
|
At the heart of GitLab is a web application [built using the Ruby on Rails
|
||
|
framework](https://about.gitlab.com/blog/2018/10/29/why-we-use-rails-to-build-gitlab/).
|
||
|
Thanks to this, we also get access to the amazing tools built right into Rails.
|
||
|
In this guide, we'll introduce the [Rails console](https://docs.gitlab.com/omnibus/maintenance/#starting-a-rails-console-session)
|
||
|
and the basics of interacting with your GitLab instance from the command line.
|
||
|
|
||
|
CAUTION: **CAUTION:**
|
||
|
The Rails console interacts directly with your GitLab instance. In many cases,
|
||
|
there are no handrails to prevent you from permanently modifying, corrupting
|
||
|
or destroying production data. If you would like to explore the Rails console
|
||
|
with no consequences, you are strongly advised to do so in a test environment.
|
||
|
|
||
|
This guide is targeted at GitLab system administrators who are troubleshooting
|
||
|
a problem or need to retrieve some data that can only be done through direct
|
||
|
access of the GitLab application. Basic knowledge of Ruby is needed (try [this
|
||
|
30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction).
|
||
|
Rails experience is helpful to have but not a must.
|
||
|
|
||
|
## Starting a Rails console session
|
||
|
|
||
|
Omnibus GitLab comes with a convenient wrapper command which automatically loads
|
||
|
the production GitLab environment:
|
||
|
|
||
|
```shell
|
||
|
sudo gitlab-rails console
|
||
|
```
|
||
|
|
||
|
For source installations, you'll have to instead run:
|
||
|
|
||
|
```shell
|
||
|
sudo -u git -H bundle exec rails console RAILS_ENV=production
|
||
|
```
|
||
|
|
||
|
Further code examples will all take place inside the Rails console and also
|
||
|
assume an Omnibus GitLab installation.
|
||
|
|
||
|
## Active Record objects
|
||
|
|
||
|
### Looking up database-persisted objects
|
||
|
|
||
|
Under the hood, Rails uses [Active Record](https://guides.rubyonrails.org/active_record_basics.html),
|
||
|
an object-relational mapping system, to read, write and map application objects
|
||
|
to the PostgreSQL database. These mappings are handled by Active Record models,
|
||
|
which are Ruby classes defined in a Rails app. For GitLab, the model classes
|
||
|
can be found at `/opt/gitlab/embedded/service/gitlab-rails/app/models`.
|
||
|
|
||
|
Let's enable debug logging for Active Record so we can see the underlying
|
||
|
database queries made:
|
||
|
|
||
|
```ruby
|
||
|
ActiveRecord::Base.logger = Logger.new(STDOUT)
|
||
|
```
|
||
|
|
||
|
Now, let's try retrieving a user from the database:
|
||
|
|
||
|
```ruby
|
||
|
user = User.find(1)
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
D, [2020-03-05T16:46:25.571238 #910] DEBUG -- : User Load (1.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
|
||
|
=> #<User id:1 @root>
|
||
|
```
|
||
|
|
||
|
We can see that we've queried the `users` table in the database for a row whose
|
||
|
`id` column has the value `1`, and Active Record has translated that database
|
||
|
record into a Ruby object that we can interact with. Try some of the following:
|
||
|
|
||
|
- `user.username`
|
||
|
- `user.created_at`
|
||
|
- `user.admin`
|
||
|
|
||
|
By convention, column names are directly translated into Ruby object attributes,
|
||
|
so you should be able to do `user.<column_name>` to view the attribute's value.
|
||
|
|
||
|
Also by convention, Active Record class names (singular and in camel case) map
|
||
|
directly onto table names (plural and in snake case) and vice versa. For example,
|
||
|
the `users` table maps to the `User` class, while the `application_settings`
|
||
|
table maps to the `ApplicationSetting` class.
|
||
|
|
||
|
You can find a list of tables and column names in the Rails database schema,
|
||
|
available at `/opt/gitlab/embedded/service/gitlab-rails/db/schema.rb`.
|
||
|
|
||
|
You can also look up an object from the database by attribute name:
|
||
|
|
||
|
```ruby
|
||
|
user = User.find_by(username: 'root')
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
D, [2020-03-05T17:03:24.696493 #910] DEBUG -- : User Load (2.1ms) SELECT "users".* FROM "users" WHERE "users"."username" = 'root' LIMIT 1
|
||
|
=> #<User id:1 @root>
|
||
|
```
|
||
|
|
||
|
Give the following a try:
|
||
|
|
||
|
- `User.find_by(email: 'admin@example.com')`
|
||
|
- `User.where.not(admin: true)`
|
||
|
- `User.where('created_at < ?', 7.days.ago)`
|
||
|
|
||
|
Did you notice that the last two commands returned an `ActiveRecord::Relation`
|
||
|
object that appeared to contain multiple `User` objects?
|
||
|
|
||
|
Up to now, we've been using `.find` or `.find_by`, which are designed to return
|
||
|
only a single object (notice the `LIMIT 1` in the generated SQL query?).
|
||
|
`.where` is used when it is desirable to get a collection of objects.
|
||
|
|
||
|
Let's get a collection of non-admin users and see what we can do with it:
|
||
|
|
||
|
```ruby
|
||
|
users = User.where.not(admin: true)
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
D, [2020-03-05T17:11:16.845387 #910] DEBUG -- : User Load (2.8ms) SELECT "users".* FROM "users" WHERE "users"."admin" != TRUE LIMIT 11
|
||
|
=> #<ActiveRecord::Relation [#<User id:3 @support-bot>, #<User id:7 @alert-bot>, #<User id:5 @carrie>, #<User id:4 @bernice>, #<User id:2 @anne>]>
|
||
|
```
|
||
|
|
||
|
Now, try the following:
|
||
|
|
||
|
- `users.count`
|
||
|
- `users.order(created_at: :desc)`
|
||
|
- `users.where(username: 'support-bot')`
|
||
|
|
||
|
In the last command, we see that we can chain `.where` statements to generate
|
||
|
more complex queries. Notice also that while the collection returned contains
|
||
|
only a single object, we cannot directly interact with it:
|
||
|
|
||
|
```ruby
|
||
|
users.where(username: 'support-bot').username
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
Traceback (most recent call last):
|
||
|
1: from (irb):37
|
||
|
D, [2020-03-05T17:18:25.637607 #910] DEBUG -- : User Load (1.6ms) SELECT "users".* FROM "users" WHERE "users"."admin" != TRUE AND "users"."username" = 'support-bot' LIMIT 11
|
||
|
NoMethodError (undefined method `username' for #<ActiveRecord::Relation [#<User id:3 @support-bot>]>)
|
||
|
Did you mean? by_username
|
||
|
```
|
||
|
|
||
|
We need to retrieve the single object from the collection by using the `.first`
|
||
|
method to get the first item in the collection:
|
||
|
|
||
|
```ruby
|
||
|
users.where(username: 'support-bot').first.username
|
||
|
```
|
||
|
|
||
|
We now get the result we wanted:
|
||
|
|
||
|
```ruby
|
||
|
D, [2020-03-05T17:18:30.406047 #910] DEBUG -- : User Load (2.6ms) SELECT "users".* FROM "users" WHERE "users"."admin" != TRUE AND "users"."username" = 'support-bot' ORDER BY "users"."id" ASC LIMIT 1
|
||
|
=> "support-bot"
|
||
|
```
|
||
|
|
||
|
For more on different ways to retrieve data from the database using Active
|
||
|
Record, please see the [Active Record Query Interface documentation](https://guides.rubyonrails.org/active_record_querying.html).
|
||
|
|
||
|
### Modifying Active Record objects
|
||
|
|
||
|
In the previous section, we learned about retrieving database records using
|
||
|
Active Record. Now, we'll learn how to write changes to the database.
|
||
|
|
||
|
First, let's retrieve the `root` user:
|
||
|
|
||
|
```ruby
|
||
|
user = User.find_by(username: 'root')
|
||
|
```
|
||
|
|
||
|
Next, let's try updating the user's password:
|
||
|
|
||
|
```ruby
|
||
|
user.password = 'password'
|
||
|
user.save
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
Enqueued ActionMailer::DeliveryJob (Job ID: 05915c4e-c849-4e14-80bb-696d5ae22065) to Sidekiq(mailers) with arguments: "DeviseMailer", "password_change", "deliver_now", #<GlobalID:0x00007f42d8ccebe8 @uri=#<URI::GID gid://gitlab/User/1>>
|
||
|
=> true
|
||
|
```
|
||
|
|
||
|
Here, we see that the `.save` command returned `true`, indicating that the
|
||
|
password change was successfully saved to the database.
|
||
|
|
||
|
We also see that the save operation triggered some other action -- in this case
|
||
|
a background job to deliver an email notification. This is an example of an
|
||
|
[Active Record callback](https://guides.rubyonrails.org/active_record_callbacks.html)
|
||
|
-- code which is designated to run in response to events in the Active Record
|
||
|
object life cycle. This is also why using the Rails console is preferred when
|
||
|
direct changes to data is necessary as changes made via direct database queries
|
||
|
will not trigger these callbacks.
|
||
|
|
||
|
It's also possible to update attributes in a single line:
|
||
|
|
||
|
```ruby
|
||
|
user.update(password: 'password')
|
||
|
```
|
||
|
|
||
|
Or update multiple attributes at once:
|
||
|
|
||
|
```ruby
|
||
|
user.update(password: 'password', email: 'hunter2@example.com')
|
||
|
```
|
||
|
|
||
|
Now, let's try something different:
|
||
|
|
||
|
```ruby
|
||
|
# Retrieve the object again so we get its latest state
|
||
|
user = User.find_by(username: 'root')
|
||
|
user.password = 'password'
|
||
|
user.password_confirmation = 'hunter2'
|
||
|
user.save
|
||
|
```
|
||
|
|
||
|
This returns `false`, indicating that the changes we made were not saved to the
|
||
|
database. You can probably guess why, but let's find out for sure:
|
||
|
|
||
|
```ruby
|
||
|
user.save!
|
||
|
```
|
||
|
|
||
|
This should return:
|
||
|
|
||
|
```ruby
|
||
|
Traceback (most recent call last):
|
||
|
1: from (irb):64
|
||
|
ActiveRecord::RecordInvalid (Validation failed: Password confirmation doesn't match Password)
|
||
|
```
|
||
|
|
||
|
Aha! We've tripped an [Active Record Validation](https://guides.rubyonrails.org/active_record_validations.html).
|
||
|
Validations are business logic put in place at the application-level to prevent
|
||
|
unwanted data from being saved to the database and in most cases come with
|
||
|
helpful messages letting you know how to fix the problem inputs.
|
||
|
|
||
|
We can also add the bang (Ruby speak for `!`) to `.update`:
|
||
|
|
||
|
```ruby
|
||
|
user.update!(password: 'password', password_confirmation: 'hunter2')
|
||
|
```
|
||
|
|
||
|
In Ruby, method names ending with `!` are commonly known as "bang methods". By
|
||
|
convention, the bang indicates that the method directly modifies the object it
|
||
|
is acting on, as opposed to returning the transformed result and leaving the
|
||
|
underlying object untouched. For Active Record methods that write to the
|
||
|
database, bang methods also serve an additional function: they raise an
|
||
|
explicit exception whenever an error occurs, instead of just returning `false`.
|
||
|
|
||
|
We can also skip validations entirely:
|
||
|
|
||
|
```ruby
|
||
|
# Retrieve the object again so we get its latest state
|
||
|
user = User.find_by(username: 'root')
|
||
|
user.password = 'password'
|
||
|
user.password_confirmation = 'hunter2'
|
||
|
user.save!(validate: false)
|
||
|
```
|
||
|
|
||
|
This is not recommended, as validations are usually put in place to ensure the
|
||
|
integrity and consistency of user-provided data.
|
||
|
|
||
|
Note that a validation error will prevent the entire object from being saved to
|
||
|
the database. We'll see a little of this in the next section. If you're getting
|
||
|
a mysterious red banner in the GitLab UI when submitting a form, this can often
|
||
|
be the fastest way to get to the root of the problem.
|
||
|
|
||
|
### Interacting with Active Record objects
|
||
|
|
||
|
At the end of the day, Active Record objects are just normal Ruby objects. As
|
||
|
such, we can define methods on them which perform arbitrary actions.
|
||
|
|
||
|
For example, GitLab developers have added some methods which help with
|
||
|
two-factor authentication:
|
||
|
|
||
|
```ruby
|
||
|
def disable_two_factor!
|
||
|
transaction do
|
||
|
update(
|
||
|
otp_required_for_login: false,
|
||
|
encrypted_otp_secret: nil,
|
||
|
encrypted_otp_secret_iv: nil,
|
||
|
encrypted_otp_secret_salt: nil,
|
||
|
otp_grace_period_started_at: nil,
|
||
|
otp_backup_codes: nil
|
||
|
)
|
||
|
self.u2f_registrations.destroy_all # rubocop: disable DestroyAll
|
||
|
end
|
||
|
end
|
||
|
|
||
|
def two_factor_enabled?
|
||
|
two_factor_otp_enabled? || two_factor_u2f_enabled?
|
||
|
end
|
||
|
```
|
||
|
|
||
|
(See: `/opt/gitlab/embedded/service/gitlab-rails/app/models/user.rb`)
|
||
|
|
||
|
We can then use these methods on any user object:
|
||
|
|
||
|
```ruby
|
||
|
user = User.find_by(username: 'root')
|
||
|
user.two_factor_enabled?
|
||
|
user.disable_two_factor!
|
||
|
```
|
||
|
|
||
|
Some methods are defined by gems, or Ruby software packages, which GitLab uses.
|
||
|
For example, the [StateMachines](https://github.com/state-machines/state_machines-activerecord)
|
||
|
gem which GitLab uses to manage user state:
|
||
|
|
||
|
```ruby
|
||
|
state_machine :state, initial: :active do
|
||
|
event :block do
|
||
|
|
||
|
...
|
||
|
|
||
|
event :activate do
|
||
|
|
||
|
...
|
||
|
|
||
|
end
|
||
|
```
|
||
|
|
||
|
Give it a try:
|
||
|
|
||
|
```ruby
|
||
|
user = User.find_by(username: 'root')
|
||
|
user.state
|
||
|
user.block
|
||
|
user.state
|
||
|
user.activate
|
||
|
user.state
|
||
|
```
|
||
|
|
||
|
Earlier, we mentioned that a validation error will prevent the entire object
|
||
|
from being saved to the database. Let's see how this can have unexpected
|
||
|
interactions:
|
||
|
|
||
|
```ruby
|
||
|
user.password = 'password'
|
||
|
user.password_confirmation = 'hunter2'
|
||
|
user.block
|
||
|
```
|
||
|
|
||
|
We get `false` returned! Let's find out what happened by adding a bang as we did
|
||
|
earlier:
|
||
|
|
||
|
```ruby
|
||
|
user.block!
|
||
|
```
|
||
|
|
||
|
Which would return:
|
||
|
|
||
|
```ruby
|
||
|
Traceback (most recent call last):
|
||
|
1: from (irb):87
|
||
|
StateMachines::InvalidTransition (Cannot transition state via :block from :active (Reason(s): Password confirmation doesn't match Password))
|
||
|
```
|
||
|
|
||
|
We see that a validation error from what feels like a completely separate
|
||
|
attribute comes back to haunt us when we try to update the user in any way.
|
||
|
|
||
|
In practical terms, we sometimes see this happen with GitLab admin settings --
|
||
|
validations are sometimes added or changed in a GitLab update, resulting in
|
||
|
previously saved settings now failing validation. Because you can only update
|
||
|
a subset of settings at once through the UI, in this case the only way to get
|
||
|
back to a good state is direct manipulation via Rails console.
|
||
|
|
||
|
### Commonly used Active Record models and how to look up objects
|
||
|
|
||
|
**Get a user by primary email address or username:**
|
||
|
|
||
|
```ruby
|
||
|
User.find_by(email: 'admin@example.com')
|
||
|
User.find_by(username: 'root')
|
||
|
```
|
||
|
|
||
|
**Get a user by primary OR secondary email address:**
|
||
|
|
||
|
```ruby
|
||
|
User.find_by_any_email('user@example.com')
|
||
|
```
|
||
|
|
||
|
Note: `find_by_any_email` is a custom method added by GitLab developers rather
|
||
|
than a Rails-provided default method.
|
||
|
|
||
|
**Get a collection of admin users:**
|
||
|
|
||
|
```ruby
|
||
|
User.admins
|
||
|
```
|
||
|
|
||
|
Note: `admins` is a [scope convenience method](https://guides.rubyonrails.org/active_record_querying.html#scopes)
|
||
|
which does `where(admin: true)` under the hood.
|
||
|
|
||
|
**Get a project by its path:**
|
||
|
|
||
|
```ruby
|
||
|
Project.find_by_full_path('group/subgroup/project')
|
||
|
```
|
||
|
|
||
|
Note: `find_by_full_path` is a custom method added by GitLab developers rather
|
||
|
than a Rails-provided default method.
|
||
|
|
||
|
**Get a project's issue or merge request by its numeric ID:**
|
||
|
|
||
|
```ruby
|
||
|
project = Project.find_by_full_path('group/subgroup/project')
|
||
|
project.issues.find_by(iid: 42)
|
||
|
project.merge_requests.find_by(iid: 42)
|
||
|
```
|
||
|
|
||
|
Note: `iid` means "internal ID" and is how we keep issue and merge request IDs
|
||
|
scoped to each GitLab project.
|
||
|
|
||
|
**Get a group by its path:**
|
||
|
|
||
|
```ruby
|
||
|
Group.find_by_full_path('group/subgroup')
|
||
|
```
|
||
|
|
||
|
**Get a group's related groups:**
|
||
|
|
||
|
```ruby
|
||
|
group = Group.find_by_full_path('group/subgroup')
|
||
|
|
||
|
# Get a group's parent group
|
||
|
group.parent
|
||
|
|
||
|
# Get a group's child groups
|
||
|
group.children
|
||
|
```
|
||
|
|
||
|
**Get a group's projects:**
|
||
|
|
||
|
```ruby
|
||
|
group = Group.find_by_full_path('group/subgroup')
|
||
|
|
||
|
# Get group's immediate child projects
|
||
|
group.projects
|
||
|
|
||
|
# Get group's child projects, including those in sub-groups
|
||
|
group.all_projects
|
||
|
```
|
||
|
|
||
|
**Get CI pipeline or builds:**
|
||
|
|
||
|
```ruby
|
||
|
Ci::Pipeline.find(4151)
|
||
|
Ci::Build.find(66124)
|
||
|
```
|
||
|
|
||
|
Note: The pipeline and job #ID numbers increment globally across your GitLab
|
||
|
instance, so there's no need to use an internal ID attribute to look them up,
|
||
|
unlike with issues or merge requests.
|
||
|
|
||
|
**Get the current application settings object:**
|
||
|
|
||
|
```ruby
|
||
|
ApplicationSetting.current
|
||
|
```
|