1
0
Fork 0
mirror of https://github.com/rails/rails.git synced 2022-11-09 12:12:34 -05:00

Added RDoc for each Active Job adapter

This commit is contained in:
Cristian Bica 2014-09-21 23:20:23 +03:00
parent 3f39ac4ee8
commit c9a4c2a5ce
14 changed files with 192 additions and 69 deletions

View file

@ -24,9 +24,9 @@ Set the queue adapter for Active Job:
``` ruby ``` ruby
ActiveJob::Base.queue_adapter = :inline # default queue adapter ActiveJob::Base.queue_adapter = :inline # default queue adapter
# Adapters currently supported: :backburner, :delayed_job, :qu, :que, :queue_classic,
# :resque, :sidekiq, :sneakers, :sucker_punch
``` ```
Note: To learn how to use your preferred queueing backend see its adapter
documentation at ActiveJob::QueueAdapters.
Declare a job like so: Declare a job like so:
@ -88,18 +88,9 @@ by default has been mixed into Active Record classes.
## Supported queueing systems ## Supported queueing systems
We currently have adapters for: Active Job has built-in adapters for multiple queueing backends (Sidekiq,
Resque, Delayed Job and others). To get an up-to-date list of the adapters
* [Backburner](https://github.com/nesquena/backburner) see the API Documentation for [ActiveJob::QueueAdapters](http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html).
* [Delayed Job](https://github.com/collectiveidea/delayed_job)
* [Qu](https://github.com/bkeepers/qu)
* [Que](https://github.com/chanks/que)
* [QueueClassic](https://github.com/ryandotsmith/queue_classic)
* [Resque 1.x](https://github.com/resque/resque)
* [Sidekiq](https://github.com/mperham/sidekiq)
* [Sneakers](https://github.com/jondot/sneakers)
* [Sucker Punch](https://github.com/brandonhilkert/sucker_punch)
## Auxiliary gems ## Auxiliary gems

View file

@ -1,4 +1,38 @@
module ActiveJob module ActiveJob
# == Active Job adapters
#
# Active Job has adapters for the following queueing backends:
#
# * {Backburner}[https://github.com/nesquena/backburner]
# * {Delayed Job}[https://github.com/collectiveidea/delayed_job]
# * {Qu}[https://github.com/bkeepers/qu]
# * {Que}[https://github.com/chanks/que]
# * {QueueClassic 2.x}[https://github.com/ryandotsmith/queue_classic/tree/v2.2.3]
# * {Resque 1.x}[https://github.com/resque/resque/tree/1-x-stable]
# * {Sidekiq}[http://sidekiq.org]
# * {Sneakers}[https://github.com/jondot/sneakers]
# * {Sucker Punch}[https://github.com/brandonhilkert/sucker_punch]
#
# #### Backends Features
#
# | | Async | Queues | Delayed | Priorities | Timeout | Retries |
# |-------------------|-------|--------|-----------|------------|---------|---------|
# | Backburner | Yes | Yes | Yes | Yes | Job | Global |
# | Delayed Job | Yes | Yes | Yes | Job | Global | Global |
# | Que | Yes | Yes | Yes | Job | No | Job |
# | Queue Classic | Yes | Yes | No* | No | No | No |
# | Resque | Yes | Yes | Yes (Gem) | Queue | Global | Yes |
# | Sidekiq | Yes | Yes | Yes | Queue | No | Job |
# | Sneakers | Yes | Yes | No | Queue | Queue | No |
# | Sucker Punch | Yes | Yes | No | No | No | No |
# | Active Job Inline | No | Yes | N/A | N/A | N/A | N/A |
# | Active Job | Yes | Yes | Yes | No | No | No |
#
# NOTE:
# Queue Classic does not support Job scheduling. However you can implement this
# yourself or you can use the queue_classic-later gem. See the documentation for
# ActiveJob::QueueAdapters::QueueClassicAdapter.
#
module QueueAdapters module QueueAdapters
extend ActiveSupport::Autoload extend ActiveSupport::Autoload

View file

@ -2,13 +2,23 @@ require 'backburner'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Backburner adapter for Active Job
#
# Backburner is a beanstalkd-powered job queue that can handle a very
# high volume of jobs. You create background jobs and place them on
# multiple work queues to be processed later. Read more about
# Backburner {here}[https://github.com/nesquena/backburner].
#
# To use Backburner set the queue_adapter config to +:backburner+.
#
# Rails.application.config.active_job.queue_adapter = :backburner
class BackburnerAdapter class BackburnerAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
delay = timestamp - Time.current.to_f delay = timestamp - Time.current.to_f
Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name, delay: delay Backburner::Worker.enqueue JobWrapper, [ job.serialize ], queue: job.queue_name, delay: delay
end end

View file

@ -2,13 +2,23 @@ require 'delayed_job'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Delayed Job adapter for Active Job
#
# Delayed::Job (or DJ) encapsulates the common pattern of asynchronously
# executing longer tasks in the background. Although DJ can have many
# storage backends one of the most used is based on Active Record.
# Read more about Delayed Job {here}[https://github.com/collectiveidea/delayed_job].
#
# To use Delayed Job set the queue_adapter config to +:delayed_job+.
#
# Rails.application.config.active_job.queue_adapter = :delayed_job
class DelayedJobAdapter class DelayedJobAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
JobWrapper.new.delay(queue: job.queue_name).perform(job.serialize) JobWrapper.new.delay(queue: job.queue_name).perform(job.serialize)
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
JobWrapper.new.delay(queue: job.queue_name, run_at: Time.at(timestamp)).perform(job.serialize) JobWrapper.new.delay(queue: job.queue_name, run_at: Time.at(timestamp)).perform(job.serialize)
end end
end end

View file

@ -1,12 +1,20 @@
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Active Job Inline adapter
#
# When enqueueing jobs with the Inline adapter the job will be executed
# immediately.
#
# To use the Inline set the queue_adapter config to +:inline+.
#
# Rails.application.config.active_job.queue_adapter = :inline
class InlineAdapter class InlineAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
Base.execute(job.serialize) Base.execute(job.serialize)
end end
def enqueue_at(*) def enqueue_at(*) #:nodoc:
raise NotImplementedError.new("Use a queueing backend to enqueue jobs in the future. Read more at https://github.com/rails/activejob") raise NotImplementedError.new("Use a queueing backend to enqueue jobs in the future. Read more at https://github.com/rails/activejob")
end end
end end

View file

@ -2,15 +2,28 @@ require 'qu'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Qu adapter for Active Job
#
# Qu is a Ruby library for queuing and processing background jobs. It is
# heavily inspired by delayed_job and Resque. Qu was created to overcome
# some shortcomings in the existing queuing libraries that we experienced.
# The advantages of Qu are: Multiple backends (redis, mongo), jobs are
# requeued when worker is killed, resque-like API.
#
# Read more about Que {here}[https://github.com/bkeepers/qu].
#
# To use Qu set the queue_adapter config to +:qu+.
#
# Rails.application.config.active_job.queue_adapter = :qu
class QuAdapter class QuAdapter
class << self class << self
def enqueue(job, *args) def enqueue(job, *args) #:nodoc:
Qu::Payload.new(klass: JobWrapper, args: [job.serialize]).tap do |payload| Qu::Payload.new(klass: JobWrapper, args: [job.serialize]).tap do |payload|
payload.instance_variable_set(:@queue, job.queue_name) payload.instance_variable_set(:@queue, job.queue_name)
end.push end.push
end end
def enqueue_at(job, timestamp, *args) def enqueue_at(job, timestamp, *args) #:nodoc:
raise NotImplementedError raise NotImplementedError
end end
end end

View file

@ -2,13 +2,25 @@ require 'que'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Que adapter for Active Job
#
# Que is a high-performance alternative to DelayedJob or QueueClassic that
# improves the reliability of your application by protecting your jobs with
# the same ACID guarantees as the rest of your data. Que is a queue for
# Ruby and PostgreSQL that manages jobs using advisory locks.
#
# Read more about Que {here}[https://github.com/chanks/que].
#
# To use Que set the queue_adapter config to +:que+.
#
# Rails.application.config.active_job.queue_adapter = :que
class QueAdapter class QueAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
JobWrapper.enqueue job.serialize, queue: job.queue_name JobWrapper.enqueue job.serialize, queue: job.queue_name
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
JobWrapper.enqueue job.serialize, queue: job.queue_name, run_at: Time.at(timestamp) JobWrapper.enqueue job.serialize, queue: job.queue_name, run_at: Time.at(timestamp)
end end
end end

View file

@ -2,13 +2,27 @@ require 'queue_classic'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Queue Classic adapter for Active Job
#
# queue_classic provides a simple interface to a PostgreSQL-backed message
# queue. queue_classic specializes in concurrent locking and minimizing
# database load while providing a simple, intuitive developer experience.
# queue_classic assumes that you are already using PostgreSQL in your
# production environment and that adding another dependency (e.g. redis,
# beanstalkd, 0mq) is undesirable.
#
# Read more about Queue Classic {here}[https://github.com/ryandotsmith/queue_classic].
#
# To use Queue Classic set the queue_adapter config to +:queue_classic+.
#
# Rails.application.config.active_job.queue_adapter = :queue_classic
class QueueClassicAdapter class QueueClassicAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
build_queue(job.queue_name).enqueue("#{JobWrapper.name}.perform", job.serialize) build_queue(job.queue_name).enqueue("#{JobWrapper.name}.perform", job.serialize)
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
queue = build_queue(job.queue_name) queue = build_queue(job.queue_name)
unless queue.respond_to?(:enqueue_at) unless queue.respond_to?(:enqueue_at)
raise NotImplementedError, 'To be able to schedule jobs with Queue Classic ' \ raise NotImplementedError, 'To be able to schedule jobs with Queue Classic ' \

View file

@ -14,13 +14,24 @@ end
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Resque adapter for Active Job
#
# Resque (pronounced like "rescue") is a Redis-backed library for creating
# background jobs, placing those jobs on multiple queues, and processing
# them later.
#
# Read more about Resque {here}[https://github.com/resque/resque].
#
# To use Resque set the queue_adapter config to +:resque+.
#
# Rails.application.config.active_job.queue_adapter = :resque
class ResqueAdapter class ResqueAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
Resque.enqueue_to job.queue_name, JobWrapper, job.serialize Resque.enqueue_to job.queue_name, JobWrapper, job.serialize
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
unless Resque.respond_to?(:enqueue_at_with_queue) unless Resque.respond_to?(:enqueue_at_with_queue)
raise NotImplementedError, "To be able to schedule jobs with Resque you need the " \ raise NotImplementedError, "To be able to schedule jobs with Resque you need the " \
"resque-scheduler gem. Please add it to your Gemfile and run bundle install" "resque-scheduler gem. Please add it to your Gemfile and run bundle install"

View file

@ -2,9 +2,21 @@ require 'sidekiq'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Sidekiq adapter for Active Job
#
# Simple, efficient background processing for Ruby. Sidekiq uses threads to
# handle many jobs at the same time in the same process. It does not
# require Rails but will integrate tightly with Rails 3/4 to make
# background processing dead simple.
#
# Read more about Sidekiq {here}[http://sidekiq.org].
#
# To use Sidekiq set the queue_adapter config to +:sidekiq+.
#
# Rails.application.config.active_job.queue_adapter = :sidekiq
class SidekiqAdapter class SidekiqAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
#Sidekiq::Client does not support symbols as keys #Sidekiq::Client does not support symbols as keys
Sidekiq::Client.push \ Sidekiq::Client.push \
'class' => JobWrapper, 'class' => JobWrapper,
@ -13,7 +25,7 @@ module ActiveJob
'retry' => true 'retry' => true
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
Sidekiq::Client.push \ Sidekiq::Client.push \
'class' => JobWrapper, 'class' => JobWrapper,
'queue' => job.queue_name, 'queue' => job.queue_name,

View file

@ -3,18 +3,30 @@ require 'thread'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Sneakers adapter for Active Job
#
# A high-performance RabbitMQ background processing framework for Ruby.
# Sneakers is being used in production for both I/O and CPU intensive
# workloads, and have achieved the goals of high-performance and
# 0-maintenance, as designed.
#
# Read more about Sneakers {here}[https://github.com/jondot/sneakers].
#
# To use Sneakers set the queue_adapter config to +:sneakers+.
#
# Rails.application.config.active_job.queue_adapter = :sneakers
class SneakersAdapter class SneakersAdapter
@monitor = Monitor.new @monitor = Monitor.new
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
@monitor.synchronize do @monitor.synchronize do
JobWrapper.from_queue job.queue_name JobWrapper.from_queue job.queue_name
JobWrapper.enqueue ActiveSupport::JSON.encode(job.serialize) JobWrapper.enqueue ActiveSupport::JSON.encode(job.serialize)
end end
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
raise NotImplementedError raise NotImplementedError
end end
end end

View file

@ -2,13 +2,28 @@ require 'sucker_punch'
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Sucker Punch adapter for Active Job
#
# Sucker Punch is a single-process Ruby asynchronous processing library.
# It's girl_friday and DSL sugar on top of Celluloid. With Celluloid's
# actor pattern, we can do asynchronous processing within a single process.
# This reduces costs of hosting on a service like Heroku along with the
# memory footprint of having to maintain additional jobs if hosting on
# a dedicated server. All queues can run within a single Rails/Sinatra
# process.
#
# Read more about Sucker Punch {here}[https://github.com/brandonhilkert/sucker_punch].
#
# To use Sucker Punch set the queue_adapter config to +:sucker_punch+.
#
# Rails.application.config.active_job.queue_adapter = :sucker_punch
class SuckerPunchAdapter class SuckerPunchAdapter
class << self class << self
def enqueue(job) def enqueue(job) #:nodoc:
JobWrapper.new.async.perform job.serialize JobWrapper.new.async.perform job.serialize
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
raise NotImplementedError raise NotImplementedError
end end
end end

View file

@ -1,5 +1,14 @@
module ActiveJob module ActiveJob
module QueueAdapters module QueueAdapters
# == Test adapter for Active Job
#
# The test adapter should be used only in testing. Along with
# <tt>ActiveJob::TestCase</tt> and <tt>ActiveJob::TestHelper</tt>
# it makes a great tool to test your Rails application.
#
# To use the test adapter set queue_adapter config to +:test+.
#
# Rails.application.config.active_job.queue_adapter = :test
class TestAdapter class TestAdapter
delegate :name, to: :class delegate :name, to: :class
attr_accessor(:perform_enqueued_jobs, :perform_enqueued_at_jobs) attr_accessor(:perform_enqueued_jobs, :perform_enqueued_at_jobs)
@ -15,7 +24,7 @@ module ActiveJob
@performed_jobs ||= [] @performed_jobs ||= []
end end
def enqueue(job) def enqueue(job) #:nodoc:
if perform_enqueued_jobs if perform_enqueued_jobs
performed_jobs << {job: job.class, args: job.arguments, queue: job.queue_name} performed_jobs << {job: job.class, args: job.arguments, queue: job.queue_name}
job.perform_now job.perform_now
@ -24,7 +33,7 @@ module ActiveJob
end end
end end
def enqueue_at(job, timestamp) def enqueue_at(job, timestamp) #:nodoc:
if perform_enqueued_at_jobs if perform_enqueued_at_jobs
performed_jobs << {job: job.class, args: job.arguments, queue: job.queue_name, at: timestamp} performed_jobs << {job: job.class, args: job.arguments, queue: job.queue_name, at: timestamp}
job.perform_now job.perform_now

View file

@ -99,41 +99,13 @@ If no adapter is set, the job is immediately executed.
### Backends ### Backends
Active Job has adapters for the following queueing backends: Active Job has built-in adapters for multiple queueing backends (Sidekiq,
Resque, Delayed Job and others). To get an up-to-date list of the adapters
see the API Documentation for [ActiveJob::QueueAdapters](http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html).
* [Backburner](https://github.com/nesquena/backburner) ### Changing the Backend
* [Delayed Job](https://github.com/collectiveidea/delayed_job)
* [Qu](https://github.com/bkeepers/qu)
* [Que](https://github.com/chanks/que)
* [QueueClassic 2.x](https://github.com/ryandotsmith/queue_classic/tree/v2.2.3)
* [Resque 1.x](https://github.com/resque/resque/tree/1-x-stable)
* [Sidekiq](https://github.com/mperham/sidekiq)
* [Sneakers](https://github.com/jondot/sneakers)
* [Sucker Punch](https://github.com/brandonhilkert/sucker_punch)
#### Backends Features You can easily change your queueing backend:
| | Async | Queues | Delayed | Priorities | Timeout | Retries |
|-----------------------|-------|--------|-----------|------------|---------|---------|
| **Backburner** | Yes | Yes | Yes | Yes | Job | Global |
| **Delayed Job** | Yes | Yes | Yes | Job | Global | Global |
| **Que** | Yes | Yes | Yes | Job | No | Job |
| **Queue Classic** | Yes | Yes | No* | No | No | No |
| **Resque** | Yes | Yes | Yes (Gem) | Queue | Global | Yes |
| **Sidekiq** | Yes | Yes | Yes | Queue | No | Job |
| **Sneakers** | Yes | Yes | No | Queue | Queue | No |
| **Sucker Punch** | Yes | Yes | No | No | No | No |
| **Active Job Inline** | No | Yes | N/A | N/A | N/A | N/A |
| **Active Job** | Yes | Yes | Yes | No | No | No |
NOTE:
* Queue Classic does not support Job scheduling. However you can implement this
yourself or you can use the queue_classic-later gem. See the documentation for
ActiveJob::QueueAdapters::QueueClassicAdapter.
### Change Backends
You can easily change your adapter:
```ruby ```ruby
# be sure to have the adapter gem in your Gemfile and follow the adapter specific # be sure to have the adapter gem in your Gemfile and follow the adapter specific