ActiveJob with Sidekiq on Rails

This article introduces how to install ActiveJob using Sidekiq in Rails.

🐞 Installation

Prepare Redis

Install Redis on macOS by Homebrew:

brew install redis

Install Redis on Linux(CentOS) by yum or etc:

sudo yum install -y redis
sudo service redis start

Edit Gemfile

Add some gem names to Gemfile. After then, please execute bundle install in terminal.

# Job queue
gem 'sidekiq'
gem 'sinatra', require: false
gem 'redis-namespace'

🎳 Configuration

Please edit config/application.rb:

config.active_job.queue_adapter = :sidekiq
Sidekiq.configure_server { |c| c.redis = { url: ENV['REDIS_URL'] } }

Please add & edit config/sidekiq.yml:

:concurrency: 2
:logfile: ./log/sidekiq.log
:queues:
  - default

Please edit routes.rb:

require 'sidekiq/web'

Rails.application.routes.draw do
  mount Sidekiq::Web => '/sidekiq'
end

After then, you can see sidekiq admin page from http://localhost:3000/sidekiq.

🎉 How to create class of ActiveJob

rails g job hoge

after then, please edit job file.

class HogeJob < ActiveJob::Base
  queue_as :default

  def perform(*args)
    # Do something later
  end
end

If you want to call the job function, please write the follow.

HogeJob.perform_later("hoge")

🍣 Admin page

Authentication with Devise

If you want to add authentication with Devise, please add config/routes.rb.

# config/routes.rb
authenticate :user do
  mount Sidekiq::Web => '/sidekiq'
end

Basic Authentication

require "sidekiq/web"

Sidekiq::Web.use Rack::Auth::Basic do |username, password|
  # Protect against timing attacks:
  # - See https://codahale.com/a-lesson-in-timing-attacks/
  # - See https://thisdata.com/blog/timing-attacks-against-string-comparison/
  # - Use & (do not use &&) so that it doesn't short circuit.
  # - Use digests to stop length information leaking (see also ActiveSupport::SecurityUtils.variable_size_secure_compare)
  ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_USERNAME"])) &
    ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_PASSWORD"]))
end if Rails.env.production?

mount Sidekiq::Web, at: "/sidekiq"

🐝 Execution

You can run a process of sidekiq by as follows:

bundle exec sidekiq -C config/sidekiq.yml

🤔 RSpec

Before you write RSpec for the above job, you should set configuration to spec/rails_helper.rb.

RSpec.configure do |config|
  config.include ActiveJob::TestHelper, type: :job
  # ...
end

Now, you can write RSpec for the Job in spec/jobs/example_job_spec.rb like this:

require 'rails_helper'

RSpec.describe ExampleJob, type: :job do
  subject(:job) { described_class.perform_later(123) }

  it 'queues the job' do
    expect { job }
      .to change(ActiveJob::Base.queue_adapter.enqueued_jobs, :size).by(1)
  end

  it 'is in urgent queue' do
    expect(MyJob.new.queue_name).to eq('urgent')
  end

  it 'executes perform' do
    perform_enqueued_jobs { job }
    # expect xxxx
  end

  it 'handles no results error' do
    allow(MyService).to receive(:call).and_raise(NoResultsError)

    perform_enqueued_jobs do
      expect_any_instance_of(MyJob)
        .to receive(:retry_job).with(wait: 10.minutes, queue: :default)

      job
    end
  end

  after do
    clear_enqueued_jobs
    clear_performed_jobs
  end
end

😎 Health Check by rake task(cron job)

namespace :active_job_sidekiq_queue do
  desc 'Check queue status of SideKiq'
  SIDEKIQ_MAX_WAIT_QUEUE = ENV['SIDEKIQ_MAX_WAIT_QUEUE'].presence || 100
  SIDEKIQ_MAX_RETRY_QUEUE = ENV['SIDEKIQ_MAX_RETRY_QUEUE'].presence || 20

  task check_health: :environment do
    messages = []
    if Sidekiq::Queue.new.size.to_i > SIDEKIQ_MAX_WAIT_QUEUE.to_i
      messages << "SideKiq waiting queue count exceeds #{SIDEKIQ_MAX_WAIT_QUEUE}. Currently, #{Sidekiq::Queue.new.size}. Please check SideKiq."
    end

    if Sidekiq::RetrySet.new.size.to_i > SIDEKIQ_MAX_RETRY_QUEUE.to_i
      messages <<  "SideKiq retry queue count exceeds #{SIDEKIQ_MAX_RETRY_QUEUE}. Currently, #{Sidekiq::RetrySet.new.size}. Please check SideKiq."
    end

    if messages.present?
      raise messages.join("\n") # raise error and send the messages to bug tracker
    end
  end
end

🎃 Health Check by API(Zapix)

Zapix と API を組み合わせて Health Check を行う方法もあるようです。

There is a a way to check queue status by combining Zapix and API.

require 'sidekiq/api'

# Count of waiting queue
get "queue-status" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.size.to_s ]] }

# Count of retrying queues
get "queue-retry" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::RetrySet.new.size.to_s ]] }

# Queue wait time
get "queue-latency" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.latency.to_s]] }

🐠 Special Thanks

🖥 Recommended VPS Service

VULTR provides high performance cloud compute environment for you. Vultr has 15 data-centers strategically placed around the globe, you can use a VPS with 512 MB memory for just $ 2.5 / month ($ 0.004 / hour). In addition, Vultr is up to 4 times faster than the competition, so please check it => Check Benchmark Results!!