ActiveJob with Sidekiq on Rails


This article introduces how to install ActiveJob using Sidekiq in Rails.

๐Ÿ  Installation

Prepare Redis

Install Redis on macOS by Homebrew:

brew install redis

Install Redis on Linux(CentOS) by yum or etc:

sudo yum install -y redis
sudo service redis start

Edit Gemfile

Add some gem names to Gemfile. After then, please execute bundle install in terminal.

# Job queue
gem 'sidekiq'
gem 'sinatra', require: false
gem 'redis-namespace'

๐Ÿ˜ผ Configuration

Please edit config/application.rb:

config.active_job.queue_adapter = :sidekiq
Sidekiq.configure_server { |c| c.redis = { url: ENV['REDIS_URL'] } }

Please add & edit config/sidekiq.yml:

:concurrency: 2
:logfile: ./log/sidekiq.log
:queues:
- default

Please edit routes.rb:

require 'sidekiq/web'

Rails.application.routes.draw do
mount Sidekiq::Web => '/sidekiq'
end

After then, you can see sidekiq admin page from http://localhost:3000/sidekiq.

๐Ÿน How to create class of ActiveJob

rails g job hoge

after then, please edit job file.

class HogeJob < ActiveJob::Base
queue_as :default

def perform(*args)
# Do something later
end
end

If you want to call the job function, please write the follow.

HogeJob.perform_later("hoge")

๐ŸšŒ Admin page

Authentication with Devise

If you want to add authentication with Devise, please add config/routes.rb.

# config/routes.rb
authenticate :user do
mount Sidekiq::Web => '/sidekiq'
end

Basic Authentication

require "sidekiq/web"

Sidekiq::Web.use Rack::Auth::Basic do |username, password|
# Protect against timing attacks:
# - See https://codahale.com/a-lesson-in-timing-attacks/
# - See https://thisdata.com/blog/timing-attacks-against-string-comparison/
# - Use & (do not use &&) so that it doesn't short circuit.
# - Use digests to stop length information leaking (see also ActiveSupport::SecurityUtils.variable_size_secure_compare)
ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(username), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_USERNAME"])) &
ActiveSupport::SecurityUtils.secure_compare(::Digest::SHA256.hexdigest(password), ::Digest::SHA256.hexdigest(ENV["SIDEKIQ_PASSWORD"]))
end if Rails.env.production?

mount Sidekiq::Web, at: "/sidekiq"

๐Ÿ„ Execution

You can run a process of sidekiq by as follows:

bundle exec sidekiq -C config/sidekiq.yml

๐Ÿš• RSpec

Before you write RSpec for the above job, you should set configuration to spec/rails_helper.rb.

RSpec.configure do |config|
config.include ActiveJob::TestHelper, type: :job
# ...
end

Now, you can write RSpec for the Job in spec/jobs/example_job_spec.rb like this:

require 'rails_helper'

RSpec.describe ExampleJob, type: :job do
subject(:job) { described_class.perform_later(123) }

it 'queues the job' do
expect { job }
.to change(ActiveJob::Base.queue_adapter.enqueued_jobs, :size).by(1)
end

it 'is in urgent queue' do
expect(MyJob.new.queue_name).to eq('urgent')
end

it 'executes perform' do
perform_enqueued_jobs { job }
# expect xxxx
end

it 'handles no results error' do
allow(MyService).to receive(:call).and_raise(NoResultsError)

perform_enqueued_jobs do
expect_any_instance_of(MyJob)
.to receive(:retry_job).with(wait: 10.minutes, queue: :default)

job
end
end

after do
clear_enqueued_jobs
clear_performed_jobs
end
end

๐Ÿ—ป Health Check by rake task(cron job)

namespace :active_job_sidekiq_queue do
desc 'Check queue status of SideKiq'
SIDEKIQ_MAX_WAIT_QUEUE = ENV['SIDEKIQ_MAX_WAIT_QUEUE'].presence || 100
SIDEKIQ_MAX_RETRY_QUEUE = ENV['SIDEKIQ_MAX_RETRY_QUEUE'].presence || 20

task check_health: :environment do
messages = []
if Sidekiq::Queue.new.size.to_i > SIDEKIQ_MAX_WAIT_QUEUE.to_i
messages << "SideKiq waiting queue count exceeds #{SIDEKIQ_MAX_WAIT_QUEUE}. Currently, #{Sidekiq::Queue.new.size}. Please check SideKiq."
end

if Sidekiq::RetrySet.new.size.to_i > SIDEKIQ_MAX_RETRY_QUEUE.to_i
messages << "SideKiq retry queue count exceeds #{SIDEKIQ_MAX_RETRY_QUEUE}. Currently, #{Sidekiq::RetrySet.new.size}. Please check SideKiq."
end

if messages.present?
raise messages.join("\n") # raise error and send the messages to bug tracker
end
end
end

๐Ÿ€ Health Check by API(Zapix)

Zapix ใจ API ใ‚’็ต„ใฟๅˆใ‚ใ›ใฆ Health Check ใ‚’่กŒใ†ๆ–นๆณ•ใ‚‚ใ‚ใ‚‹ใ‚ˆใ†ใงใ™ใ€‚

There is a a way to check queue status by combining Zapix and API.

require 'sidekiq/api'

# Count of waiting queue
get "queue-status" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.size.to_s ]] }

# Count of retrying queues
get "queue-retry" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::RetrySet.new.size.to_s ]] }

# Queue wait time
get "queue-latency" => proc { [200, {"Content-Type" => "text/plain"}, [Sidekiq::Queue.new.latency.to_s]] }

๐ŸŽณ Special Thanks

๐Ÿ–ฅ Recommended VPS Service

VULTR provides high performance cloud compute environment for you. Vultr has 15 data-centers strategically placed around the globe, you can use a VPS with 512 MB memory for just $ 2.5 / month ($ 0.004 / hour). In addition, Vultr is up to 4 times faster than the competition, so please check it => Check Benchmark Results!!