DE Let's Talk Let's Talk

Architecture

Migrating delayed_job to Sidekiq Without Data Loss

By USEO Engineering Team Updated

BLUF (Bottom Line Up Front): delayed_job relies on an SQL database, causing severe deadlocks and high queue latency at scale. Sidekiq uses Redis, offering massive concurrency. To migrate safely, you must run both processors simultaneously during the transition and use a script to re-enqueue stranded delayed_job payloads into Sidekiq.

Phase 1: The Relational Bottleneck

delayed_job uses row-level locking in PostgreSQL or MySQL. When hundreds of jobs are processed concurrently, the database spends more CPU time managing locks than executing queries.

Synthetic Engineering Context: High Queue Latency

In your APM (like New Relic), you notice background jobs are taking 15 minutes to start, even though the actual execution time is 200ms.

# Database log showing lock contention
FATAL:  terminating connection due to conflict with recovery
DETAIL:  User query might have needed to see row versions that must be removed.
STATEMENT:  UPDATE "delayed_jobs" SET locked_at = '2026-04-23 10:00:00', locked_by = 'host:worker-1' WHERE id = 15432

Phase 2: The Migration Strategy

A hard cutover will result in lost jobs. You must adopt a dual-boot approach.

Execution: Step 1 - Dual Configuration

Configure ActiveJob to push new jobs to Sidekiq, but keep the delayed_job worker running to drain the old queue.

# config/application.rb
# Direct all NEW jobs to Sidekiq
config.active_job.queue_adapter = :sidekiq

Deploy order matters. Ship the adapter switch first and let the existing delayed_job workers keep draining the old table. Do not stop them until Delayed::Job.where(failed_at: nil).count reaches zero or you run the re-enqueue script below. Running the script while delayed_job workers are still polling the table risks double execution: the script re-enqueues a job into Sidekiq at the same moment a DJ worker locks and runs it. Stop the DJ workers first, then migrate the stragglers.

Execution: Step 2 - The Re-enqueue Script (PoC)

If you have jobs scheduled for weeks in the future (e.g., subscription reminders), you cannot wait for the delayed_job queue to drain naturally. You must extract them from the database and push them to Redis persistency.

# lib/tasks/migrate_jobs.rake
namespace :jobs do
  desc "Migrate scheduled delayed_jobs to Sidekiq"
  task delayed_to_sidekiq: :environment do
    Delayed::Job.find_each do |dj|
      # Parse the YAML payload generated by DelayedJob
      payload = YAML.load(dj.handler)
      
      # Extract the original ActiveJob class and arguments
      job_class = payload.job_data['job_class'].constantize
      args = payload.job_data['arguments']
      
      if dj.run_at > Time.current
        # Schedule it in Sidekiq
        job_class.set(wait_until: dj.run_at).perform_later(*args)
      else
        # Enqueue immediately
        job_class.perform_later(*args)
      end
      
      # Delete the migrated record
      dj.destroy!
    end
  end
end

Execution: Step 3 - Handling Non-ActiveJob Payloads

The script above only works for jobs enqueued through ActiveJob. Legacy codebases predating Rails 4.2 often call object.delay.some_method or handle_asynchronously, which serializes a Delayed::PerformableMethod instead. Two traps here:

  1. Psych 4 breaks the YAML parsing. On Ruby 3.1+ (Psych 4), YAML.load refuses to deserialize arbitrary classes and raises Psych::DisallowedClass. For the migration script you need YAML.unsafe_load(dj.handler), or Delayed::Job deserialization helpers if the gem is still loaded.
  2. PerformableMethod has no Sidekiq equivalent. A handler like --- !ruby/object:Delayed::PerformableMethod wraps an object, a method name, and args. You must rewrite each of these call sites as an explicit ActiveJob class before the migration, otherwise there is nothing to re-enqueue them into.

Grep for the legacy API to size this work before you commit to a cutover date:

grep -rn "\.delay\.\|handle_asynchronously" app/ lib/ | grep -v spec

Retry Semantics Are Not the Same

delayed_job retries 25 times with an (attempts ** 4) + 5 backoff and then leaves the row in the table with failed_at set. Sidekiq also retries ~25 times over about 21 days, but then moves the job to the dead set, which is capped (10,000 jobs by default) and pruned after 6 months. Two consequences:

  • Any alerting built on Delayed::Job.where.not(failed_at: nil).count must be replaced with monitoring of Sidekiq::DeadSet.new.size and queue latency (Sidekiq::Queue.new("default").latency).
  • Jobs that are not idempotent and relied on max_attempts: 1 in delayed_job need sidekiq_options retry: 0 explicitly, otherwise Sidekiq will happily re-run a failed charge 25 times.

If the old code depended on delayed_job’s implicit table-level uniqueness checks, add sidekiq-unique-jobs or an application-level guard; Sidekiq itself deduplicates nothing.

Phase 3: Next Steps & Risk Mitigation

Redis is an in-memory data store. If you do not configure Redis persistency (AOF or RDB backups), a server restart will wipe out all scheduled Sidekiq jobs. You must ensure your Redis infrastructure is hardened before running the migration script. For a subscription platform, losing the scheduled-jobs set means losing every future renewal reminder, so verify appendonly yes in redis.conf and test a restart on staging before cutover.

Post-migration, watch three metrics for the first week: queue latency per queue, the size of the retry set, and Redis memory usage (scheduled jobs live in a sorted set and large argument payloads add up; pass IDs, not serialized objects). Keep the empty delayed_jobs table around for a release or two: it is your rollback target if a hidden .delay call site surfaces in production.

Need Help Stabilizing Your Legacy App? Background job migrations carry a high risk of dropping critical business events (like billing emails). Our team at USEO executes zero-downtime infrastructure migrations.

Contact us for a Technical Debt Audit