Infrastructure Medium severity

Redis Connection Leaks

A Redis connection pool sized smaller than the Sidekiq concurrency or number of Puma threads, producing Redis::TimeoutError: Waited 5 seconds errors and queue instability because workers block waiting for a connection that the pool cannot supply.

Before / After

Problematic Pattern

# config/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = {
  url: ENV['REDIS_URL'],
  size: 5 # too small
}
# but concurrency is 25
end

# config/initializers/redis.rb
# Global Redis client, no pool
$redis = Redis.new(url: ENV['REDIS_URL'])
# Used from Puma threads and Sidekiq workers.
# Threads block on each other.

Target Architecture

# config/sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = {
  url: ENV['REDIS_URL'],
  size: Sidekiq.options[:concurrency] + 5,
  network_timeout: 5
}
end

Sidekiq.configure_client do |config|
config.redis = {
  url: ENV['REDIS_URL'],
  size: ENV.fetch('RAILS_MAX_THREADS', 5).to_i
}
end

# config/initializers/redis.rb
require 'connection_pool'

REDIS_POOL = ConnectionPool.new(size: 10, timeout: 5) do
Redis.new(url: ENV['REDIS_URL'])
end

# Usage
REDIS_POOL.with { |r| r.set('k', 'v') }

Why this hurts

Sidekiq runs jobs on a thread pool sized by the concurrency setting. Each thread needs a Redis connection for the duration of the job to push heartbeats, update state, and claim the next job from the queue. If the Redis connection pool is smaller than the concurrency, threads block on pool checkout before they can do any real work. A concurrency of 25 with a pool size of 5 means that at any given moment only 5 of the 25 threads are making progress; the other 20 wait on ConnectionPool#checkout, which defeats the whole purpose of concurrency.

The failure mode stays hidden because it looks like general slowness rather than a specific configuration error. Sidekiq’s web UI shows healthy queue depth but steadily increasing latency. Redis::TimeoutError raises after the pool’s timeout expires, which generic rescue => e blocks catch and report as “flaky Redis”. Operators add retries, extend timeouts, and scale horizontally, none of which address the root cause.

Process-wide Redis clients (a single $redis = Redis.new) used across multiple threads produce silent corruption under load. Redis’s wire protocol is not safe for concurrent use on one connection: two threads writing commands simultaneously interleave bytes on the wire, and the server returns responses that cannot be matched to the correct caller. The redis-rb gem mitigates this with a thread-level lock, but the lock becomes the bottleneck and all concurrency collapses to serial access.

A Rails application typically uses Redis for three purposes: Sidekiq internal queues, Rails cache (Rails.cache), and application-specific usage (rate limiting, feature flags, cached API responses). Each subsystem maintains its own pool, and misconfiguring one starves the others because they all connect to the same Redis instance. The aggregate concurrent load on Redis is the sum of every pool configured across the process, not just the largest one.

Get Expert Help

Inheriting a legacy Rails codebase with this problem? Request a Technical Debt Audit.