Solid Queue 1.6.0: Fiber Workers Bring Lighter Concurrency to Rails Background Jobs

Solid Queue, a database-backed Active Job backend for Rails, has released version 1.6.0. The headline feature: fiber workers. For teams running background jobs on Rails, this update promises lower memory usage per task and more straightforward concurrency tuning. This article breaks down what that means in practice.

What Are Fiber Workers?

To understand the update, it helps to recall how concurrent jobs are usually executed. Most background job libraries rely on threads or processes. Threads are managed by the operating system and carry a relatively heavy stack — typically around 1–8 MB per thread, depending on configuration. That overhead adds up: a process with 20 threads may consume an extra 20–160 MB of memory just for thread stacks.

Fibers, in contrast, are cooperative lightweight concurrency primitives. They are scheduled in user space, not by the OS. A fiber can switch to another fiber at any point, but only when explicitly told to do so. Ruby's core libraries support fibers, and the fiber gem provides additional tools. Because fibers use a smaller stack (often a few kilobytes initially) and share the same thread, they are substantially cheaper to spawn.

Solid Queue 1.6.0 introduces the ability to run job workers on fibers instead of threads. The release notes describe this as a way to increase concurrency without proportionally increasing memory use.

Why Does This Matter for Rails Apps?

In a typical Rails deployment, memory is often the bottleneck when scaling background job processing. For instance, a Sidekiq or Solid Queue process might be configured to run 10–25 threads. If each thread consumes several megabytes of stack space, the process memory footprint grows quickly. Heroku, Fly.io, and other PaaS providers charge for memory, so reducing overhead directly reduces infrastructure costs.

Fiber workers let you keep concurrency high while keeping memory low. This is especially useful for jobs that spend most of their time waiting — for example, making external API calls, reading from a database, or waiting on a socket. In such cases, a fiber can pause and yield control to another fiber, so a single thread can handle hundreds of lightweight tasks.

Threads vs. Fibers: A Quick Comparison

This table is a general comparison, not something directly from the release notes. However, it helps set the context for why the Solid Queue maintainers chose to add fiber support.

What’s New in Solid Queue 1.6.0

The release notes for v1.6.0 highlight fiber workers as the main addition. Key points include:

• A new configuration option turns on fiber-based execution for Solid Queue workers.
• Existing thread-based workers remain fully supported, so this is an opt-in migration.
• The project integrated with the fiber gem to manage scheduling.

For users, the API stays the same. You configure concurrency via Solid Queue’s existing max_threads or a new equivalent that now applies to fibers. The upgrade path is straightforward: bump the gem version, add the appropriate setting, and deploy.

The maintainers also mention that fiber workers are particularly beneficial for workloads with high I/O wait time, such as delivering webhooks or processing uploads. For CPU-bound tasks, fibers will not provide speedups by themselves — you still need multiple threads or processes for true parallel execution.

A Practical Example: Webhook Delivery

Consider an application that sends webhooks to dozens of third-party services. Each job opens an HTTP connection and waits for a response. With thread-based workers, a pool of 10 threads can handle 10 slow webhooks concurrently; the rest of the jobs wait in the queue. Increasing the pool to 50 threads would allow more concurrent requests, but it would consume hundreds of megabytes of RAM.

With fiber workers, you can configure a concurrency level of 100 or more on a single thread. Each job uses a fiber, and when the HTTP request blocks, Ruby automatically switches to another fiber that is ready to run. The memory cost is far lower — the application can use the extra RAM for other purposes.

This pattern is described in the Solid Queue documentation and is the typical use case for fiber-based execution.

How to Get Started

If you are already using Solid Queue, upgrading to 1.6.0 is the first step. The release notes recommend setting fiber_workers: true in your Solid Queue configuration, or something equivalent — the exact syntax may change as the feature matures.

Before switching, run a quick experiment with your actual job mix. Measure RSS (resident set size) with thread-based workers and then with fiber workers at the same concurrency level. In many I/O-heavy workloads, you should see a noticeable memory drop with no loss in throughput.

Here is a sample configuration sketch:

# config/solid_queue.yml
production:
  workers:
    - queues: "*"
      threads: 1      # or whatever your thread count is
      fiber_workers: true
      fibers_per_thread: 100

This is not a verbatim example from the release notes, but it illustrates the direction. Always check the official documentation for the current syntax.

Tradeoffs and Caveats

Fibers are not a silver bullet. Because they are cooperative, a fiber that performs CPU-intensive work without yielding can block all other fibers on that thread. In practice, this means you should not use fiber workers for jobs that involve heavy processing, such as image resizing or complex encryption, unless you also maintain a pool of threads for those tasks.

Also, some libraries are not fiber-safe. They may use thread-local variables or make syscalls that block the entire thread. The Solid Queue maintainers note that fiber support requires Ruby 3.0+, and that you should test third-party libraries before deploying fiber workers in production.

Conclusion

Solid Queue 1.6.0’s fiber workers are a thoughtful improvement for I/O-bound background workloads. By enabling higher concurrency with lower memory usage, the feature fits well into modern Rails deployments where efficiency and cost control are priorities. The upgrade is non-disruptive, and the existing thread-based approach remains fully supported.

If you run a large number of background jobs that spend time waiting on external services, give fiber workers a try. Measure the memory and throughput differences in your own environment — that’s the only reliable way to decide if this new mode pay if for your use case.

For the full list of changes and exact configuration options, refer to the official release notes:
Source