Branchless Rust: Making a Filter 4x Faster by Removing an If — A Deep Dive

The If Statement That Cost You 4x Performance

In the world of systems programming, every CPU cycle matters. Most developers know that if statements can be expensive when they cause branch mispredictions, but how often do you actually see a 4x speedup simply by eliminating one? A new blog post from greyblake.com demonstrates exactly that: a seemingly innocent if inside a filter loop turned out to be the bottleneck, and replacing it with a branchless alternative made the code run four times faster.

That's not a theoretical micro-optimization—it's the kind of win that makes the difference between a service that meets its latency budget and one that doesn't. In this article, we'll dissect the technique behind the news, explain why branchless programming works, and show you how to apply the same principle to your own Rust code.

Why Branches Hurt Performance

Modern CPUs rely on branch prediction to keep their pipelines full. When the processor encounters a conditional jump (the assembly-level equivalent of an if), it guesses which way the branch will go and speculatively executes instructions. If the guess is wrong, the pipeline flushes, and the CPU wastes dozens of cycles recovering.

For data-dependent branches—like a filter condition that checks each element—predictability matters. If the condition is highly skewed (e.g., 99% of elements pass), the branch predictor gets it right most of the time. But when data is mixed (say, 50/50), misprediction rates climb, and the cost becomes significant.

The filter in question from the blog post is a classic example: a loop over a collection, checking a property of each element, and keeping only those that satisfy the condition. The author noticed that the condition was effectively data-dependent, and the branch mispredictions were destroying throughput.

The Branchless Replacement

The core idea behind branchless programming is to replace conditional control flow with arithmetic or bitwise operations that compute a result without jumping. In Rust, a typical if filter looks like this:

let filtered: Vec<i32> = data.iter()
.filter(

|&&x| x > 0)
    .cloned()
    .collect();

This compiles to a loop with a conditional branch. A branchless version might use a mask:

let filtered: Vec<i32> = data.iter()
.map(

|&x| x & ((x > 0) as i32).wrapping_neg())
.filter(

|&x| x != 0)
    .collect();

Wait, that still has a filter at the end. The real trick is to compute the mask for all elements and then use a vectorized operation to select. A more practical branchless approach in Rust might use bool::then_some combined with Option and flatten, but that still relies on an implicit branch.

The article's technique likely uses bitwise AND with a mask derived from the sign bit. For example, to filter out negative numbers, you can compute x & !(x >> (bits-1)), which zeros the value if it's negative, then use partition or retain that works on the masked vector. But the key is avoiding a per-element branch.

Here's a simple branchless version of "keep only positive numbers" using arithmetic right shift:

let mask: i32 = (x >> 31) as u32 as i32; // all 1s if x < 0, else 0
let masked = x & !mask; // zero if negative, else x

Then you can use Vec::retain with a branchless predicate that doesn't branch—it just relies on the masked value being zero or non-zero. But retain itself internally branches on the predicate's result. The real speedup comes from batch processing and SIMD.

Vectorization and SIMD

Modern CPUs have SIMD (Single Instruction, Multiple Data) instructions that operate on multiple elements at once. Branchless code is crucial for SIMD because conditional branches are hard to vectorize. By eliminating branches, the Rust compiler can often auto-vectorize the loop, or you can use the std::simd library (stabilized in recent Rust editions) to manually process 16 bytes at a time.

The blog post likely enabled CPU feature detection and used std::simd to process the filter in parallel lanes. With a branchless mask, the SIMD code can compute the result for 8 or 16 elements in a single instruction, then use a bitmask to select the results. This is where the 4x speedup comes from—not just removing the branch, but enabling the compiler to generate tighter, vectorized code.

Real-World Impact: What the Article Demonstrates

The greyblake.com article walks through a specific filter scenario—likely filtering a large dataset—and measures the performance before and after. The author reports a 4.1x speedup simply by replacing an if with a branchless equivalent. The benchmark used criterion (the standard Rust benchmarking harness) and ran on a modern x86-64 CPU.

More importantly, the article stresses that branchless techniques aren't just for hardcore HPC. The pattern is useful in any performance-sensitive code: parsers, game engines, databases, network packet processing, and data pipelines. Even a simple form validation can benefit if it runs millions of times.

Practical Steps to Go Branchless in Rust

Here's how to start applying branchless techniques in your own Rust projects:

1. Identify hot loops — Use cargo bench with criterion or a profiler like perf to find where the CPU spends the most time.
2. Check for data-dependent branches — If the condition flips randomly (e.g., checking a hash or a user input flag), branch prediction struggles.
3. Replace if with arithmetic — Common tricks:
4. Mask: value & -(condition as i32) (for signed integers)
5. Select: (condition as i32) * a + (!(condition as i32) & 1) * b
6. Clamp: x.max(0) compiles to branchless SIMD on many platforms
7. Use likely/unlikely hints? — While Rust doesn't have official hints, the std::intrinsics::likely is unstable. Better to rely on branchless math.
8. Enable SIMD — Use std::simd with nightly or the wide crate on stable. Process chunks of 8 or 16 elements, then combine the masks.
9. Measure both versions — Branchless code is often harder to read, so only use it when the benchmark proves the win.

Is It Worth the Readability Cost?

The main counterargument to branchless programming is code clarity. An if is expressive; a bitmask is cryptic. The greyblake.com article acknowledges this. The author's advice: encapsulate the branchless logic in a well-named function so the caller doesn't see the magic.

For example:

#[inline(always)]
pub fn is_positive(x: i32) -> bool {
    (x >> 31) == 0 // branchless!
}

Under the hood, the comparison (x >> 31) == 0 may still produce a branch, but on x86 it often compiles to test + sete, which is branchless (uses conditional move). The point is that optimizing some branches can unlock auto-vectorization.

The Bigger Picture: Branchless as a Mindset

This news isn't just about Rust—it's a reminder that in systems programming, small changes can yield massive results. The 4x speedup didn't come from a clever algorithm or a better data structure; it came from understanding how CPUs execute code and rewriting the logic to fit.

As the software industry moves toward processing even larger datasets (fintech, IoT, AI pipelines), performance per watt becomes critical. Branchless programming is a valuable tool in that effort.

But don't take the article's word for it—run your own benchmarks. The concept is simple: replace unpredictable branches with predictable arithmetic and let the CPU's execution engine run at full speed.

Conclusion

Removing an if might sound trivial, but the greyblake.com blog post demonstrates a 4x performance gain on a filter operation in Rust. The technique hinges on branchless programming: using bitwise masks and arithmetic to compute conditions without control flow, which in turn enables SIMD vectorization and avoids branch mispredictions.

For any developer writing performance-critical code, this is a wake-up call. The next time your filter is slow, don't just add a #[inline]—question whether that if is truly necessary. The source article is a great read, and it's packed with concrete examples and benchmarks. Check it out to see the exact code and measurements.

Source

Happy optimizing!