Optimization

Deeply Nested has enjoyed an influx of new subscribers since hitting the top of Hacker News last week. Welcome!

This is a young publication, and we can collectively decide what it should be when it grows up. Will we focus on tech? Career growth? Project management? Should we dive into code, or see no code at all? Please use the comments section to make your voice heard.


Many moons ago, I told a classmate I wanted to be a scientist, and they told me they wanted to be a philosopher. When I asked them why, they said scientists get to answer questions, but philosophers get to question answers.

Engineers get to solve problems. Part mathematician, part technician, an engineer bridges the chasm between theory and practice. Good engineering means compromise, and the most crystalline form of compromise may be optimization.

Optimization is the art of making existing systems run faster, use less memory, have longer battery life, or generally use resources more efficiently. It usually entails sacrifice, like consuming more of a plentiful resource to preserve a scarce one. Any time we optimize a system, we should be clear not only on what we’re gaining, but what we’re giving up. Here are some guidelines for software engineers focused on optimization.

  • Know what you’re optimizing. For each broad category of input you care about (big, small, common, pathological...), define a benchmark that includes sample input and a script to run your code. (If you’re optimizing a microservice, the input may be a series of replayed requests.) The script output should include a high-level summary of resource consumption. Keep an eye on these high-level numbers over time, to see whether things are getting better or worse.

    By the way, if you personally contribute to an improvement in your product’s resource requirements, cite those data at your salary review: “Reduced median page load time by 300 ms,” or “Extended battery life by 10%.” It feels good to bring data to an opinion fight.

  • Recognize trade-offs. As one aspect of a system is optimized, other aspects are often inadvertently pessimized, much as squeezing a stress ball makes it bulge out in other places. If you’re optimizing run time, benchmark memory footprint to make sure it’s not bulging.

  • Profile first. While benchmarking gives you an overview of resource consumption, profiling tells you what parts of the code specifically are consuming those resources. Identify bottlenecks, and don’t muck up non-bottleneck code in the name of performance. As Don Knuth tells us: Premature optimization is the root of all evil.

    The counterintuitive nature of bottlenecks is illustrated by the classic Car Talk puzzler SUV or Hybrid? A Familial, MPG Conundrum. Enjoy.

  • Prefer simple tools and techniques to powerful ones. Avoid profilers with a high learning curve, limited license availability, or other barriers to use by other members of your team. For example, if a program is written in Go, consider relying entirely on Go’s built-in pprof support. If the program is in Rust, limit yourself to cargo bench as much as you can.

    Help other people make sense of the data. Empowering your teammates can give your work multiplicative, rather than merely additive, benefit. Even if you work alone, keeping things simple reduces your cognitive load; and that brain of yours is the most precious resource of all.

If you have a background in computer science, you might wonder why we haven’t talked about reducing the Big O complexity of our algorithms. Computational complexity is a sticky subject that we’ll consider in an upcoming post.


If you're new to Deeply Nested, be advised that you have access to all existing content.