Many moons ago, I told a classmate I wanted to be a scientist, and they told me they wanted to be a philosopher. When I asked them why, they said scientists get to answer questions, but philosophers get to question answers.
Engineers get to solve problems. Part mathematician, part technician, an engineer bridges the chasm between theory and practice. Good engineering means compromise, and the most crystalline form of compromise may be optimization.
Optimization is the art of making existing systems run faster, use less memory, have longer battery life, or generally use resources more efficiently. It usually entails sacrifice, like consuming more of a plentiful resource to preserve a scarce one. Any time we optimize a system, we should be clear not only on what we’re gaining, but what we’re giving up. Here are some guidelines for software engineers focused on optimization.
Know what you’re optimizing. For each broad category of input you care about (big, small, common, pathological...), define a benchmark that includes sample input and a script to run your code. (If you’re optimizing a microservice, the input may be a series of replayed requests.) The script output should include a high-level summary of resource consumption. Keep an eye on these high-level numbers over time, to see whether things are getting better or worse.
By the way, if you personally contribute to an improvement in your product’s resource requirements, cite those data at your salary review: “Reduced median page load time by 300 ms,” or “Extended battery life by 10%.” It feels good to bring data to an opinion fight.
Recognize trade-offs. As one aspect of a system is optimized, other aspects are often inadvertently pessimized, much as squeezing a stress ball makes it bulge out in other places. If you’re optimizing run time, benchmark memory footprint to make sure it’s not bulging.
Profile first. While benchmarking gives you an overview of resource consumption, profiling tells you what parts of the code specifically are consuming those resources. Identify bottlenecks, and don’t muck up non-bottleneck code in the name of performance. As Don Knuth tells us: Premature optimization is the root of all evil.
The counterintuitive nature of bottlenecks is illustrated by the classic Car Talk puzzler SUV or Hybrid? A Familial, MPG Conundrum. Enjoy.
Prefer simple tools and techniques to powerful ones. Avoid profilers with a high learning curve, limited license availability, or other barriers to use by other members of your team. For example, if a program is written in Go, consider relying entirely on Go’s built-in pprof support. If the program is in Rust, limit yourself to cargo bench as much as you can.
Help other people make sense of the data. Empowering your teammates can give your work multiplicative, rather than merely additive, benefit. Even if you work alone, keeping things simple reduces your cognitive load; and that brain of yours is the most precious resource of all.
If you have a background in computer science, you might wonder why we haven’t talked about reducing the Big O complexity of our algorithms. Computational complexity is a sticky subject that we’ll consider in an upcoming post.
Deeply Nested has enjoyed an influx of new subscribers since hitting the top of Hacker News last week. Welcome! If you're new here, be advised that you have access to all existing content.
Maybe it's because I'm not a software engineer, but I think the tone of these posts has been great. They are more why vs specifically how. I interview many engineers and I'm consistently surprised at how implementation focused they are and how poor they are at the why. I often ask the question "How do you design something?" and see it turn into a ramble of implementation details vs thinking about what problem they are trying to solve and why. Problem solving is the most undeveloped skill that I see as a whole. I find that people coming from non-engineering backgrounds will often have a better developed sense of this, but lack the experience to know strong implementations or design patterns, but they do a great job of understanding the problem.