Generating Ideas and Driving them to Completion

It is impossible to achieve a successful and fulfilling career in Software Engineering only by following someone else’s orders. A significant part of taking the lead is coming up with ideas to innovate and move the team or the company forward.

The two models of idea generation

Over the years, I’ve witnessed companies using two main ways to generate ideas: on-demand and organic.

On-demand idea generation

The on-demand idea generation model works as follows: the manager shows up out of nowhere and demands, “We need some ideas for X!” They organize a brainstorming meeting where the team members try to invent some ideas. After the meeting, everyone returns to their work, feeling they have fulfilled their idea generation duty until the next time.

My experience with on-demand idea generation has been mixed. This setting usually seeks big ideas, which are generally quite hard to come up with on the spot and under pressure. In the end, only a few ideas are proposed, and barely any are implemented.

The most important reason why on-demand idea generation is ineffective is that most interesting ideas strike at unexpected times and not during a scheduled meeting.

Organic idea generation

Organic idea generation is the opposite of the on-demand model. Here, the ideas stem from observations made when working on daily tasks:

writing or reviewing code
investigating issues reported by users
struggling with tools or infrastructure
mitigating incidents
discussing issues with co-workers

All these activities are great opportunities to identify problems and propose improvements.

One of the biggest advantages of organic idea generation is that it is a continuous process. As a result, it allows for the generation of many ideas.

Most ideas generated organically are small: refactor some code, add test coverage, or fix a non-critical but annoying bug. Some are medium, e.g., redesigning a component for better extensibility. Occasionally, you will stumble upon a big idea that may lead to revamping your entire architecture and unlocking previously unthinkable possibilities.

Executing ideas

Even the best idea is not worth much if not acted upon. However, careless execution may have negative consequences. For example, failing to deliver a promised feature on time due to working on unplanned and non-critical refactoring is hard to justify. Here is my approach to avoiding these problems.

I start by noting the idea in my work log. This way, I rest assured that I won’t forget about it and will consider it when planning my work for the next week. Implementing small ideas is usually a matter of finding time to work on it. If I don’t have the bandwidth, I may ask a fellow developer working on related code to pick it up or use it to ramp up a new team member.

Medium and big ideas require more thinking. When planning my week, I block a couple of hours to write a one-pager describing the idea in more detail. Writing allows me to get more clarity on my idea, understand its feasibility, and weigh the costs and benefits.

Most ideas never reach the execution stage. Some are just not great, and external circumstances may block others. Over time, when these circumstances change, an infeasible idea may become viable. I recently revived an idea I had a year ago when I learned that our partner team had fixed a long-standing issue in their system.

I share ideas I believe are worth pursuing with my team and my manager to gather feedback. Depending on this feedback, I either shelve the idea or continue working on it until completed, often with other teammates.

But there is one more step after successful implementation: spreading the word about what we did, how we did it, and who made it possible. This step is especially important for bigger projects that take a while to implement and involve other team members. Everyone who contributed deserves to get the credit.

Conclusion

The most successful software developers generate many ideas because they understand that only some will come to fruition. But ideas are only the first step. The key is execution. Successfully executing an idea, letting the right people know, and sharing the credit is a huge career booster.

Use Test Plans to become a more effective Software Developer

Shipping high-quality software is the responsibility of each software developer. Not only are the days when handing untested code to the QA team for validation a common practice long gone, but many companies have also moved away from QA-based testing, making developers fully own the quality of the product. This creates a problem: how can you ensure developers do their due diligence and validate their changes? At Meta (a.k.a. Facebook), we use “test plans.”

What are test plans?

Test plans describe how authors tested their changes. They are an integral part of Meta’s code review process: the code review tool does not allow submitting code for review if the test plan is empty.

While it is common for test plans to say: “unit tests,” many are much more interesting and often include:

screenshots showing the UI before and after the change
videos showing the change in action
API requests and corresponding responses (e.g., JSON payloads)
dashboard snapshots from canary runs
terminal printouts
Funny memes – especially if the testing was not comprehensive or applicable (e.g., auto-generated code, changes to unit tests only, etc.)

Benefits of requiring Test Plans in commits

The most obvious benefit of requiring test plans is forcing developers to think about validating their changes. While not all test plans are comprehensive, most are decent.

Occasionally, test plans reveal that the code change does not work as the author intended. I once proudly included a graph hovering a little below 100% as my test plan, only for a reviewer to point out that my graph represented not the success rate, as I claimed, but the error rate.

However, one often overlooked benefit of requiring test plans is that they can be lifesavers when working with unfamiliar code.

Imagine you are tasked with building a new feature in a mobile app. While working on the feature, you discover that the API powering your app doesn’t return all the necessary information. You can ask the API team to add it, but they may not be able to accommodate your request on short notice. Perhaps it would be faster if you implemented this change yourself. The only problem is that you are not familiar with the backend code. You don’t know how to test it to ensure you didn’t break anything. In these situations, test plans can come in extremely handy. You can check the commit history of the code you want to change and see how developers who regularly contribute to it test it. In addition, checking past test plans may help you discover edge cases you need to consider. I successfully used this strategy multiple times to change an unfamiliar codebase I would otherwise be afraid to touch.

“My unit test coverage is 100%”

Test plans should not be considered a replacement for unit tests. As great as unit testing is, it is not always sufficient. Additional end-to-end validation helps confirm that the changes worked as intended outside of the isolation provided by unit tests and that they didn’t introduce unwanted behavior. I have seen (and caused) situations where my application wouldn’t start even though all unit tests were passing.

Call To Action

Including test plans in pull requests is not a common practice, let alone a requirement, in most companies or teams. Despite that, I encourage you to follow this practice. Your team members will notice it and may start doing the same if they find it useful. And even if they don’t, you will still benefit from this habit. Going the extra mile can help you find issues before they impact users. With time, it will get easier because you can reuse past test plans to validate some of your current changes. You may need to tweak them a little, but you won’t have to start from scratch each time.

Code is tax – have a good reason to write it

I once told software engineers on my team: “We’re not here to write code”. They were flabbergasted. But I strongly believe that our job is not to write code but to solve problems. It just so happens that for many problems, code is the best, if not the only solution.

We often get this backwards. We identify as software developers, we love writing code so we will write code whether it is needed or not.

Years of experience taught me to think about a problem before writing a single line of code. I usually try to answer two questions:

does this problem need to be solved now or, even at all?
can this problem be solved without writing code?

Surprisingly often, this exercise allows me to discover alternative ways of solving a problem that are more effective than writing code.

Not all problems need to be solved

Sometimes, assessing the importance of a problem is difficult, especially when done in isolation. The issue might seem significant, prompting us to solve it with code, only to realize after the fact (or maybe even not) that this work didn’t yield any noticeable improvement.

Consider a service making blocking I/O calls. Such a service won’t scale well. However, if it receives only a few requests per minute rewriting it to improve its throughput won’t have significant impact and is not necessary.

While many such problems will never require further attention, some might need to be revisited. If the service mentioned above needs to be integrated with a system generating many requests per second, fixing the blocking I/O calls could be a top priority. So, it is important to stay on guard and address issues that were initially punted if they resurface again.

Not all problems need to be solved with code

Many problems can be solved without writing any code.

There is an entire class of problems that can be mitigated by changing the configuration of the software. For example, if your streaming service occasionally crashes due to the Out-Of-Memory exception when processing spikes of data, reducing the maximum allowed batch the service fetches for processing could be a perfectly valid solution. Processing the spikes may take a little longer, but the overall processing time may not be affected in a noticeable way and the issue is fixed without changing the code.

Often, you can solve problems making careful design choices. Take user accounts on a website, for example. When deciding how users log in, there are two common options: using their email or letting them create a unique username. Coming up with unique user names can be challenging for applications with millions of users, so many of them suggest an available user name. Using an email as the user name is an elegant way of solving the problem of unique user names without writing any code. It also simplifies the account verification and password recovery flows.

What’s the problem with writing code anyway?

In my perspective, code should be treated like a tax. Just as taxes, once code is added, it is rarely removed. Every person on the team pays this tax day in and day out because each line of code needs maintenance and is a potential source of alerts, bugs or performance issues. This is why I believe that, as software engineers, we should opt for no-code solutions when possible, and write code only when absolutely necessary.