Accelerate your software engineering career by fixing something every week.

We, software engineers, are so used to living in pain that we stopped noticing it. We die a death by a thousand papercuts every day only to start fresh the next day. Flaky tests, broken builds, workarounds, outdated documentation is our daily bread. We live with all of that forgetting how much it costs.

Imagine an annoying issue you need to spend a few minutes on every week. Now, multiply this time by the number of people on your team – likely, they all are facing this issue. I also bet this is not the only “small” issue you and your team have. If you add all this up it may turn out that your team could use another engineer whose full-time job is just to deal with all these annoyances.

With time, it only gets worse because small issues tend to grow. One flaky test leads to a flaky suite and soon no one can tell what the quality of the product is. A convoluted function is patched multiple times without anyone really understanding it to the point when everyone prays that they don’t have to touch it. Missing test coverage results in multiple bugs reported by users.

The interesting thing is – most small issues don’t take a lot of time to fix. You yourself (i.e., not counting your team) may be able to recoup the time invested in a couple of weeks.

There are many examples of the positive impact of taking care of a small issue. Here is just a handful examples from my experience:

  • Fixing an acknowledged bug deprioritized consistently from release to release made lives of some users better (and discussing the bug repeatedly took more time than just fixing it)
  • Refactoring code that was incomprehensible made adding a new feature easier and faster
  • Updating an internal wiki helped save time spent on answering the same question again and again
  • A “flaky” test was a result of a product bug that could have serious consequences
  • Tuning a noisy alert prevented from waking up a team member who was on-call in the middle of the night

But wait, there is more! If you keep doing this regularly (a weekly cadence worked best for me) people will notice. Your team members may even start doing the same! And if they don’t, it will at least make it harder for them to make things worse – for example, it is much easier to ignore a new failing test if tens of tests are already failing than to ignore the first failing test in a suite. In any case, you are now helping to strengthen the engineering culture of your team leading by example.

And what if no one notices? Oh well, you made at least the life better for yourself – and this is already a win!

Accelerate your software engineering career by writing clean diffs

As a software engineer writing diffs (also called PRs – Pull Requests, or CRs – Code Reviews) is your bread and butter. You want to code your changes, have them reviewed, merge them, and start the process again. Repeating these cycles effectively is essential for delivering new features and building products quickly. There are not many things that can slow this process down like low-quality diffs. Some signs of a low-quality diffs are:

  • Compilation errors
  • Tests failures
  • Lint warnings

Low quality diffs leave both the author and the reviewer(s) frustrated. The author is frustrated because there is a lot of back and forth and they fell like they will never be able to merge their changes. The reviewers are frustrated because they feel their time is being wasted on diffs that are clearly not ready for review.

Sending a low-quality diff happens occasionally to everyone (e.g., this new file you forgot to include in your diff or that test you didn’t run due to a typo). However, when done repeatedly, it will have a detrimental effect on your career because your team members will simply try to avoid reviewing your code. This will prevent you from iterating quickly and will make it harder for you to adhere to schedules.

While spending a few minutes to double check your diff is in good shape may seem like a waste of time – especially if timelines are tight, it is an investment that in a long run will save you even more time. And once you build this habit it will become your second nature.

std::optional? Proceed with caution!

The std::optional type is a great addition to the standard C++ library in C++ 17. It allows to end the practice of using some special (sentinel) value, e.g., -1, to indicate that an operation didn’t produce a meaningful result. There is one caveat though – std::optional<bool> may in some situation behave counterintuitively and can lead to subtle bugs.

Let’s take a look at the following code:

  bool isMorning = false;
  if (isMorning) {
    std::cout << "Good Morning!" << std::endl;
  } else {
    std::cout << "Good Afternoon" << std::endl;
  }

Running this code prints:

Good Afternoon!

This is not a surprise. But let’s see what happens if we change the bool type to std::optional<bool> like this:

  std::optional<bool> isMorning = false;
  if (isMorning) {
    std::cout << "Good Morning!" << std::endl;
  } else {
    std::cout << "Good Afternoon!" << std::endl;
  }

This time the output is:

Good Morning!

Whoa? Why? What’s going on here?

While this is likely not what we expected, it’s not a bug. The std::optional type defines an explicit conversion to bool that returns true if the object contains a value and false if it doesn’t (exactly the same as the has_value() method). In some contexts – most notably the if, while, for expressions, logical operators, the conditional (ternary) operator (a complete list can be found in the Contextual conversions section on cppreference) – C++ is allowed to use it to perform an implicit cast. In our case it led to a behavior that at the first sight seems incorrect. Thinking about this a bit more, the seemingly intuitive behavior should not be expected. An std::optional<bool> variable can have one of three possible values:

  • true
  • false
  • std::nullopt (i.e., not set)

and there is no interpretation under which the behavior of expressions like if (std::nullopt) is universally meaningful. Having said that, I have seen multiple engineers (myself included) fall into this trap.

The problem is that spotting the bug can hard as there are no compiler warnings or any other indications of the issue. This is especially problematic when changing an existing variable from bool to std::optional<bool> in large codebases because it is easy to miss some usages and introduce regressions.

The problem can also sneak easily to your tests. Here is an example of a test that happily passes but only due to a bug:

TEST(stdOptionalBoolTest, IncorrectTest) {
  ASSERT_TRUE(std::optional<bool>{false});
}

How to deal with std::optional<bool>?

Before we discuss the ways to handle std::optional<bool> type in code, it could be useful to mention a few strategies that can prevent bugs caused by std::optional<bool>:

  • raise awareness of the unintuitive behavior of std::optional<bool> in some contexts
  • when a new std::optional<bool> variable or function is introduced make sure all places where it is used are reviewed and amended if needed
  • have a good test unit coverage that can detect bugs caused by introducing std::optional<bool> to your codebase
  • if feasible, create a lint rule that flags suspicious usages of std::optional<bool>

In terms of code there are few ways to handle the std::optional<bool> type:

Compare the optional value explicitly using the == operator

If your scenario allows treating std::nullopt as true or false you can use the == operator like this:

std::optional<bool> isMorning = std::nullopt;
if (isMorning == false) {
  std::cout << "It's not morning anymore..." << std::endl;
} else {
  std::cout << "Good Morning!" << std::endl;
}

This works because the std::nullopt value is never equal to an initialized variable of the corresponding optional type. One big disadvantage of this approach is that someone will inevitably want to simplify the expression by removing the unnecessary == false and, as a result, introducing a regression.

Unwrap the optional value with the value() method

If you know that the value in the given codepath is always set you can unwrap the value by calling the value() method like in the example below:

    std::optional<bool> isMorning = false;
    if (isMorning.value()) {
      std::cout << "Good Morning!" << std::endl;
    } else {
      std::cout << "Good Afternoon!" << std::endl;
    }

Note that it won’t work if the value might not be set – invoking the .value() method if the value was not set will throw the std::bad_optional_access exception

Dereference the optional value with the * operator

This is very similar to the previous option. If you know that the value on the given code path is always set you can use the * operator to dereference it like this:

std::optional<bool> isMorning = false;
if (*isMorning) {
  std::cout << "Good Morning!" << std::endl;
} else {
  std::cout << "Good Afternoon!" << std::endl;
}

One big difference from using the value() method is that the behavior is undefined if you dereference an optional whose value was not set. Personally, I never go with this solution.

Use value_or() to provide the default value for cases when the value is not set

std::optional has the value_or() method that allows providing the default value that will be returned if the value is not set. Here is an example:

std::optional<bool> isMorning = std::nullopt;
if (!isMorning.value_or(false)) {
  std::cout << "It's not morning anymore..." << std::endl;
} else {
  std::cout << "Good Morning!" << std::endl;
}

If your scenario allows treating std::nullopt as true or false using value_or() could be a good choice.

Handle std::nullopt explicitly

There must have been a specific reason you decided to use std::optional<bool> – you wanted to enable the scenario where the value is not set. Now you need to handle this case. Here is how:

    std::optional<bool> isMorning = std::nullopt;    
    if (isMorning.has_value()) {
      if (isMorning.value()) {
        std::cout << "Good Morning!" << std::endl;
      } else {
        std::cout << "Good Afternoon!" << std::endl;
      }
    } else {
      std::cout << "I am lost in time..." << std::endl;
    }

Fixing tests

If your tests use ASSERT_TRUE or ASSERT_FALSE assertions with std::optional<bool> variables they might be passing even if they shouldn’t as they suffer from the very same issue as your code. As an example, the following assertion will happily pass:

ASSERT_TRUE(std::optional{false});

You can fix this by using ASSERT_EQ to explicitly compare with the expected value or by using some of the techniques discussed above. Here are a couple of examples:

ASSERT_EQ(std::optional{false}, true);
ASSERT_TRUE(std::optional{false}.value());

Other std::optional type parameters

We spent a lot of time discussing the std::optional<bool> case. How about other types? Do they suffer from the same issue? The std::optional type is a template so its behavior is the same for any type parameter. Here is an example with std::optional<int>:

    std::optional<int> n = 0;
    if (n) {
      std::cout << "n is not 0" << std::endl;
    }

which generates the following output:

n is not 0

The problem with std::optional<bool> is just more pronounced due to the typical usages of bool. For non-bool types it is fortunately no longer a common practice to rely on the implicit cast to bool. These days it is much common to write the condition above explicitly as: if (n != 0) which will compare with the value of as long as it is populated.

Enums and Exhaustive switch statements in C++

Exhaustive switch statements are switch statements that do not have the default case because all possible values of the type in question have been covered by one of the switch cases. Exhaustive switch statements are a perfect match when working with scoped enum types. This can be illustrated by the following code:

#include <iostream>
#include <string>
enum class Color {
Red,
Green,
Blue,
};
std::string getColorName(Color c) {
switch(c) {
case Color::Red: return "red";
case Color::Green: return "green";
case Color::Blue: return "blue";
}
}
int main() {
std::cout << "Color: " << getColorName(Color::Green) << std::endl;
return 0;
}
view raw example.cpp hosted with ❤ by GitHub

Now, if a new enum member is added:

enum class Color {
Red,
Green,
Blue,
Yellow,
};

the compiler will show warnings pointing to all the places that need to be fixed:

enum.cpp:12:12: warning: enumeration value 'Yellow' not handled in switch [-Wswitch]
switch(c) {
^
enum.cpp:12:12: note: add missing switch cases
switch(c) {
^
enum.cpp:17:1: warning: non-void function does not return a value in all control paths [-Wreturn-type]

If you configure warnings as errors, which you should do if you can, you will have to address all these errors to successfully compile your program.

Without the exhaustive switch, the compiler would happily compile the program after adding the new enum member because it would be handled by the default case. Even if the default case was coded to throw an exception to help detect unhandled enum members, this exception would only be thrown at runtime which could be too late to prevent failures. In a bigger system or application there could be many switch statements like this and without the help from the compiler it can be hard to find and test all of them. To sum up – exhaustive switch statements help quickly find usages that need to be looked at and fixed before the code can ship.

So far so good, but there is a problem – C++ allows this:

Color color = static_cast<Color>(42);
view raw cast.cpp hosted with ❤ by GitHub

Believe it or not, this is valid C++ as long as the value being cast is within the range of the underlying enum type. If you flow an enum value created this way through the exhaustive switch it won’t match any of the switch cases and since there is no default case the behavior is undefined.

The right thing to do is to always use enums instead of mixing integers and enums which ultimately is the reason to cast. Unfortunately, this isn’t always possible. If you receive values from users or external systems, they would typically be integer numbers that you may need to convert to an enum in your system or library and the way to do this in C++ is static casting.

Because you can never trust values received from external systems, you need to convert them in a safe way. This is where the exhaustive switch statement can be extremely useful as it allows to write a conversion function like this:

Color convertToColor(int c) {
auto color = static_cast<Color>(c);
switch(color) {
case Color::Red:
case Color::Green:
case Color::Blue:
return color;
}
throw std::runtime_error("Invalid color");
}
view raw convert.cpp hosted with ❤ by GitHub

If a new enum member is added, the compiler will fail to compile the convertToColor function (or at least will show warnings), so you know you need to update it. For enum values outside of the defined set of members the convertToColor throws an exception. If you use a safe conversion like this immediately after receiving the value, you will prevent unexpected values from entering your system. You will also have a single place where you can detect invalid values and reject and log incorrect invocations.

Running Puppeteer in a Docker container on Raspberry Pi

Puppeteer is a Node.js module that allows interacting with a (headless) web browser programmatically. This is extremely useful for automating website testing, generating screenshots and PDFs of web pages or programmatic form submission.

Docker offers numerous benefits including a standardized environment, isolation, and rapid deployment to name a few. These benefits might be why you’d want to run Puppeteer inside a Docker container. Unfortunately, doing so on Raspberry Pi is not straightforward. There are a few issues that make it harder than usual. Luckily, they are all solvable. Let’s take a look.

Problem 1: Chromium included in Puppeteer does not work on Raspberry Pi

Puppeteer by default downloads a matching version of Chromium which is guaranteed to work out of the box on supported platforms. Unfortunately, Chromium does not currently provide an arm build that works on Raspberry Pi and running stock Puppeteer on Raspberry Pi will end up with a crash. This can be solved by installing Chromium with apt-get install chromium -y and telling Puppeteer to use it by passing  the executablePath: '/usr/bin/chromium' to the launch() function as follows:

   const browser = await puppeteer.launch({
        executablePath: '/usr/bin/chromium',
        args: []
    });

When doing this, Puppeteer no longer should need to download Chromium as it will be using the version installed with apt-get, so it makes sense to skipping this step by setting the corresponding environment variable in the Dockerfile:

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true

This will significantly reduce the time needed to install node modules.

Problem 2: The base image installs an old version of Chromium

Some node Docker images are based on old distributions that contain only older versions of Chromium. Most notably the node:16 image is based on buster. If you use this base image the only version of Chromium you will be able to install with apt-get is 90.0.4430.212-1. Unfortunately this version doesn’t work in a Docker container – it just hangs indefinitely. Moving to the node:16-bullseye base image allows installing a much newer version of Chromium (108.0.5359.124) where this is no longer a problem.

Problem 3: Puppeteer crashes on launch

Puppeteer will not launch in a Docker container without additional configuration. Chromium is not able to provide sandboxing when running inside a container so it needs to be launched at least with the --no-sandbox argument. Otherwise it will crash with the following error message

Failed to move to new namespace: PID namespaces supported, Network namespace 
 supported, but failed: errno = Operation not permitted

Sandbox is a security feature and running without a sandbox is generally discouraged. Unfortunately, running without a sandbox appears to be currently the only way to run Puppeteer inside a Docker container. In the past the --no-sandbox option required running Puppeteer as root, only increasing the risk. Luckily, this no longer seems to be the case – it is possible now to launch puppeteer with the --no-sandbox option as a non-privileged user.

There are a few more options that might be worth exploring if launching Puppeteer inside a container fails:

  • --disable-gpu – disables GPU hardware acceleration (which is usually not available when running in Docker)
  • --disable-dev-shm-usage – prevents from using shared RAM (/dev/shm/)
  • --disable-setuid-sandbox – disabled setui sandbox

Putting everything together

The information provided above should be all that is needed to be build a Docker image for a Node.js app that uses Puppeteer and runs on Raspberry Pi. Below is an example Dockerfile for such a Docker image. It contains comments to make it easy to notice how the solutions discussed above were applied.

# Ensure an up-to-date version of Chromium 
# can be installed (solves Problem 2)
FROM node:16-bullseye 
# Install a working version of Chromium (solves Problem 1)
RUN apt-get update
RUN apt-get install chromium -y
ENV HOME=/home/app-user
RUN useradd -m -d $HOME -s /bin/bash app-user 
RUN mkdir -p $HOME/app 
WORKDIR $HOME/app
COPY package*.json ./
COPY index.js ./
RUN chown -R app-user:app-user $HOME
# Run the container as a non-privileged user (discussed in Problem 3)
USER app-user
# Make `npm install` faster by skipping 
# downloading default Chromium (discussed in Problem 1)
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
RUN npm install
CMD [ "node", "index.js" ]

Because the application also requires a couple modifications to how the headless browser is launched here is a small example application illustrating these changes with comments:

const puppeteer = require('puppeteer');
(async() => {
    const browser = await puppeteer.launch({
        // use Chromium installed with `apt` (solves Problem 1)
        executablePath: '/usr/bin/chromium',
        args: [
            // run without sandbox (solves Problem 3)
            '--no-sandbox',
            // other launch flags (discussed in Problem 3)
            // '--disable-gpu,
            // '--disable-dev-shm-usage',
            // '--disable-setuid-sandbox',
        ]
    });
    const page = await browser.newPage();
    await page.goto('https://www.google.com/', {waitUntil: 'networkidle2'});
    let e = await page.$('div#hplogo');
    let p = await e?.getProperty('title');
    if (p) {
      console.log(`Today's doodle: ${await p.jsonValue()}`);
    } else {
      console.log('No Doodle today :(');
    }
    browser.close();
})();

Finally, here is the output of this application when run in a container:

Both the application and the Dockerfile are also available on Github

Conclusion

Running Puppeteer inside a Docker container is tricky – especially, when doing so on Raspberry Pi. The post discussed the key obstacles and provided solutions to overcome them. In addition, a demo containerized app was included to illustrate the main points.