Years ago, I had the opportunity to work on a liveblogging feature for a large media organization. They were covering a major tech event and were supposed to be the only way anyone could experience the event live outside of being physically on site. It was a great project with exciting challenges.
And I made a huge mistake.
Accountability
I have a set of default questions I ask in every job interview:
- What is it you hope to learn in this role?
- What interested you in this role at this company?
- What’s the biggest mistake you’ve ever made and what did you learn from it?
Rarely, I encounter someone who can’t answer the third question. Everyone makes mistakes. It’s what makes us human. Mistakes are how we learn and grow. But in order to learn from a mistake, you have to tack accountability for it.
Those who refuse to acknowledge or own up to their own mistakes are, at best, poor performers. At worst, they’re actively toxic and will eventually push the responsibility for cleaning up their mess onto others.
How someone answers this question in an interview usually tells me whether or not they’re going to be a fit before I get to any other questions.
The mistake
While the mistake I made with the liveblogging project was major, it is far from the only mistake I’ve ever made. It is, however, a useful story to tell among engineers. In many situations, it helps dwarf more minor mistakes made by newer teammates so they feel more comfortable acknowledging their own missteps.
My mistake was a failure to check for and protect against edge cases in a script.
The website used polling to check for news updates from the backend. If there was an update, it would render to the screen then wait a set interval before checking again. If there was no update, it would immediately return to a waiting mode, but with an exponential backoff (similar to a circuit breaker) to avoid hammering an idle backend.
Unfortunately, a typo in my backoff code led to the script attempting to add an integer and a NaN. While NaN in javascript is “numeric,” it is explicitly not a number and the addition was always resulting in a timeout period of NaN.
If you ask a million computers to wait NaN milliseconds before polling a server, the result is a self-induced DDoS attack. Of a major media website. During a highly popular live event.
Thankfully, the lead engineer in one of the three datacenters I inadvertently took down caught the issue, reverted the commit, and called to yell at me for it. A few embarassing conversations later and I lost my push-directly-to-production privileges, fixed the problem, and installed a new policy where everyone needed a round of code review regardless of their seniority.
No one is perfect
I make mistakes. You make mistakes. We all make mistakes. But the biggest mistake ever is refusing to practice humility and failing to recognize your own fallibility.
Not only will that mistake limit your own potential for growth, it will also set those around you up for failure. Newer engineers will likely make more mistakes than you do, but any arrogance on your part will make their missteps more paralyzing for personal growth.
