In this post, I want to talk about how to solve the vague, fuzzy problems that show up in real work.

Before we get into it, try recalling the one piece of work you put the most effort into last quarter. Maybe it was a refactor that took a few weeks, a new tool your team adopted, or an ambitious improvement project. Keep that work in mind and try answering this question.

What exactly was the problem that work was meant to solve, and what can you show to prove that the problem actually shrank?

Ask this question and the first answers are usually things like “The code was too complex, so I finished the refactor” or “I thought it would improve productivity.” Those are closer to the feeling of having solved something than to actually solving a real problem. You clearly worked hard, and something does feel better, but it’s hard to say plainly “this got better because of that.”

This isn’t only true of individual work. Look at an organization’s goal documents and you’ll find the to-do items packed in tight, while the parts about which problem those items are meant to solve, and how anyone would know it got solved, are often blank. I’ve asked similar questions many times while helping several organizations refine their goal documents, and looking back, those questions mostly boiled down to four.

The ability to define a problem properly is less a developer skill than a basic competency for anyone who is good at their job, whatever their role. Yet in my experience, surprisingly few people have really internalized it. Plenty of people work hard, but few can properly explain what problem their work is actually solving.

On top of that, few abilities pay off as broadly once you’ve learned them. Whether you’re fixing code, bringing in a new tool, or setting a team’s quarterly goals, the shape of the work differs, but the process of first defining what the problem is and what needs to get better applies in exactly the same way.

From here on, I’ll follow those four questions one by one to lay out what it means to define and solve a problem, and at the end I’ll think about how this ability can be developed. Most of the examples come from development organizations, since that’s the world I know best, but the process itself applies to any kind of work.

That said, not every piece of work needs all four questions. If you demand a problem definition before even small refactors like renaming a variable or extracting a function, the cost of defining the problem outgrows the cost of the problem itself. In his 2015 letter to shareholders, Jeff Bezos split decisions into ones that can’t be reversed and ones that can, and argued that reversible decisions should be made lightly and quickly. The process in this post is most valuable for work that takes weeks or more, is hard to undo, and spends other people’s time as well.

What Problem Does This Solve?

If you found the opening question hard to answer, the reason usually lies in the first sentence you wrote when the work began. We write down tasks instead of problems more often than we think, and when the task is done, we consider the work done too. So what exactly is the difference between a task and a problem, and why do we so naturally start by writing tasks?

Tasks and Problems Are Different

First, let’s line up three sentences you’ll often hear in a development organization.

The payment module code feels way too complex. It’d be nice to refactor it this quarter.

Our LCP p99 is too high. Let’s get it under 2 seconds.

Our goal for the second half of the year is to raise the organization’s productivity.

Hear any of these in a meeting and you’d probably nod along. But look closely, and none of them actually states a problem.

The first two describe only the means, with no goal. Refactoring and performance work are just means of solving some problem, yet these sentences never say what that problem is.

The last one, on the other hand, is too abstract. “Raise productivity” looks like a good goal, but it doesn’t say what is low, by how much, or what state things should reach. Wanting to raise productivity implies productivity is currently low, but everything is missing: how productivity is defined, and why it’s considered low.

This kind of naive problem definition isn’t limited to developers either. “Let’s redesign the product detail page” has only a means, and “Let’s raise brand awareness” has only a direction.

In the end, whichever sentence you pick, what’s missing is an observation of what is actually happening right now.

I define a problem rather dryly: a problem is the gap between the state you expect and the state you actually have. Defined this way, stating a problem requires at least two states, where things are now and where they should be. This isn’t a method only I use. Toyota is well known for defining a problem as “the gap between how things should be and how they are,” and for calling the process of closing that gap improvement.

This may sound like a small play on words, but the distinction largely determines whether you can later answer the question “Why did we do this?” Work that starts as a task can only be explained as a task once it’s done. “We refactored” and “We adopted an AI tool” are facts, but since what gap they were supposed to close was never defined in the first place, there’s no way to explain it. That’s how you end up with “I think it got better.”

Why Means and Directions Come to Mind First

So why do we so naturally reach for means or vague directions first?

Daniel Kahneman, the psychologist who won the Nobel Prize in economics, explains this tendency with the phrase “what you see is all there is.” People quickly build a plausible story out of whatever information is in front of them, and rarely consider that something might be missing. When you’re staring at code, most of the information in front of you is code, so you look for both the cause and the fix in code. When you’re looking down at the whole organization, most of what’s in front of you is a vague impression, so the goal grows just as big and blurry.

There’s one more force at work here: the sense of progress that comes from knocking out tasks. Defining a problem rarely leaves a visible artifact. You can think about it all day and not close a single ticket. A refactor, on the other hand, changes files, opens PRs, and gets merged. (Few things deliver dopamine as reliably as a green merge button) Since the feeling of getting work done is much stronger with the latter, we keep jumping into solution mode before we’ve understood the problem well enough.

How Do We Know the Problem Actually Exists?

Once you know tasks and problems are different, the next step is to check whether the problem really exists. Problems mostly start from a feeling that “something seems off,” and the job at this stage is to turn that feeling into a phenomenon you can share with other people. And once the phenomenon is written down properly, you also start to see whether the problem is big enough to be worth solving now.

Intuition Is Only the Starting Point

I talked earlier about the tendency to reach for means first, but that doesn’t mean you should throw away the intuition that “this code seems too complex.” If anything, I think most good problem definitions start from exactly this kind of intuition. When someone who touches that code every day senses that something is off, there’s usually something there.

The problem lies less in the intuition itself than in locking it in as the problem as-is. Intuition is a hypothesis, and a hypothesis stays a hypothesis until it’s confirmed. So how do you confirm it?

The first thing you can do is think about what you’d actually expect to observe around you if the intuition were true. If the payment module really is a problem because it’s too complex, you’d likely observe things like these.

Adding payment-related features often takes longer than originally estimated
Bugs keep popping up elsewhere after changes to the payment module
PRs touching the payment module take unusually long to review or collect lots of comments
New team members ask questions about the payment module a lot

If even one of these is actually observed and you can count how often it happens, the intuition turns into a phenomenon. Compare the estimated and actual timelines of last quarter’s payment tickets, or count how often the payment module was named as the cause in incident postmortems. None of this needs heavy analysis; most of it can be checked in an hour or two. (These days, an LLM can pull it together for you in no time)

One more thing matters here: before you actually look, decide in advance what result would make you admit your intuition was wrong. If you don’t, whatever number comes up, you’ll read it in a biased way that supports your intuition. People naturally tend to be better at spotting information that supports what they already believe.

For the payment module, you might set it like this.

“If fewer than three pieces of work touched the payment module last quarter, it’s not a problem worth solving now, however complex the code is.”

This condition captures something quite important: complex code that nobody touches costs less than you’d think. Most of the cost of complexity arises when you have to change that code, so if the code isn’t going to change, whether it’s complex or not probably makes little difference to the business.

Now suppose that, with this criterion set, you actually look and find that of 8 payment-related feature tasks last quarter, only 3 finished within the estimated timeline, and 4 of the 5 that ran over took more than twice as long.

With 8 pieces of work touching the payment module, the falsification condition doesn’t apply, and the intuition that “this seems complex” becomes a phenomenon anyone can verify.

In other words, confirming an intuition through observation doesn’t just tell you whether it was right or wrong. Even when it’s right, it also reveals whether the problem is worth solving now.

Write Down Phenomena, Not Interpretations

Once observation has confirmed the intuition, it’s time to write down the current state. This is what’s often called the As-Is, and the most common mistake here is writing an interpretation instead of the observed phenomenon.

Take the payment module from earlier: “The payment module’s code quality is low” is an interpretation, while “5 out of 8 payment-related tasks last quarter ran past their estimated timelines” is a phenomenon.

Interpretations vary from person to person, so if someone replies to “the code quality is low” with “Looks fine to me,” the conversation stops there. But if someone disagrees with “5 out of 8,” you can just open the tickets together.

The same goes for a design system. “Twenty-three different button styles serve the same role across the service, and 40% of last quarter’s design QA issues were style inconsistencies” says far more than the interpretation “The UI is all over the place.”

That said, a sentence doesn’t become a phenomenon just because it contains a number. When performance comes up, you often see sentences like this.

Service A’s LCP p99 is high at 3.2 seconds.

There’s a clear number here, and since it looks at the slowest 1% rather than the average, it seems like a pretty good metric was chosen.

Yet even after reading it, some things remain unknown: who the users in that 1% are, what page they’re stuck on and what they’re trying to do there, and what isn’t happening as a result. Set a goal of bringing the whole service’s p99 down to 2 seconds without these, and you’ll likely end up fixing the easy pages before the important ones.

Written as a phenomenon, the same situation might look like this.

The product detail page is the page 60% of all orders pass through, and its LCP p99 is 3.8 seconds. Visits where the first screen took over 3 seconds convert at about half the rate of other visits, and there are roughly 20,000 such visits a day.

This sentence contains where (the page 60% of orders pass through), what is being experienced (the first screen takes more than 3 seconds), and what is being lost as a result (20,000 visits a day converting at half the rate).

The last part is especially important, because it answers not only whether the problem actually exists but whether it’s big enough to be worth solving now. The same 3.8-second p99 on a settings page that gets only a few hundred visits a day could quite reasonably wait behind other urgent work.

Problems usually exist several at a time, and our time is limited. So when you write down a phenomenon, you need to include what it’s costing and how much, so that you can pick which problem to tackle first.

After writing an As-Is, I always check one thing: if a colleague seeing this sentence for the first time read it, would they picture the same phenomenon I do? Number or no number, where the problem occurs, what is being experienced, and how big it is need to be shared before everyone can picture the same thing at the next step, the goal.

Why This Approach?

Once you’ve confirmed the problem actually exists, it’s time to decide what to do. But if you jump straight into a solution here, it’s easy to slide back into the task-centered sentences we saw earlier. Before choosing an approach, you first need to write down what state things should reach, and then trace why the gap between now and that state exists. Only then can you answer the question “Why this approach?”

Write Goals as Outcomes

With the current state written down, next comes writing down the state things should reach. This is what’s often called the To-Be, and it’s also where most of my feedback lands. The question “So what exactly gets better?” mostly comes up here.

Take AI tool adoption, something many organizations care about these days. Ask for goals when bringing a new tool into an organization, and you’ll often get sentences like these.

90% adoption of AI coding tools across all developers
Maintain a weekly active user rate of 70% or higher
AI-generated code in 30% or more of all commits

These are clear and easy to measure, but every one of them does nothing more than describe what we did. I split this into outputs and outcomes. An output is something we built or changed; an outcome is how people’s behavior or circumstances changed because of that output. What belongs in a goal is closer to the outcome than the output.

Suppose you work hard on the rollout and finally hit 90% adoption. So what got better?

To answer that properly, you have to go back to what we first defined: what problem were we trying to solve with the AI tool? Adopting a new tool is a means of solving a problem, and the means itself can’t be the goal.

Conversely, if the problem was properly defined, the goal naturally takes the shape of an outcome. Say you observed that “repetitive migration work, like major library version upgrades, eats up 20% of the team’s available time every quarter.” Then the goal could be “Cut the time spent on migrations in half, and use the time freed up to increase the number of product iterations per quarter.” AI is just one of several means of reaching that goal.

Among the cases I’ve seen, the ones that went well mostly settled on metrics before tools. Before adopting, they defined what they’d look at for development productivity and for product quality, and watched how those metrics moved as the tool spread. That way, the adoption rate isn’t the goal but an interim check on whether the means is being used properly.

Outputs are easy to write because we control them, and whether they’ve been achieved is clear. So it’s natural for outputs to end up in goal documents. The trouble is that achieving an output doesn’t guarantee that the gap you originally defined gets smaller.

In an earlier post, What Leaders Should Really Worry About Isn’t Productivity, I wrote that AI can rapidly increase output while quietly eroding the ability to verify and understand it. If you only watch output metrics like adoption rate, you’ll never see that shift.

Refactoring is no different. “Consolidate payment state management in one place” is an output; “Raise the share of payment-related work finished within the estimated timeline from 3/8 to about 6/8” is an outcome. And once you’ve written it this far, you can finally explain how the refactor connects to the business. For an organization that runs payment conversion experiments often, payment work finishing on schedule means being able to run more payment experiments, more predictably, in the same amount of time.

I think the business value of refactoring mostly shows up this way. Refactoring doesn’t generate revenue by itself, but it changes the speed and stability of the changes to come. So to explain the value of a refactor, you also have to talk about what changes are expected to hit that code, and how many. If you can’t make that connection, refactoring tends to be seen as a matter of developer taste; if you can, it may even become something the product team asks for first.

As-Is and To-Be: Why Does the Gap Exist?

Once you’ve written down the current state and the state things should reach, the gap between them becomes visible. Now you have to ask why that gap exists. People often skip this step and go straight to a solution, but I’ve fairly often seen this be exactly the step where you learn the solution you first came up with was wrong.

Let’s start with the payment module. Gather the postmortems of the 5 tasks that ran over, and a common story emerges: the code that changes payment state is scattered across many places, so figuring out the blast radius of a change took a long time. Ask why once more, and the reason you only pay for a misjudged blast radius at the QA stage is that the payment flow has almost no automated tests. In that case, closing the payment module’s gap is less about rewriting the code in one go and more about catching problems early with tests and consolidating the scattered state-change paths.

Now let’s go back to the goal “Raise the organization’s productivity.” Suppose writing down the phenomenon produced a picture like this.

On average, it takes 3 weeks from a feature request to deployment. Of that, the time actually spent writing code averages about 4 days.

So where did the remaining two weeks or so go? In cases like this, I tend to ask “why?” repeatedly, digging past the visible symptoms down to the cause.

Why does it take 3 weeks to deploy? Because even after coding is done, there’s a lot of waiting.
Why is there waiting? Because waiting for review, waiting for another team’s API work, and waiting for a QA slot follow one after another.
Why wait for other teams? Because the teams are split so that building a single feature always requires work from three teams.
Why did it end up structured that way? Because the teams were originally divided by expertise, and as the product grew, the boundaries between teams drifted out of line with the units of user-facing features.

This isn’t a method only I use, either. Toyota has long used the 5 Whys, asking “why?” five times whenever a problem arises, and it has since become a common sight in many companies’ incident postmortems.

Dig down this way, and it becomes quite likely that the cause of “low productivity” isn’t the speed of writing code at all. This is also where it connects back to the AI tool discussion. If the cause is mostly waiting, then even if an AI tool cuts coding time from 4 days to 2, the time to deployment only drops from 3 weeks to about two and a half. The tool adopted to raise productivity never touches the biggest part of the gap.

Something similar often happens with design systems. Seeing that there are 23 button styles, it’s tempting to think “we don’t have shared components,” but take a closer look and often the components already exist and just aren’t being used. They’re hard to find, there’s no way to customize them when you need something slightly different, or the designs handed over by designers are already inconsistent to begin with. If the cause lies there, building more components won’t close the gap.

Of course, the 5 Whys doesn’t always converge on a single right answer. Causes are usually tangled together, and depending on which branch you follow, you can arrive at different ones. I see the 5 Whys less as a tool for finding the right answer and more as a tool that makes you doubt the first solution you came up with. In the end, what you want to confirm at this step is just one thing: if you remove this cause, does the gap you defined earlier really shrink?

Small Steps, Not One Big Swing

Once you’ve found the cause, you plan a strategy to remove it. The most common strategy here is the one-big-swing strategy. For a refactor, “rewrite everything in the new structure”; for an AI tool, “roll it out company-wide at once”; for a design system, “build every component and then ship.”

The biggest problem with strategies like these is that you only find out whether they worked on the very last day. For months, there’s only investment and nothing to check. And when it’s over and someone asks “So what got better?”, you’re in the sad position of still struggling to answer. So much changed at once that there’s no way to tell what actually made the difference.

I think a strategy should be broken into small steps wherever possible, and each step should visibly shrink the gap defined earlier. The Strangler Fig pattern introduced by Martin Fowler carries the same idea: instead of replacing the old system at once, the new system takes over features one at a time and gradually shrinks the old one.

If you were improving the payment module over the second half of the year, you could break the strategy down like this.

September: Add automated tests to the three core payment paths. Watch whether work bouncing back from QA decreases.
October: Consolidate the two most frequently changed pieces of payment state-change code into a single path. Watch whether schedule overruns on work related to those two areas decrease.
November: Move the remaining state-change code onto the same path. Look at the cumulative on-schedule rate for all payment work since September.
December: Look back on the whole half-year and check how close you got to the goal you originally defined.

The same goes for AI tool adoption. Instead of rolling it out company-wide at once, try it first with the one team that spends the most time on migrations, and check whether that team’s migration time actually went down. If it did, expand to similar teams; if it didn’t, figure out why before deciding on the next step.

Broken down this way, it becomes clear what the picture should look like at the end of each month. That’s exactly why I kept asking leads, “What will things look like at the end of September, October, and November?”

Without a monthly picture, you only learn in December whether the strategy worked, and by then it’s too late to change course. On the other hand, if you add tests in September and the work bouncing back from QA doesn’t decrease at all, you’ve already gotten a signal in September that something in your cause analysis was wrong. Then you can change October’s plan, whereas getting the same signal in December just means a failed project.

String this whole process together in one line, and activities, outputs, and outcomes link up like a chain.

Activity: consolidate the payment state-change code into a single path
Output: a payment module with its state-change paths organized in one place
Outcome: a higher share of payment work finishes within the estimated timeline
Business impact: more payment experiments can be run, more predictably, in the same amount of time

This chain is the first thing I check when I look at a strategy document. And in my experience, the link that breaks most often is the one from output to outcome. People usually write down what they’ll build just fine. But ask them to write down why building it will change the outcome, and the sentences suddenly go blurry. “It should get better” almost always comes from right here.

That said, not every step shows up in the numbers within a month. If a metric swings 10% week to week and a single step’s effect is 3%, it’s hard to see improvement step by step, and some investments, like code quality work, only show up in metrics such as incident counts months later.

In cases like this, you might as well just lengthen the check-in cycle, or look instead at an intermediate metric you can reasonably expect to lead to the outcome. A principle of checking the numbers every month shouldn’t turn into a demand to confirm what can’t be confirmed.

How Do We Know the Approach Actually Solved It?

Once you’ve chosen an approach and put it into action, the last remaining question is whether it actually shrank the problem. It’s also the second half of the question from the beginning of this post. I think answering it takes numbers written down before you act, and the habit of checking those numbers against reality.

The Numbers You Need to Say “It Got Better”

This is exactly where “I think it got better” and “It got better” part ways.

When this topic comes up, there’s a reaction I see often. Suggest setting a quantitative criterion, and people go looking for a number as irrefutable as 1+1=2.

That reaction usually comes from a line of thought like “Even if the time to deployment went down, how do we know it was because of what we did? Maybe the requirements were easier in the meantime, or people changed teams.” And honestly, that’s not wrong. But start reasoning that way and there’s almost no number in the world that can fully prove the effect of any improvement.

If you’re a developer, this reaction probably feels especially familiar, and I think the reason lies in the nature of the world we’re used to. The world of code is mostly deterministic. The same input gives the same output, and tests either pass or fail. But measurement in a world where people and organizations are intertwined doesn’t work that way. The same action produces slightly different results every time, and the factors affecting the result are countless.

So I think it’s better to look at measurement a little differently. Measurement isn’t about proving something; it’s about making what you don’t know a little less unknown. If deployment used to take 3 weeks and now takes 2, you can’t prove it was all thanks to what you did, but it’s more than enough reason to shift your belief toward “it had an effect.”

Then how do you tell which numbers are useful among those that aren’t 100% accurate? I care about two things more than precision.

The first is whether you wrote the number down before acting. If you say “let’s find a number that shows what got better” after the work is done, you can almost always find at least one number that improved. Out of dozens of metrics, one will have gone up, if only by chance. A number picked after the fact is unconvincing no matter how accurate it is. A number written down before acting, as in “this will move by this much,” is far stronger even if it’s a bit rough. It’s not proof but a prediction, and a prediction that comes true is a sign that you understood the problem correctly.

In fact, the more people are asked for numbers after the fact, the stronger the urge can become to find axiomatic numbers. A request to show the value of already-finished work in numbers feels like a request for proof, and proof needs a number that can’t be refuted. Ask the same question before acting, and the answer becomes a prediction. Predictions are allowed to be wrong, which makes it much easier to offer a roughly sensible number.

The second is whether the number can change your next action. If the on-schedule rate for payment work doesn’t rise at all in October, will you change November’s plan? If you would, the number means something. If you’d go ahead with the same plan no matter what the number said, the number is closer to decoration for a report than to measurement.

Rather than hunting for a single perfect metric, it’s more realistic to look at two or three imperfect metrics that point in the same direction. For performance, you could look at LCP p99, bounce rate, and the rate at which product detail visits move on to the cart together. All three are affected by factors other than performance, but if all three move in the same direction at once, the odds that it’s a coincidence drop considerably.

For organizational productivity, the metrics compiled by Google Cloud’s DORA research team can be a good starting point, but rather than looking only at speed metrics like deployment frequency and lead time for changes, you should look at stability metrics like change failure rate and time to restore alongside them. Of course, these too are reference points, not axioms that prove productivity.

I think the same principle applies to setting key results in OKRs. John Doerr, who first introduced OKRs to Google, says key results must be measurable and verifiable, but there’s no need to read that as “it must be a logically perfect number.” Numbers like that barely exist in reality anyway. A number agreed on before acting, whose result can change your next action, can make a perfectly good key result even if it’s a bit rough.

Numbers Are the Finger Pointing at the Moon

Reading up to here, it might sound as though all you need is to set the right numbers, but there’s one thing you have to hold on to after setting them: a number isn’t the problem itself but a finger pointing at the problem. As the old saying goes, when a finger points at the moon, the fool looks at the finger. Set a number as the target, and over time people’s attention drifts from the problem toward the number.

For example, set a 90% test coverage target and tests that verify nothing start to multiply; set a deployment frequency target and you may just get more deployments that change nothing for users. Workarounds emerge to push the number up rather than to solve the real problem. The number goes up, but the gap you originally wanted to close stays the same. This is the phenomenon commonly known as Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.”

That doesn’t mean you have to give up on numbers. When a number moves, I tend to check one more thing alongside it: did the phenomenon I originally wrote down improve as well? If coverage went up, did the work bouncing back from QA go down too? If deployment frequency went up, did the time it takes for features to reach users get shorter too? If only the number moved and the phenomenon stayed put, that’s a sign you’ve been looking at the finger. Writing numbers down before acting and looking at several imperfect metrics together, as suggested earlier, are ultimately devices to keep your eyes on the moon.

On the flip side, just because it’s hard to point a finger at something doesn’t mean the moon isn’t there. Robert McNamara, the U.S. Secretary of Defense during the Vietnam War, is criticized for judging the course of the war by countable numbers like enemy body counts while missing everything the numbers couldn’t capture. This trap, measuring only what’s easy to measure and treating what’s hard to measure as if it didn’t exist, is named after him: the McNamara fallacy. Things like documentation, onboarding, and the cognitive load felt by people reading code are hard to capture in numbers, but they clearly exist. Rather than dropping such problems because they’re hard to quantify, even finding rough metrics or steadily recording what you observe makes them well worth addressing.

Compare Predictions With Reality

Once you’ve set your numbers and executed your strategy, it’s time to compare predictions with reality.

At the end of each month, put the picture you wrote down at the start next to the actual picture. If at the end of September you predicted that work bouncing back from QA would drop by half but it actually dropped by only 20%, jot down why, even briefly. Maybe the paths you added tests to weren’t the ones that change often, or maybe more of the bounce-backs had causes tests can’t catch than you expected.

This matters because it’s where you develop a feel for choosing “roughly sensible numbers.” At first, everyone’s predictions are way off. They’re too optimistic, or they pick the wrong metrics. But after writing predictions down, getting them wrong, and checking why a few times, you develop a sense of how much precision is enough and which metrics actually reflect the problem well.

Without this process, on the other hand, a goal document becomes something you write once at the start of the quarter and leave in a drawer. And when the quarter ends and someone asks “Why did we do this?”, you end up right back at “I think it got better.”

The Ability to Define Problems Grows From the Questions You Ask Yourself

So how can this ability be developed? I’ve laid out a fairly long process, but I don’t think having grown this ability means having memorized that process. It’s closer to a state where the necessary questions come to mind on their own before anyone asks.

I once turned this process into a course and ran it, and what I learned then was that learning a method and pulling it out when facing real work are quite different things. In a classroom, the template asks the questions for you, but in real work, nobody asks them on your behalf. So what’s needed isn’t some grand course of study, but a habit of asking yourself those questions right where you work every day.

Imagine Someone Asking You

The method is simpler than you’d think. Before starting a piece of work, just once, imagine someone asking you this.

What problem does this solve?

How do we know the problem actually exists?

Why this approach?

How do we know the approach actually solved it?

As you may have noticed, these four questions were the section headings of this post. The first makes you separate tasks from problems, the second makes you turn intuition into observation and phenomena, the third makes you derive the approach from the goal and the cause, and the fourth makes you write down the numbers and predictions to check before acting. If you get stuck on even one of the four, it’s a sign the problem hasn’t been defined well enough yet.

If you’re a developer, rubber duck debugging makes this easy to understand. Explain your code line by line to the rubber duck on your desk, and at some point the bug becomes visible, even though the duck never says a word. The moment you try to explain, the parts you’d been glossing over in your head get exposed. An imaginary questioner plays the same role. The moment you try to answer, the blanks you’d been covering with “it should get better” get exposed.

That said, people tend to go fairly easy on themselves when they only imagine things in their heads, so I recommend two things.

One is to actually write down an answer to each question, even just a sentence each. An answer that sounded plausible in your head quickly shows where it’s empty once you try to put it into words. The other is to make the questioner specific. Picturing the face of the colleague or lead who asks the sharpest questions works far better than some vague someone.

If You’re a Lead, Be That Questioner

If you’re a lead, you can go one step further: at first, you can be the actual questioner. When I kept asking similar questions every time I reviewed a goal document, that’s ultimately the role I was playing.

But if these questions live only in the lead’s head, team members end up playing a game of guessing the lead’s criteria instead of defining the problem. So I think it’s better not to hide the questions but to put them out there in advance. Share these four questions before people start writing their documents, and they can ask themselves those questions about their own documents before ever meeting the lead. Where the classroom template was a set of questions attached to practice problems, these are questions attached to the very moment of writing a real document.

After a few cycles, answers start showing up in the documents before the lead even asks. The imaginary questioner has quietly taken up residence inside them. I think that’s close to what it really means for the ability to define problems to grow.

Closing Thoughts

Let’s go back to the three sentences we lined up at the start.

“Let’s refactor the payment module” could become “Only 3 of 8 payment tasks are finishing within the estimated timeline, so let’s get that to 6 and run payment experiments more often.” “Let’s get LCP p99 under 2 seconds” could become “On the product detail page, which 60% of orders pass through, there are 20,000 visits a day where the first screen takes more than 3 seconds, and those visits convert at half the rate, so let’s start with that segment.” And “Let’s raise the organization’s productivity” could become “Let’s cut the waiting time that takes up more than 2 of the 3 weeks it takes a feature to ship.”

There’s no grand methodology in these rewritten sentences. They’re simply the result of answering the four questions once each. And these aren’t questions you answer once and are done with; you need to ask them again every time your understanding of the problem shifts as you move in small steps. Toyota’s 5 Whys, Bezos’s approach to decisions, and the OKRs Google made famous differ in form but ultimately rest on similar questions, so this is less a new story than an old one.

Even so, I think this process often goes unfollowed because every step feels less productive than knocking out tasks. Time spent counting, polishing sentences, and writing down predictions doesn’t close tickets. Still, I believe this time ends up saving the most time of all. That’s all the more true when you think of the weeks spent solving problems that didn’t need solving, and the months spent never knowing whether a problem was solved at all.

At the start of this post, I asked you to recall the one piece of work you put the most effort into last quarter. When you start your next piece of work, just once, imagine someone asking you four questions about it. By the time the work you can answer them for starts to add up, you’ll find you can give a somewhat firmer answer than “I think it got better” when someone asks “Why did we do this?”

And with that, I’ll bring this post on properly defining ambiguous problems to a close.

On Properly Defining Ambiguous Problems

댓글