Modernization under uncertainty

Decisionmaking under extreme uncertainty: how to make the “fog of war” less foggy

Nobody in this position has good information. Not the people who appear to, and not the ones presenting confidently at the conference.

By Jacob Andra/Talbot West/08-04-2026

Legacy operations of any complexity face a mix of pressures that pull hard in opposite directions. Executives are under pressure to modernize to stay competitive, and under a mandate not to break the production systems that already work.

Resources need to be allocated to relieve these pressures (choosing not to allocate resources is itself a resource allocation decision), yet extreme opacity makes optimal allocation non-obvious. Meanwhile, the cost of poor resource allocation is high.

What separates the executives who come out of a modernization cycle with both their operation and their standing intact is not that the fog lifted for them. It is that they made a different kind of decision inside it, on grounds that do not require knowing what cannot yet be known. Those grounds are learnable, and they are mostly not in circulation.

01

What the fog is made of

01.1

"Fog of war" is a phrase we hear a lot for this situation, and "fog" is both apt and inadequate, because it suggests a single condition that either lifts or doesn't.

What obstructs these decisions is a set of separable issues with different sources and different remedies. Here is what we find in C suites on a regular basis.

Position

Yields to structure.

  1. 01

    Pressure to act arrives from outside and inside at once, and both versions come pre-loaded with position. Competitive pressure to "adopt before we get left behind" outstrips any actionable assessment of what should be adopted and how. Meanwhile internal opinion splits along predictable lines, some factions pushing to modernize and others for the status quo.

  2. 02

    Part of the decision was made before it arrived. A platform is already bought. A vendor relationship has an owner. A commitment exists in public, and a prior initiative has to be seen as having worked. None of that shows up in an options analysis, and all of it narrows the real option set before the first slide goes up.

Information

Yields to targeted investigation guided by the proper questions and assumptions.

  1. 03

    The documented operation and the real one are two different things. The process that runs the business lives in workarounds, exception handling, and a few people's heads, and none of it is written down anywhere you can go read it.

  2. 04

    Nobody has the whole map of the terrain, which consists of internal operational reality and current technological trends and market conditions. Smart internal people hold fragments of it. Vendors may hold other scraps. You need to assemble the most important fragments, the ones in the critical path. And it's not always easy to know which those are.

  3. 05

    Almost everything you know about your own operation arrived through people with a stake in how it gets described. Not through dishonesty, mostly, but through ordinary self-presentation, compounded at every level it passes through on the way up. The practical consequence is that confidence in the description tends to increase with distance from the work.

  4. 06

    Technological capability is hard to assess in advance, because a demo tests in isolation while complexity changes things. Production tests the organization around it: the exceptions, the handoffs, the people who have to trust the output enough to change what they do. What fails is rarely the capability.

  5. 07

    Most of the confident information arriving from outside the building comes from someone with something to sell. Biased vendor advice pushes executives toward suboptimal platforms, tech sprawl, and fragmentation. Peer signal is no help either: what comparable companies announce publicly bears almost no relationship to what is running in production at those companies.

Time

Yields only to shortening the loop.

  1. 08

    The subject of the evaluation changes faster than the evaluation does. What the technology can do, what it costs, which vendors will still be here, and what competitors have put into production all move on a shorter cycle than a careful assessment takes. A thorough twelve-month evaluation returns a verdict on a world that has already been replaced, and the more thorough it is, the more true that is.

  2. 09

    Whatever gets decided, there's no control group to assess the decision against. And there was usually never a baseline either, because nobody measured what the current process costs or how often it fails, so there's nothing to compare against afterward. The call gets graded on whatever the operation looks like a year and a half later, by people reconstructing causation from memory.

Cost

Yields to choosing moves you can walk back.

  1. 10

    The cost of a wrong call isn't only money. It's spent change tolerance and spent credibility, and neither one can be bought back. Political capital goes with the misstep, the company becomes more change-averse, and the pressures keep building.

Order of operations

A single decision is hard enough. A sequence is infinitely more complex.

  1. 11

    There's an optimal order of operations, many suboptimal sequences, and some disastrous ones as well. It's difficult to tell which is which, and extremely improbable that you land on the optimal order immediately.

02

Tiers of answer

02.1

Executives need to make expensive resource allocation decisions under extreme uncertainty. Unknowns sort into three tiers of accessibility.

Cheap answers

  • One conversation with the person who does the work
  • One sample export covering a real month
  • One question about who signs when the output is wrong

These get skipped constantly, not because anyone decides to skip them, but because they are too small to appear on a plan, or they get lost in the other two categories.

If cheap answers are in the critical path, get answers to them ASAP and your world of uncertainty shrinks immediately.

Expensive answers

  • Reconciling two systems that disagree about what a part is
  • Establishing the current human error rate on a process nobody has ever measured

Expensive answers are sometimes worth answering. Pursuing one is itself a funded decision with its own order-of-operations question attached.

It should be argued on those terms rather than filed under discovery and waved through.

Near-impossible answers

  • What the technology will be able to do in eighteen months
  • Whether a key person will remain with the company
  • What moves your competitors will make next quarter

Don't invest in trying to answer unanswerables. Identify them early on and set them aside.

You can develop mitigation strategies, but analysis can turn against itself and drag the overall project down.

02-A

Errors with answer tiers

02.2

Organizations tend to spend most of their study budget on the third category, where it cannot succeed, treat the second as though it were free, and never systematically mine the first.

The discipline that redirects it costs nothing
Category 01

Identify your critical "Category 1s" and answer them.

Category 02

Triage your Category 2s and decide on a case by case basis.

Category 03

Draw a circle around your Category 3s and don't overinvest in them.

For each thing you do not know, ask whether knowing it would change what you do next.

If the answer is no, it is context, not a blocker. A large share of what stalls these decisions turns out to be context.

02.3

Cost to resolve is only half the sort. The other half is how much leverage an answer provides.

An expensive answer that unblocks four initiatives is more valuable than a cheap one that changes nothing. Prioritize exploration of unknowns that multiple other questions depend on.

03

The "on paper" version vs reality

03.1

The documented process and the running process are different objects.

Every workaround is a repair made by someone competent at the exact point where the official process fails, and it encodes three things worth more than most process documentation:

Where the failure is
What it costs
What the organization was willing to spend to route around it
03.2

This is why walking the exception path teaches more in a week than reading the process library teaches in a month.

The exceptions are where the real constraints are recorded. Anything built against the documented version of a process meets the real version in production and loses.

That nobody holds the whole map is not a staffing failure anyone can hire their way out of, and the remedy is not convening more people. It is knowing, for the specific decision in front of you, which part of the map is missing and what that particular ignorance costs. That question is answerable.

04

Three questions worth asking

04.1

Under these conditions most of the standard decision apparatus stops working. You cannot put meaningful probabilities on outcomes when the inputs are this soft, and doing it anyway produces theater.

The questionHow it is restructured
01

What kind of wrong does this produce and what type of currency does it spend?

Not how likely you are to be wrong. What type of cost. Money spent badly is often recoverable. Change tolerance and political capital are often more expensive, which means a small initiative that fails visibly can cost more than a large one that fails quietly. Sort candidate moves by the currency they can lose, and the ranking changes.

Change its error shape, by making it advisory instead of authoritative.

02

How fast will you know?

Feedback latency is the most underrated variable in the whole problem. A decision that reports back in six weeks is worth more than a better decision that reports back in three quarters, because by the third quarter the sponsor has moved, the team has turned over, and nobody can reconstruct what caused what. An organization with long feedback loops is crippled, regardless of how capable the people in it are.

Shorten its feedback loop, by instrumenting the process before changing it, or by choosing the one site or product line where the answer arrives in weeks.

03

What does it cost to find out, and what does it cost to get back out?

The price of the test is a separate quantity from the price of the decision, and almost nobody prices it separately. Neither is the same as the cost of reversal.

Lower its exit cost, by renting instead of buying, or running in parallel instead of cutting over.

Error shape, feedback speed, and exit cost are design variables, not facts about the situation.

The same objective can be pursued in ways that have entirely different error shapes, feedback speeds, and exit costs. We almost never see them treated that way on purpose, so the choice gets made by default.

Reversibility is a social property, not a technical one.

The rollback is usually the easy part. What does not roll back is the trust spent asking a floor supervisor to work a new way, and the appetite consumed by a visible attempt that went sideways. A change that is trivial to undo in the system can be permanent in the organization.

05

Judge the decision, not the outcome

05.1

Whatever standard an organization uses has to be one it can apply at the time of the decision rather than after the fact.

Under uncertainty this deep, good decisions produce bad outcomes at a steady rate.

Mechanism

An organization that grades on outcomes teaches everyone who works in it that the safest available move is not to decide, which is the mechanism behind the change-averse spiral described above.

Nobody chooses that. It gets installed one post-mortem at a time.

05.2

The counter is to define, in advance, what a well-made decision looks like independent of how it turns out:

The recordWritten down before you commit
01What was known
02What was assumed
03What would have changed the call
04What the exposure was if it went wrong
05How long it would take to find out
05.3

If the outcome is poor and the record shows the decision was well made, you have a defensible position instead of a search for someone to blame. This is also the only real protection for the credibility and political capital budget.

06

Buy the answer that matters

06.1

Each move has a cost, and each buys information.

Ask what information a move buys, and whether it is on the critical path.

For every move, pose the following two questions

If it succeeds, does the plan change?

If it fails, does the plan change?

Often, organizations are buying information they already have, or that is non-critical.

06.2

A technology pilot is a common move that answers whether a capability functions, which is worth knowing when capability is the critical question (and it usually is not).

See Addendum D for other common failure modes.

07

Uncertainty in dynamic systems, and sequencing decisions

Portfolio

A sequence is a route through a constraint graph, chosen under uncertainty and paid for out of accounts that deplete.

Everything prior in this article models the complexity of making a single decision under uncertainty. But in reality, decisions need to be made in clusters. A portfolio of decisions is much thornier than a single decision. The portfolio question is about sequencing, and sequencing requires more advanced logic from go / no-go.

07.1

A yes/no scores candidates one at a time. That is a legitimate thing to explore, but it cannot answer what to do in what order, because it has no representation of prerequisites, dependencies, adjacencies, or how much scarce capacity two apparently unrelated initiatives share.

In the field

The initiative everyone agrees is the most valuable is, more often than not, the one furthest down a chain of dependencies nobody has built.

It gets funded, it stalls the moment execution exposes the blockers, and the review that follows re-ranks it lower without saying so, as though the idea had gotten worse rather than the order having been wrong.

07.2

Order of operations also determines what accumulates. Each capability leaves behind things the next one either inherits or duplicates. A move that leaves a reusable foundation beats one that leaves an orphan, and a yes/no rubric cannot see this.

Point solutions almost always reduce legibility, which is the mechanism behind sprawl, and the reason the fifth purchase is harder to place than the first. This is the primary argument behind Talbot West steering clients toward compounding capabilities and away from fragmentation, though there are other benefits to the integrated approach as well.

07-A

What is actually in the way?

Premise

Ordering by what is in the way requires knowing what being in the way looks like, and the answer is less technical than the conversation usually assumes. The binding constraint is almost never whether a given technology can do a given task.

A fact about data

Two systems disagree about what counts as the same customer or the same part, and someone has been reconciling the difference by hand for years.

A fact about authority

A vendor owns the system of record and sets its own terms for who gets access this year, and no amount of internal agreement changes that.

A fact nobody has said out loud

Nobody has agreed who signs when the output is wrong. Nobody knows what the current error rate is on the process being replaced, which means there will be no way to prove the replacement is better even after it is running.

Note

Each of these costs something different to clear, and each has its own way of being found out. Most of those discovery tests are small; discovering which constraint binds a given initiative is most of the diagnosis, and it is the part that almost never gets done before the funding decision. Which constraint binds is also the highest-leverage thing you do not know, because it is the answer that reorders everything else. That is why finding it out is worth funding ahead of the initiative it gates. The full set, with the test for each, is in Addendum C.

08

The budgets this gets paid out of

08.1

Sequence is constrained by three accounts that deplete, and initiatives that occupy separate lanes on a portfolio slide usually draw on the same accounts at once.

Integration capacity

The number of people who can safely change a system of record is small, effectively fixed for the next several quarters regardless of headcount, and every initiative wants the same names. The org chart shows teams. The constraint is individuals.

Recovers with slack

Change absorption

An organization can absorb only so much simultaneous change before adoption degrades across everything already in flight, not just the newest addition. This is a capacity problem, not a motivation problem, and it behaves like one. Exceed it and the initiatives already running slow down too, so the cost almost never gets charged back to the decision that caused it.

Recovers with slack

Credibility and political capital

This one behaves differently from the other two. Integration capacity and change absorption recover with slack: ease off for a quarter and some comes back. Credibility only refills when something visibly works. A program that spends two rounds in a row on invisible foundation work cannot replenish it, and by the time it produces something legible the sponsorship that was real at kickoff has often already gone.

Only refills when something visibly works
09

The reasoning applied

09.1

Let's imagine a manufacturer running several production programs at once, each with its own schedule commitment. Leadership is considering investing in demand forecasting, supplier risk scoring, automated quality inspection, automated quoting, and a knowledge system to capture process expertise before it retires out the door. Every one is worth wanting. Scored independently, several tie.

The first pass is not scoring. It is asking what each one is waiting on, and which of those things does not exist yet.

Quoting and demand forecasting both turn out to depend on the same part existing under two different identities in two different systems, reconciled today by one planner who knows both. The knowledge system and quality inspection both depend on the same few senior engineers to specify what correct looks like. No value-versus-feasibility grid has a column for either one. Both decide the order.

One manufacturer / one set of facts
  • The purchase order history is already clean
  • Delivery performance is scattered through email
  • Part identity differs between the ERP and the MES
  • The same few senior engineers specify what correct looks like
First

Supplier risk scoring

Built read-only from purchase order history that is already clean, it produces advice a buyer can take or ignore. The error shape is recoverable, the feedback arrives in weeks, and the exit cost is close to nothing. It also answers the question everything else is waiting on, which is whether the delivery record is complete enough to support any predictive work at all. 

Would move later if the purchase order history were not already clean, which makes it a data project before it is a scoring one.

Then

The identity project

Two later capabilities need it, it will not get cheaper, and the first move earned the credibility that makes an unglamorous foundation project fundable.

Would move first if quoting were the commitment with a date attached to it.

Alongside

Quality inspection

It needs a labeling effort and contends for a different pool of people.

Would move later if the labeling effort drew on the same engineers as the knowledge system.

Last

The knowledge system

Last by design: it consumes the scarcest input in the operation, and starting it early would have starved everything else while producing the least visible result.

Would move first if the retirements were closer than the schedule commitments.

Nothing in that order matches the ranking that a pure yes/no decision provides.

The ranking asked which capability was worth the most. The sequence accounted for the dynamic interplay between factors, the non-obvious dependencies, and other factors that a pure ranking wouldn't uncover.

Full walkthrough / Addendum A

10

When a simpler approach will do

10.1

Under the following three conditions, complex sequencing logic may be overkill.

Condition 01

When the operation is single-domain, the coupling is low, the move is reversible, and the binding constraint is obvious to everyone involved, the reasoning collapses to yes/no. 

Condition 02

In environments where incremental discovery is not permitted, the analysis has to be front-loaded.

Condition 03

In an organization with slack to spare, where the integration bench is deep and change capacity is nowhere near its ceiling, a ranked system may be adequate.

The method we built around this

FRAME: Future Readiness Assessment & Modernization Engineering

Talbot West built a method around the discovery process detailed in this article and named it FRAME. The methodology also adds instrumentation, a standard discovery test for each class of gate, and a set of eyes with no position in your internal argument.

One question to start with

Take whatever is currently at the top of your list and ask what it is waiting on.

Not what it is worth. What it is waiting on, named specifically enough that you could find out this quarter whether the answer is two weeks of work or two quarters. If nobody on your leadership team can answer, you have learned something more useful than a ranking.

And if the answer turns out to be the same two people for each of your top three, then the portfolio is one initiative with three names on it, and the order was always going to matter more than the ranking did.

Portrait of Jacob AndraJacob Andra
About the author

Jacob Andra is the CEO of Talbot West and host of The Applied AI Podcast. He writes and speaks on digital transformation, AI integration, and business process improvement. He developed FRAME, Talbot West’s assessment and sequencing methodology, and APEX, its method for prioritizing which AI initiative to build first. He is also co-developer of Cognitive Hive AI (CHAI), a modular, composable ensemble framework.

A – D

Deep dive addenda

Addendum A

Manufacturing scenario

The main body walks through this scenario and states the order it resolves to. Here is the derivation: what each candidate is waiting on, which gate class from Addendum C that falls under, and the test that settles it.

CandidateWhat it is waiting onGate class and testWhere it lands
Supplier risk scoring

Delivery performance, which is scattered through email. Purchase order history is already clean.

Data availability

Pull a sample export of delivery records covering one real month and count what is missing.

First. Read-only, recoverable, reports back in weeks, and it settles the record question the other four inherit.

Automated quoting

Part identity being consistent between the ERP and the MES, which it is not.

Entity resolution

Compare a small sample of part records from both systems and count the manual matches.

After the identity project. Nothing can be built while one planner is the reconciliation layer.

Demand forecasting

The same part identity, plus a current forecast error rate nobody has measured.

Entity resolution, then evaluation baseline

Ask for the present forecast error rate.

After the identity project, and behind quoting while the baseline is missing, since there would be no way to show it improved anything.

Quality inspection

Labels. The images exist; the labels do not.

Data availability

Ask who would label, and whether they are the senior process engineers.

Alongside the identity project, as long as labeling draws on a different pool of people.

The knowledge system

The senior process engineers, who specify what correct looks like and who every other candidate also needs.

SME availability

Name them, and check the overlap across the other four.

Last. It consumes the scarcest account in the building.

The graph resolves quickly once it is drawn. Part identity gates two of the five candidates. The senior engineers gate four. Neither one is visible on a value-versus-feasibility grid, and both determine the order.

Change one fact

The order above is not a template. It is what this constraint graph resolves to. A second manufacturer with the same five candidates and one fact different gets a different route.

The purchase order history is not clean

Supplier risk scoring stops being the cheap first move and becomes a data project. The identity work leads instead, without a quantified case behind it.

Quoting carries a dated commitment

The identity project goes first, and the read-only move that would have priced it follows rather than precedes it.

Labeling needs the senior engineers

Quality inspection leaves the parallel track and queues behind the knowledge system, because the two now draw on the same account.

The retirements land before the schedule commitments

The knowledge system goes first, and everything else absorbs the delay that decision creates.

Addendum B

Read-only vs write-back

Read-only

Produces advice

A human reads it, acts or ignores it, and the record of the business is unchanged. Blast radius is small, errors are recoverable, and comparing the system's recommendation against what the human did anyway supplies the evaluation baseline. It is the cheapest way to find out whether a capability's judgment is worth trusting.

Write-back

Changes the record

That requires rollback design, an approval topology that says which writes are automatic and which are held, an audit trail that survives an outside party reading the decision back years later, and a defined behavior for the case where the downstream system rejects the write halfway through a batch. None of that is exotic engineering. None of it appears on a value-versus-feasibility slide either, and together it routinely doubles the distance to production.

B.1

Read-only capabilities usually clear faster, build the evaluation baseline, and earn the standing that write-back will require. Ordering a write-back capability first commits the organization to rollback, approval, and audit design before anyone has evidence that the underlying judgment is sound.

Addendum C

Gate classes

Gate classWhat bindsHow you find outWhat clearing it costs
Data availability

The record was never captured, or exists only on paper

Ask for a sample export covering a real month

Ranges from weeks to a capture program

Entity resolution

Two systems disagree about what an entity is

Compare a small sample of records from both systems and count the manual matches

Usually a project of its own, not a task

Freshness and lineage

Data lands later than the decision needs it

Compare acceptable staleness against actual feed timing

Pipeline engineering, sized by the lag

Integration surface

A vendor controls the system of record

Read the access terms in the contract

Sometimes unbuyable this year

Identity and permissions

Cross-system output crosses permission boundaries

Ask who is allowed to see the output

Cheap if answered early, expensive at launch

Write path and rollback

Changing the record requires reversal and audit design

Ask what happens when a batch fails halfway

Multiples of the read-only equivalent

Evaluation baseline

Current human performance is unmeasured

Ask for the present error rate

Small, and skipping it costs the argument later

Decision rights

Nobody owns being wrong

Ask who signs

Free to discover, slow to resolve

Regulatory and contractual

Review sets a floor on the calendar

Involve counsel before design

Fixed, and not compressible by engineering

SME availability

Specification needs people the operation cannot spare

Name them and check the overlap across initiatives

The most common hidden coupling

Addendum D

Failure literature

95%

of organizations in the sample saw no measurable return on $30 to $40 billion of enterprise GenAI investment. The shortfall traced to approach rather than model quality.

MIT Project NANDA / 300+ initiatives analyzed
80%+

AI project failure rate, roughly twice the rate of non-AI IT projects. Interviews point to a misunderstanding of project purpose and context rather than to the technology.

RAND / Root-cause study RRA2680-1
D.1

Existing prioritization frameworks treat identification and prioritization as separate steps, and some call for portfolio views that expose readiness and resource conflicts alongside the ranking (Agility at Scale, Strategy of Things). A broader review of portfolio selection research reaches a similar conclusion from the academic side: interdependencies, shared resources, and information-assessment cost change the selection problem. None of these supply the decision rule connecting a ranking to a route.

See our article on failure modes in enterprise AI implementation
D.2

The three-budgets framing draws on theory of constraints (binding constraints set throughput; improving anything else produces local efficiency and a queue of unfinished work), on change-saturation research showing organizations have a measurable, finite capacity for simultaneous change (Prosci, IMA Worldwide), and on Monte Carlo simulations of shared-resource contention in multi-project portfolios (Ghaffari and Emsley).

The information-ordering argument draws on real-options reasoning for staged investment under uncertainty and on value-of-information analysis, which prices a piece of research by how much it improves a decision's expected payoff (ISPOR).

Sources

  1. 01MIT Project NANDA, The GenAI Divide: State of AI in Business 2025.
  2. 02RAND Corporation, The Root Causes of Failure for Artificial Intelligence Projects, RRA2680-1.
  3. 03Prosci, change saturation model.
  4. 04IMA Worldwide, change fatigue as a capacity problem.
  5. 05The Change Compass, change saturation and portfolio failure.
  6. 06Lean Production, theory of constraints overview; Velociteach, theory of constraints in project portfolio management.
  7. 07The Decision Lab, real options analysis; Balasubramanian, Kulatilaka, and Storck, managing information technology investments using a real-options approach; Bardhan, Bagchi, and Sougstad, a real-options approach for prioritization of a portfolio of information technology projects.
  8. 08Agility at Scale, AI use case identification and prioritization.
  9. 09Strategy of Things, AI project prioritization.
  10. 10Project portfolio selection considering interdependencies: a review of terminology and approaches.
  11. 11Ghaffari and Emsley, the boundary between good and bad multitasking in critical chain project management.
  12. 12ISPOR Value of Information Analysis Task Force, value of information analysis for research decisions.