Staffing Coverage Under Uncertainty
A forecasting model for anesthesia coverage at a large pediatric health system
- Discipline
- Data product design
- Domains
- Healthcare · AI and Automation
- Timeframe
- 2025 to 2026
- Status
- Approved in discovery. Delivery moved to an internal data science team.
What was this project?
Staffing Coverage Under Uncertainty is a data product design for anesthesia operations at a large pediatric health system, where every operating room, procedure room, and imaging suite that needs anesthesia has to have a clinician assigned before the day can run.
Anesthesia operations at a hospital of this size is a daily coverage problem with a hard floor. A location without an assigned anesthesiologist or anesthetist does not run at reduced capacity. It does not run. Cases move, families wait, and the surgical schedule absorbs the difference.
The department ran its scheduling on a commercial platform and locked each day's schedule ninety days ahead. The work of building those schedules was manual, and it was done well. That is the part most people get wrong about this problem: there was no failure to point at.
What problem did this solve?
Anesthesia coverage was planned on a rule that treats one clinician as exactly one staffed location, so a single unplanned absence closed a room and there was no way to see that risk before the day arrived.
The rule is a reasonable simplification and it is what made the manual process tractable. It is also a point estimate. It carries no information about how often people call out, which locations are hardest to backfill, or what a given staffing level costs when demand comes in above or below plan.
Everyone downstream felt that. Operations rebuilt coverage by hand when someone was out. Perioperative services found out late. The section leadership wanted to advance its analytics but had no quantitative basis for a staffing conversation, only the schedule itself and the judgment of the people who built it.
The department was not asking for automation. It was asking whether its assumptions held.
What did I observe?
The manual scheduling process was not broken. It was accurate, and its accuracy depended on a small number of experienced people holding the whole picture in their heads, which is a different kind of risk than an inaccurate process.
Three things stood out once I mapped the existing workflow with the operations manager and the analytics sponsor.
- The ninety day schedule lock was a real constraint and also a real asset. It meant a forecast had a fixed horizon to be measured against, and it meant a model could be evaluated against months of history without touching a live schedule.
- A semi automated coverage report already existed and was maintained by the analytics sponsor. The ingestion problem was smaller than it looked. What was missing was a curated dataset with agreed definitions, not a new data source.
- There was no location inventory anyone could point to as canonical, and no agreed flag for whether a location was operational on a given day. Coverage percentages cannot be computed without that denominator, so it had to be built before any metric would mean anything.
The most useful thing I learned was what the department did not want. Automating the schedule was explicitly not the goal, and a proposal that led with automation would have been rejected on arrival by the people whose judgment currently made the process work.
What was my role?
I was the product manager on this work and personally owned the problem definition, the workflow analysis, the product design, and the written proposal, up to the point where it was approved and handed to a delivery team.
- Interviewed anesthesia operations and the clinical analytics sponsor, and mapped the existing manual scheduling workflow end to end.
- Identified the coverage rule as a point estimate and framed the problem as decision making under uncertainty rather than as a scheduling automation problem.
- Evaluated candidate approaches with the data science group and selected the Newsvendor formulation.
- Designed the data product: source ingestion, the curated coverage dataset, the fact and dimension model, the metric definitions and their calculations, and the stakeholder reporting layer.
- Wrote the metric definitions, including the coverage denominator problem and the role equivalence question that had to be answered before any of them could be computed.
- Wrote the phased delivery plan, the risk register, and the open questions, and secured stakeholder approval.
I did not build the model, write the pipeline, or run the analysis. The statistical design was to be done with a principal data scientist, and implementation belonged to the data engineering and data science teams.
What was the proposed solution?
The proposed system is a curated coverage dataset feeding a Newsvendor optimization model that recommends a staffing level for each date, shift, and location, and reports the expected cost of being wrong in each direction rather than a single number.
The Newsvendor model is a standard operations research formulation for committing to a quantity before demand is known, when being short and being long both cost something and the two costs are not equal. Anesthesia staffing fits it closely. Understaffing closes a location. Overstaffing spends clinician time that had alternative uses. The schedule locks ninety days out, which is the commitment point.
The output is deliberately not a schedule. It is a recommended level, a distribution around it, and a comparison against what the manual process planned, so the two can be examined side by side.
Four components:
- Ingestion of scheduling exports and the existing semi automated coverage report into a curated dataset with agreed fields for clinician, role, shift, date, and location.
- A coverage fact table joined to a location inventory carrying an operational flag, which is what makes a coverage rate computable at all.
- The Newsvendor model, with documented assumptions, back tested against historical periods and reported with sensitivity to the demand and absence inputs.
- A reporting layer for operations and section leadership with coverage rate, shortfall count, absence impact, and the gap between the model recommendation and the manual plan.
The manual process keeps running throughout. The model advises, it is measured against the humans, and nobody's schedule changes because a model said so.
How was the system designed?
The architecture runs scheduling exports into a curated coverage dataset, feeds that dataset to the optimization model, and returns recommendations to a reporting layer, with the existing manual process running in parallel the whole way.
The parallel track in the diagram is the important part. The model never writes to the schedule. It produces a recommendation that sits next to the human plan, and the difference between them is itself one of the reported metrics.
The ninety day lock acts as the evaluation gate. Once a schedule locks, the recommendation for that period is frozen too, and both can be scored against what actually happened.
Four metrics were specified with their calculations, because a metric without a written denominator becomes three different numbers in three different meetings:
- Coverage match rate: locations with an assigned clinician divided by total operational locations for a date and shift, expressed as a percentage, excluding locations flagged non operational.
- Coverage shortfall count: total operational locations minus covered locations.
- Absence impact: the locations affected when a given clinician is absent, which requires an agreed reallocation rule to compute and did not have one.
- Recommendation versus actual: model recommended headcount minus planned headcount, signed, so over and under are distinguishable.
Why was it designed this way?
The design is built to earn trust from a team whose current process already works, which ruled out every approach that would have required them to stop doing what they were doing in order to find out whether the new thing was better.
| Choice | Alternative considered | Why the choice won |
|---|---|---|
| Newsvendor formulation | Multi objective linear programming across the full schedule | Newsvendor is explainable to operations in one sentence and its assumptions can be argued about directly. The linear program is more powerful and its recommendations are harder to challenge, which is a liability when the goal is to test assumptions rather than replace judgment. |
| Analytics first, automation deferred | Building toward scheduling automation from the start | The manual process was validated as effective. Automating it first and measuring second would have removed the baseline needed to know whether the model was any good. |
| Model runs alongside the human plan | Model output feeds the schedule directly | Keeps a human accountable for every staffing decision, and makes the recommendation versus actual gap an observable metric instead of an invisible substitution. |
| Reuse the existing coverage report | New direct integration with the scheduling platform | The report already existed and was trusted. A new integration adds an access dependency and a schema drift surface for no gain in the first phase. |
On risk: this product handles staffing data rather than clinical data, so no protected health information was expected in scope. That expectation had to be verified rather than assumed, and confirming it was a named open question in the proposal, not a footnote.
The assumption most likely to be wrong was the role equivalence one. Treating a physician and an anesthetist as interchangeable units of coverage is what makes the one clinician equals one location rule computable, and if it does not hold uniformly across locations then every metric built on it inherits the error. I wrote it up as an open question rather than resolving it myself, because operations and clinical leadership owned that answer.
How would it be implemented?
The implementation plan is five phases that each produce something usable on their own, ordered so that the cheapest way to be proven wrong comes first.
- Access and definitions. Provision read access to the scheduling exports, confirm export fields and refresh cadence, and settle the location inventory and the operational flag. Nothing downstream is computable until the denominator exists.
- Curated dataset. Normalize clinician, role, shift, date, and location into a coverage fact joined to location, shift, and calendar dimensions. Add completeness and duplicate checks and reconcile against the existing report.
- Baseline analytics. Publish the four core metrics against history and establish what the manual process actually achieves. This phase has value even if the model is never built.
- Model design and back test. Build the Newsvendor model with a principal data scientist, back test against historical locked periods, and report sensitivity to the demand and absence assumptions alongside the point recommendation.
- Stakeholder review and scenario tooling. Walk operations through the outputs, add what if scenarios for absence and case volume changes, and decide from evidence whether an automation phase is warranted.
The gate between phase three and phase four is deliberate. If the baseline shows the manual process is already at or near the achievable coverage rate, the correct decision is to stop, and the plan says so.
What impact was expected?
Every figure below is a projection from the design, not a measured result, because the system described here was not built and no outcome was ever observed.
| What would change | Baseline today | How it would be measured |
|---|---|---|
| Coverage risk becomes visible before the schedule locks | No probabilistic view exists. Risk surfaces on the day, as an absence. | Shortfall probability per date and shift at lock time, scored against realized shortfalls after the fact. |
| Staffing assumptions become testable | The one clinician equals one location rule is unexamined. | Model recommended headcount minus planned headcount, tracked over time and reviewed with operations. |
| Scheduling knowledge stops living only in people | Accuracy depends on a small number of experienced schedulers. | Documented metric definitions, a data dictionary, and a reproducible pipeline. |
| Scenario questions get answered in minutes | What if questions require manual rebuilding of the schedule. | Time to answer a stated absence or volume scenario. |
The success criterion I wrote was that model recommendations should match or beat the current manual coverage match rate. That framing matters. It sets the existing human process as the bar rather than assuming the model improves on it, and it makes a negative result a legitimate and publishable outcome.
I did not put an hours saved number in the proposal. The honest answer was that nobody knew what the manual process cost in hours, and phase one was the thing that would have found out.
What happened?
The approach was approved by clinical operations and the analytics sponsor during discovery, and the work moved to internal data science and data engineering teams for delivery.
Three decisions were agreed in discovery: use the Newsvendor formulation for the initial analytics and optimization approach, retain the manual process because it was performing well, and secure statistician or optimization expertise to support the modeling.
I designed and specified it. I did not build it, and I have no measured outcome to report. The open questions I documented, particularly the location inventory and the role equivalence rule, were still open when the design left my hands.