Start with the decision, not the method
A simple threshold is better when it gives people sufficiently useful guidance with less work to explain, operate and maintain. Machine learning can justify its extra cost through better decisions even when a rule is possible. Neither deserves deployment when the evidence is unreliable or nobody can respond. Compare them against the same operating need, not against each other's branding.
Consider a hypothetical carton buffer serving manual packing benches. The question is whether to prompt an operator to check and replenish usable, correct-format empty cartons. This is a low-risk advisory task, not machine control, product acceptance or a safety function. Existing approved checks and replenishment procedures remain in place.
- Decision
- Request an operator's attention to the carton buffer now.
- Useful result
- Attention arrives when replenishment is warranted and possible.
- Excluded result
- No automatic order, machine movement or permission to bypass checks.
The operations supervisor opens decision record MC-11 in the operations improvement register, the system of record for this comparison. It names the buffer, current carton format, operating modes, operator response and method versions. Unresolved observations remain unresolved rather than becoming evidence that the stock was adequate.
Give a direct rule a fair starting point
A threshold compares a known value with an agreed boundary. For this invented worksheet, the first candidate prompts when the usable carton count is fewer than 12. Exactly 12 does not prompt. These numbers are comparison inputs, not recommended stock limits or actual production settings.
The rule is easy to explain: show the count, show the boundary and show the result. That helps an operator question a bad count or an unsuitable boundary. It does not make the rule correct. A transparent instruction can still produce unnecessary attention during startup or miss demand when more benches are active.
- Confirm the count describes usable cartons of the current format.
- Compare that count with the hypothetical boundary of 12.
- Show the advisory result and the evidence used.
Missing or stale counts do not become zero cartons or an all-clear. The comparison pauses for an operator check under the existing process. Otherwise a data problem could look like a method's success or failure.
Google's Rules of Machine Learning supports starting without machine learning when appropriate and cautions that complicated heuristics can become difficult to maintain. The practical choice is not always a small rule versus a large model.
Check whether context needs a calculation
More than one input does not require learning. Suppose active packing benches matter. A deterministic calculation follows an explicit formula rather than learning a relationship from examples. Here the second candidate uses cartons per active bench during steady production. It prompts below an invented eight cartons per bench. Startup remains under the existing startup check, without an ordinary production advisory.
| Case | Count rule | Mode-aware calculation | Reference expectation |
|---|---|---|---|
| Startup: 6 cartons, 1 bench | Prompt: 6 is below 12 | No ordinary production advisory | Use the startup check |
| Steady: 12 cartons, 1 bench | No prompt | 12 / 1 = 12; no prompt | No prompt |
| Steady: 12 cartons, 2 benches | No prompt | 12 / 2 = 6; prompt | Prompt |
The fixed rule disagrees with the assumed reference in startup and in the two-bench case. The calculation matches these scripted expectations, but three selected cases do not establish operational performance. The supervisor must check whether those expectations and the bench-count relationship fit the real task.
This is a way to expose an assumption, not a fair contest already won by the calculation. Its boundary was chosen for this worksheet. In a real comparison, record why each candidate is a credible option and review its weak cases, rather than giving one candidate carefully selected favorable inputs.
Zero active benches, unknown mode and a changing carton format require explicit handling before this calculation is eligible. Dividing by zero or guessing a mode is not a useful alternative to machine learning.
Ask what a learned pattern would improve
A learned model could use recent consumption and operating context to advise attention. Its relationship comes from data rather than only from a written formula. That may be useful if the simpler candidates repeatedly miss meaningful situations. A model can also earn its place by reducing unnecessary prompts without losing needed ones, even where a rule could represent the decision.
| Candidate | Claim to examine | Evidence still needed |
|---|---|---|
| Direct threshold | Count alone is sufficient | Reviewed attention outcomes across relevant modes |
| Mode-aware calculation | Known context explains the useful difference | Reliable mode and bench counts; checked relationship |
| Learned pattern | Additional useful decisions justify additional support | Comparable outcomes and an affordable operating burden |
There are no model outputs in this worksheet, so its performance is unknown. It cannot be declared a winner or loser. A more sophisticated score, attractive chart or larger input list is not evidence of better replenishment decisions. Compare the same advisory at the same decision point, not a current-stock rule against an unrelated forecast.
Review attention, not a headline accuracy score
Before comparing candidates, agree what warranted attention means and who can judge it. For the buffer, the supervisor reviews operator notes, observed demand and the opportunity to replenish. A prompt that caused replenishment may prevent a shortage; the absence of a later shortage does not automatically make that prompt unnecessary.
- Needed attention, no prompt
- Record a possible miss and its operational consequence.
- No attention needed, prompt shown
- Record unnecessary attention and the work it caused.
- Needed attention, prompt shown
- Check whether the prompt was useful and actionable.
- Reference outcome uncertain
- Keep unknown; obtain review rather than force a verdict.
Keep the reviewed baseline's outcome record, not just the new candidate's attractive examples. The official scikit-learn DummyClassifier documentation describes an input-ignoring baseline for comparison with more complex classifiers. That supports comparison with a reference, not deploying that classifier on this buffer.
Here the meaningful operational reference is the best eligible reviewed rule or calculation, alongside the current manual process. MC-11 links each decision, inputs, advisory, operator action and reviewed outcome. For a hypothetical steady-production observation CB11-003, the record would retain the count of 12, two active benches, the calculation's prompt, the operator's replenishment action and the supervisor's later finding. Until that finding exists, the outcome remains pending. Useful improvement must concern the chosen task; a high overall score could conceal the missed attention that matters most.
Agree beforehand which missed requests are unacceptable and how much unnecessary checking the team can absorb. Keep those judgments consistent across candidates. If a model exchanges fewer interruptions for more missed replenishment, make that tradeoff explicit for the owner instead of hiding it inside one score.
Include the work needed to keep it useful
Method cost includes more than building it. Someone must keep carton identity, counts and operating context usable, explain disputed advisories and maintain the implementation. A sprawling rule book may be harder to support than a modest model. Conversely, a model that barely changes decisions may not justify its additional dependencies.
Ask for a support estimate covering routine review, disputed cases and recovery from unavailable inputs. Use the same accounting boundary for every candidate. A rule is not free because it has no model service, and a model is not valuable merely because its development cost is already spent.
| Method | Support question | If unavailable |
|---|---|---|
| Threshold | Who maintains the count and approved boundary? | Use existing manual checks, not an invented count. |
| Calculation | Who owns mode definitions and bench configuration? | Do not guess missing context. |
| Model | Who supports inputs, model version and service? | Show unavailable; retain the approved manual path. |
A model outage is not evidence that stock is adequate. A fallback rule is acceptable only if separately approved for the current inputs and mode, not simply because it is available. The comparison should also consider whether operators can understand and challenge the explanation. Nobody responding to either method means neither solves this task.
Choose the least burdensome adequate method
Use MC-11 to record the choice, evidence, unresolved questions and support owner. The operations supervisor decides whether a proposed advisory merits operational approval through the site's existing process. Technical staff supply method limitations, not permission to change operating settings.
| Finding | Next decision |
|---|---|
| Direct rule meets the need | Prefer it unless added complexity earns its cost. |
| Known context explains the gap | Review a deterministic calculation before adding learning. |
| Model adds useful, measured benefit | Weigh that benefit against support and unavailable-method behavior. |
| Evidence or response is inadequate | Select neither; resolve the gap using the approved manual process. |
Record the modes and carton formats covered, the comparison date and what would reopen the decision. A changed packing task or an unresolved pattern of disputed prompts warrants review, not an automatic complexity upgrade. The business value is useful attention at a supportable cost. A threshold, calculation or model must earn that value on the same terms.