Write down the decision the forecast can change
Give the trial a specific deadline, unit and action. For the hire desk, the deadline is Thursday at noon. The unit is the number of collections expected between opening and noon on Saturday. The action is to book one extra colleague for that shift. A forecast for the whole weekend could be accurate while putting the extra pair of hands on the wrong day.
Agree how to count demand. If a long queue causes people to leave, completed collections alone hide some of the load. If dispatch records can only show completed collections, label the trial accordingly. Ask staff to note queues and abandoned visits during the pilot so the manager can see where the count gives an incomplete picture.
Give last week’s figures a place in the trial
Start with a rule the desk can run in a spreadsheet. Use last Saturday’s morning count as the forecast for this Saturday. This is a weekly version of the seasonal naïve method described by forecasting researchers Rob Hyndman and George Athanasopoulos. It costs little to reproduce and tells the buyer what extra value the proposed model has to deliver.
Record the manager’s current plan as well. An experienced supervisor may already take a bank holiday, a local fair or a large reservation into account. Beating last week’s number is interesting. Improving on the decision the supervisor would actually make is the stronger reason to change the rota.
Recreate Thursday lunchtime before revealing Saturday
For each past Thursday, rebuild the information available by noon. Bookings placed on Friday belong outside that Thursday’s forecast. If the model uses weather, supply the forecast issued by the cut-off. Putting Saturday’s observed rainfall into an old test gives the model information the manager could never have used.
Work forward through the calendar. Fit the model on earlier weeks, forecast the next shift, record the prediction, then move the cut-off forward. Scikit-learn’s guidance on time series splits explains why mixing future and earlier observations in a random split can give a misleading test. Date-ordering the rows still leaves a problem if their values were amended later. Check when an amendment became visible.
If the booking system overwrites old values and you cannot reconstruct the Thursday order book, collect a fresh snapshot at the next cut-off. Save both forecasts and the supervisor’s planned cover before Saturday arrives. That makes a slower trial, but every comparison can be repeated.
Illustrative example — four Saturday mornings
Suppose the desk books extra cover when it expects at least 20 collections. The figures below are fictional. Each row is one trial Saturday. The baseline uses the count from the preceding Saturday, so the trial dates need not be consecutive.
The baseline misses by 4, 3, 4 and 3 collections. Its mean absolute error is 3.5 collections per shift. The AI misses by 1, 1, 3 and 1, giving 1.5. This measure averages the size of each miss and gives equal weight to forecasting too high and too low.
Look again at shift C. A prediction of 19 looks close to the actual 22, yet it leaves the desk short of cover under the agreed rule. Across the four shifts, the AI removes one unnecessary extra shift and catches one busy morning the baseline missed. Both methods still miss C. That remaining miss belongs in the buying decision.
| Shift | Last Saturday | AI forecast | Actual collections |
|---|---|---|---|
| A | 18 | 21 | 22 |
| B | 21 | 17 | 18 |
| C | 18 | 19 | 22 |
| D | 24 | 22 | 21 |
Decide what a better rota is worth
Ask the supervisor what happened on the shifts where the two methods disagreed. On a quiet morning, an extra colleague might still clear a backlog of returns. On a busy one, a missed shift could mean a queue for the first half-hour, or customers walking out throughout the morning. Use observed staffing costs and service records where available. Keep any assumed cost of a queue visible as an estimate.
Four shifts illustrate the arithmetic. They give too little evidence to set a permanent staffing rule. Run the forecasts alongside the usual rota across a mix of ordinary and busy periods. Keep a final stretch of shifts aside for the decision, especially if you change the threshold after looking at early results.
At the next rota meeting, bring a single sheet showing each forecast, the cover it suggested, actual demand and the supervisor’s reason for any override. Agree which improvement would justify the running cost and who can fall back to the existing plan if the forecast arrives late. The trial can then end with a concrete choice about the next rota.