Buyer guide · Practical decisions

Does an AI-written reminder help customers keep appointments?

A workshop rewrites its appointment reminder with AI. The new message asks customers to reply YES, and the inbox fills with confirmations. On Monday morning the service manager still has two empty bays. To decide whether to keep the wording, she needs the booking diary as well as the inbox.

Published Sources checked

Read the guide ↓
01 /

Choose what the reminder is meant to improve

Ask the person who runs the diary what they would do with a better reminder. A workshop that orders parts for each visit may care most about customers attending their original slot. Another workshop may be equally happy with an early cancellation if it can offer the bay to someone else. Those goals lead to different measures.

For the first workshop, write down the main measure as the number of customers who attend their original appointment divided by the number of eligible bookings assigned to that message. Record early cancellations, moves, missed visits and calls to reception separately. Agree what counts as early enough to refill a slot. Use the same rules in both groups.

GOV.UK’s guidance on A/B testing recommends choosing a primary measure and planning the sample size. For a small desk, start by calculating how many eligible appointments it actually handles each week. A difference of one extra attendance per hundred bookings may matter financially yet be difficult to distinguish from chance in a short trial.

02 /

Test the actual words customers will receive

Ask AI for a draft, then get reception to read it as if they had an appointment tomorrow. Can they tell where to go, what time to arrive and how to rearrange? Check that any promise in the draft is something staff can deliver. If customers can reply YES, someone must know what happens to that reply.

Approve one version and keep it fixed for the trial. Fill the date, time and location from the same booking fields used by the existing reminder. Send both versions through the same channel with the same lead time. If you change the wording and send the new version a day earlier, you can only judge the combined change.

Comparing two fixed messages tells you whether to adopt this piece of copy. If the winning text can be saved as a template, it can be reused without paying a model to rewrite it for every booking. Generating a different reminder for each customer would be a separate purchasing decision and a different test.

03 /

Assign the version before anyone replies

For a simple pilot, include only the first eligible booking per customer during the trial. Randomly assign each to the current or revised message and record the assignment against the booking ID before sending. Run both versions over the same weeks. Giving one version to regulars and the other to new customers would mix the effect of the message with differences between customers.

Keep failed sends in the assigned group and record the failure separately. If the desk drops failed sends or only exports people who replied, it loses part of the group it set out to measure. Reconcile assignment totals with the booking diary and check that each appointment has had time to take place before classifying it.

An even split need not produce exactly equal groups. Microsoft Research describes a larger-than-expected imbalance as a warning that assignment or reporting may be faulty. Investigate unexplained missing bookings before interpreting a difference in attendance.

04 /

Illustrative example — 200 workshop appointments

Suppose 200 customers with one booking each are assigned to the trial. All their appointment dates have passed, and reception has reconciled every booking. These figures are fictional. A customer can reply, ring reception and attend, so the rows below overlap.

The revised message gets 28 more replies per 100 bookings. Attendance moves from 84% to 85%, an observed difference of one percentage point. Reception also takes six more clarification calls per 100 bookings. A reply such as “Yes, but can I bring it later?” counts as engagement while creating work for the desk.

The extra attendance in this small example gives the manager little basis for predicting a lasting gain. Read a sample of the clarification calls. If people think they will lose their slot unless they reply, revise that line before testing another version. Microsoft’s experimentation guidance describes this use of secondary measures to spot unwanted effects alongside the main result.

Outcomes per 100 bookings assigned to each message
OutcomeCurrent reminderAI-written reminder
Replied to the reminder3058
Attended the original appointment8485
Called reception for clarification410
05 /

Make the rollout decision with reception

Set the review date before launch and allow the final bookings to mature. Review serious problems as soon as they appear. A reminder that sends someone to the wrong site needs fixing straight away. For ordinary day-to-day variation, keep the agreed comparison period so an unusually good morning cannot decide the result.

At the review, put attendance in both groups, the uncertainty around the difference and the extra work for reception on one sheet. Compare the range of plausible effects with the smallest improvement worth acting on. If the range includes both a worthwhile gain and a loss, record the test as inconclusive. For a low-volume workshop, a few observed customer sessions may be a quicker way to find confusing wording before investing in a longer trial.

Practical answers

Before you take the next step.

Should we measure only customers who opened or replied to the reminder?

Use all assigned bookings for the main comparison. Looking at replies can help explain what happened, but selecting respondents after sending changes who is being compared.

Work directly with Matthew

Discuss an appointment workflow.

Bring the current reminder, the booking outcome and the question reception hears most often.

Explore AI automation ↗Discuss your project with me ↗Prefer to write? Send a message. No sensitive records needed for an initial enquiry.