AI Officer field notes
Five jobs the elves gave AI
Two mugs are packed for dispatch. The address is blank. What would your agent do with order 18?
StarApple AI Editorial · 21 November 2026

Give the workshop five jobs
The elves in our holiday workshop have a modest assignment for their assistant. Sort the orders by dispatch date. Flag any blank address. Draft the packing list. Compare what was ordered with what was packed. Prepare a pickup notice for the shift lead. Each job has a distinct output a colleague can inspect.
Build a practice sheet with order IDs, dispatch dates, the ordered and packed quantities and a flag showing whether an address exists. You can run the entire exercise using made-up records. The notice stays in the draft queue until the shift lead checks the parcel.
Put a bad order in a good-looking queue
Order 18 contains two mugs. Both are packed. Dispatch is due on 4 December. Leave the address blank. Before you run the assistant, write down the expected response: the packing list should show two mugs, the address check should fail and dispatch should go on hold. A matching count clears one check. It does not supply an address.
Give the assistant a second order with an address and a missing mug. Watch whether it can tell the difference between a contact problem and a packing problem. A vague warning leaves the shift lead searching for the fault.
Run the same order again
In the 2024 τ-bench study, Shunyu Yao and colleagues tested agents on simulated customer-service tasks. In their retail setting, the reported measure for completing a task correctly on all eight attempts was below 25%. The benchmark describes those agents and tasks. It gives a workshop manager a reason to test repeatability on the actual job.
Decide who can release the parcel
Run fresh copies of order 18. Record whether each attempt caught the blank field, what notice the agent drafted and whether it tried to dispatch. Keep the release decision with the shift lead. If one attempt misses the address, revise the process and run the same cases again before giving the assistant more authority.
Try it yourself
Using the supplied order ID, dispatch date, packed count, ordered count and address status, prepare the packing list. Flag discrepancies and missing fields. Place incomplete orders on hold. Draft notices for a person to review.