The useful preparation
A customer-service assistant that saves staff time can still create a cost the operations manager did not budget for: a wrong promise, a leaked document or an action nobody approved. Before you extend a pilot across a Caribbean business, decide who checks its work and who can stop it.
“AI Apocalypse” in this guide means disruption to work and trust. The useful response is an operating plan. These five tips connect vendor checks to daily controls, so a team can show why a workflow is safe enough to try, what remains uncertain and when it must return to human handling.
Why these checks matter now
On 8 October 2026, the OWASP Gen AI Security Project published its Q3 2026 exploit roundup. It brings reported incidents together with the project’s security analysis. Those reports are a prompt to test your own controls, not evidence that your organisation has suffered the same failures.
The NIST AI Risk Management Framework offers a voluntary structure for governing, mapping, measuring and managing risk. The five tips below translate that work into everyday checks. They do not replace the specialist review a regulated or high-impact use case needs.
1. Buy evidence about your workflow
Write the business problem in a sentence your customer would understand. If your team needs to answer enquiries sooner, measure response time and incorrect answers together. A speed target on its own rewards quick mistakes.
Use Adrian Dunkley’s People First, Problem Second, AI Last approach: identify the people affected, define the problem, then choose the tool. In a vendor review, that order changes the demonstration. The supplier must work through your representative examples and explain the failures.
Ask for the model provider and version, hosting locations, subprocessors, data rights, retention settings, update policy and support route. Check the licence for any model or dataset your team will host. Request evidence on tasks and language relevant to the Caribbean; a broad leaderboard score is insufficient for a local service decision.
Before approval: Name the business owner and technical owner. Document the data flow. Test difficult cases. Record supplier answers and gaps. Set acceptance criteria and an exit plan before committing the workflow to the product.
2. Set a data boundary staff can apply
Translate “protect confidential information” into examples from the actual job. A reservations team should know whether it can use a public room description, a guest’s email, a passport scan or a payment record. Give staff a safe test dataset so they can learn without guessing.
Verify the contracted service and account settings. Check training use separately from storage, logging, support access and deletion. Confirm how uploaded files and connected systems are handled. Remove unneeded personal fields before data reaches the model; review the remaining context for indirect identifiers.
OWASP’s sensitive-information guidance supports limiting data access and carefully managing sources. Put those restrictions in the application and connected systems. A paragraph in a system prompt cannot substitute for permissions, an approved data agreement or staff training.
Before a pilot: Classify inputs. Approve the account. Restrict folders and users. Review retention and access. Use synthetic records in testing. Tell staff exactly where to ask when a case falls outside the agreed boundary.
3. Test for wrong answers before measuring speed
Build a small evaluation set from realistic, authorised examples. Include ambiguous requests, missing information, conflicting sources and a case the assistant must escalate. Write the expected behaviour before running the test. Otherwise a persuasive answer can move the goalposts.
Require reviewers to inspect the source behind important claims. Check numbers in an independent calculation. Record unsupported statements and omissions as errors, even when the response sounds useful. Test whether local names, currencies and date formats survive the workflow unchanged.
Give experienced staff time to review. This is an application of Cognitive Debt: if staff stop learning from the decisions they delegate, the organisation loses part of its ability to notice a mistake. Rotate examples and discuss corrections rather than rewarding acceptance of the first answer.
At the review: Count correct outputs, harmful errors and escalations. Include human checking time in the result. Retain the model version and test configuration. Keep claims, hiring, credit and other consequential decisions with qualified reviewers.
4. Limit what an agent can change
A pilot that only suggests a response has a different risk from one that updates a customer record or issues a refund. Draw that boundary in the architecture. Use a read-only service account where possible, restrict the records it can see and remove tools it does not need.
Place approval gates at the point of action. A reviewer should see what will be sent or changed, to whom and with what financial effect. Check authorisation in the downstream system. Do not let generated text determine whether the agent has permission to act.
OWASP’s excessive-agency guidance identifies unnecessary functionality, permissions and autonomy as sources of harm. A retrieved document or inbound message can contain malicious instructions. Test that those instructions cannot expand the agent’s access or silently skip the reviewer.
At deployment: Separate draft and execute steps. Limit permissions. Require human approval for high-impact actions. Set transaction and usage limits where appropriate. Test rejected actions and repeated requests as well as the successful path.
5. Give the pilot an owner and a stop rule
Put the workflow on a register with an owner, a review date and a description of its permitted use. Log errors and near misses with enough detail to investigate, using access-controlled storage. Monitor changes to the provider, model, retrieval sources and permissions.
Agree the stop conditions before the launch. An unauthorised disclosure or action should pause the affected workflow while the incident owner investigates. Keep a manual service route and make sure staff know how to use it when the tool is unavailable.
NIST’s Generative AI Profile provides a reference for managing risks through the lifecycle. After a failure, preserve evidence, contain the problem, assess who was affected and let the responsible team decide on notifications. Retest the correction before restoring access. A supplier saying the model has improved is not your acceptance test.
Each review: Inspect error trends and overrides. Check whether staff are working around controls. Retest after material changes. Confirm that the stop mechanism and fallback still work. Record who authorised any expansion.
A Saint Kitts and Nevis example: test the handoff
Use a worked example of a tour business coordinating a Basseterre departure with a Charlestown partner in Saint Kitts and Nevis. An assistant may draft a response covering the visitor’s request and the operator’s approved itinerary. The person responsible for the booking still checks which island, port and date the customer means.
This two-island handoff makes a useful test case. Include a request that confuses Saint Kitts with Nevis, a price with no currency label and a sailing time copied from an old message. The correct response may be to ask a question or refer the case to staff. Rewarding a complete-looking itinerary would encourage the assistant to fill the gaps.
These are synthetic scenarios, not claims about a particular operator or the current transport timetable. Their purpose is to test whether a Caribbean workflow preserves local facts when it moves between organisations. Our inference is that the general research supports testing these boundaries; it does not predict this business’s savings or establish that its chosen system is reliable.
What the evidence can tell you
The November 2023 version of Generative AI at Work studied 5,179 support agents. It reported 14% more issues resolved per hour on average with an assistant. That is a result for the studied setting, not a forecast for your team.
A field experiment with 758 consultants found 12.2% more completed tasks and 25.1% faster completion on tasks within the tested AI’s capability. The same paper reports worse performance on a task outside that capability. Its GPT-4 experiment does not validate today’s product on your workflow.
The ILO’s 2025 occupational-exposure update estimates that one in four workers worldwide (25%) is in an occupation with some exposure to generative AI. Exposure describes potential task change. It is not a count of people who have lost their jobs.
Stanford’s 2025 AI Index recorded 233 reported AI incidents in 2024, up 56.4% on 2023 in its cited database. Reporting patterns and coverage affect that count; it cannot tell you the probability of an incident in your business.
A practical test for this week
This is a suggested exercise, not a claim about likely savings. Choose a low-risk task with public or synthetic information. Do not use the trial to make decisions about real customers or employees.
- Write the task, owner, permitted data and stop conditions on one page.
- Measure a manual baseline on 10 representative examples. Record time, correctness and missing information.
- Test the same examples with AI and human review within seven days. Include checking and correction time, then compare both quality and total time with the baseline.
- Review every error with the owner before expanding use. Pause if the trial exposes information or acts beyond its permission. Keep a record of what must change.
Ten examples can reveal a problem; they cannot establish a reliable error rate or certify safety. Higher-impact work needs a larger, more varied evaluation and specialist review before deployment.
Where the checklist can still fail
A careful reviewer can miss an error, especially when tired or unfamiliar with the subject. A supplier can change a model or a setting. A small test can overlook a rare case that causes serious harm. The people bearing that harm may be customers whose information is exposed, staff judged on an incorrect record or a business that makes a promise it cannot fulfil.
Match the control to the consequence. Keep sensitive decisions with qualified people, test access boundaries and give affected people a way to challenge an outcome. The weakest part of this advice is the local evidence: the cited studies do not measure these example Caribbean workflows. Use them to justify careful evaluation, then collect your own evidence. Training and written rules need enforcement and time for review.
Questions to settle before you start
How do we compare two vendors fairly?
Use the same authorised examples, expected answers and scoring rules for both vendors. Keep the model version, configuration and review time in the record. Include failures and escalation behaviour in the comparison. A supplier should explain missing evidence rather than substituting an unrelated benchmark.
What if a model update arrives during the pilot?
Record the change and rerun the agreed evaluation set before expanding use. Where the service allows it, control the version used for a consequential workflow. If you cannot pin a version, ask the supplier how changes are announced and include that uncertainty in the approval decision.
Who owns a failure involving several suppliers?
Name an internal incident owner before deployment. Keep a map of the application, model, data and integration suppliers, with support contacts and contractual responsibilities. Your team still needs a route to pause the service and help affected people while the suppliers investigate their respective components.
Does the Survival Guide course award certification points?
Ask StarApple AI to confirm any assessment or certification arrangements for this specific course before booking. The course enquiry page does not promise points, accreditation or a certificate. Existing certification pathways have their own details and should be reviewed separately from this practical risk-management course.
Put the checks into practice
Survival Guide to the AI Apocalypse: A Practical Guide to AI Risk Management
Discuss the course with StarApple AI and work through the controls for your context. Confirm format, availability, fees and any assessment requirements before booking.
View the courseRead Survival Guide to the AI Apocalypse by Adrian Dunkley for the book’s current ordering options.