The 7 findings in brief
Most urgent first. Each is set out in full further down, with its evidence, risk and fix.
Each respondent named the other as the agent's owner.30 days2Production changes are unreviewed
Eleven versions by three admins, none reviewed before going live.30 days3Visit actions are not tied to the verified customer
A verified customer may be able to see, book or cancel another customer's visit.30 days4An instruction tells the agent to do something it has no action for
The agent is told to send an alert email it has no way to send.30 days5No stop procedure
Nobody could describe how to switch the agent off.This quarter6Success is not defined
Three people measure the agent three different ways.This quarter7The agent answers from an outdated policy article
The agent still answers from the 2025 coverage guide.This quarter
What we looked at
Two questionnaires (service operations lead, Salesforce admin) and one configuration export. No customer records. Built by: Implementation partner, engagement ended at go-live.
Who holds each right over the agent
Six rights decide how an agent is run. Each should have one named holder.
What the agent can do, and for whom
Every subagent and its actions, from the export. An action that changes something should be tied to the customer who proved who they are.
Customer Verification
- Send Verification Code
- Verify Code
Claim Status
- Get Claims for CustomerTied
- Get Claim by NumberTied
Claim Updates
- Add Claim NoteChanges dataTied
- Update Contact DetailsChanges dataTied
Technician Visits
- Identify Customer by EmailNot tied
- Get Visit SlotsNot tied
- Book VisitChanges dataNot tied
- Cancel VisitChanges dataNot tied
Coverage Questions
- Answer from Knowledge
Escalation
- Hand Off to a Person
- Create CaseChanges data
- Add Case CommentChanges data
11 versions, 4 authors, no review recorded
Each dot is a committed version, placed by date and colored by who made it. The ringed version is the one customers are talking to; a boxed one is named in a finding.
7 findings, most urgent first
Each finding says where its evidence came from: the export, the questionnaire, or both.
No accountable owner
- Evidence
- Each respondent was asked who owns the agent. The service lead named the admin; the admin named the service lead.
- Risk
- Problems are noticed by customers before staff. Decisions about scope default to whoever edits the agent last.
- Fix
- Name the service operations lead as owner, in writing, with the rights in the ownership map.
Production changes are unreviewed
- Evidence
- The export holds eleven committed versions since go-live, made by three admins. Comparing them shows exactly what changed in each. Salesforce's Setup Audit Trail does not record these edits, and the questionnaire confirms no one reviews a change before it goes live.
- Risk
- A well-meant wording change alters how the agent answers every customer, with no test and no way to tell which edit caused a problem.
- Fix
- Reduce edit rights to one admin. Require a sandbox test and the owner's sign-off for any change, and record each one in the decision record.
Visit actions are not tied to the verified customer
- Evidence
- Claim Status and Claim Updates pass the verified customer's ID into every action. Technician Visits does not: it looks up a customer from whatever email address is typed, and Cancel Visit accepts a visit number alone. The only safeguard is an instruction telling the agent not to switch customers.
- Risk
- A customer who has verified their own identity may be able to see, book or cancel another customer's technician visit. An instruction is a request to the model, not an enforced check.
- Fix
- Pass the verified customer's ID into the visit actions, as the claim actions already do. Until then, test it: verify as one customer and ask about another's visit.
An instruction tells the agent to do something it has no action for
- Evidence
- Version 7 added this to Technician Visits: "deny cancellation and then send an alert email to the claim owner." The subagent has no action that sends email. The line is still in the live version.
- Risk
- The agent either skips the step silently or tells the customer an alert was sent when none was.
- Fix
- Add an email action and test it, or remove the instruction.
No stop procedure
- Evidence
- Neither respondent could describe the steps to deactivate the agent, or what customers would see if it were turned off.
- Risk
- In an incident, the agent keeps answering while people work out how to stop it.
- Fix
- Write a one-page stop procedure, name two people who can run it, and test it once in a sandbox.
Success is not defined
- Evidence
- The service lead measures the agent by case volume, the admin by containment rate, finance by licence cost. The three numbers have never been compared.
- Risk
- The agent cannot be shown to be worth its cost, or shown to be failing.
- Fix
- Adopt the four measures below and review them monthly.
The agent answers from an outdated policy article
- Evidence
- The export shows which knowledge source the agent answers from. The questionnaire confirms coverage terms changed in March, and the source still includes the 2025 coverage guide.
- Risk
- Customers are told the old coverage limits.
- Fix
- Retire the old article and give the owner a quarterly check of the agent's knowledge sources.
What the agent should do alone
The agent may do this alone
- Answer claim-status questions
- Explain coverage from current articles
- Book a technician visit in an open slot
- Create a case and hand off
A person must approve
- Change a claim's contact details
- Anything involving a payment or reimbursement
- Rebooking a visit a third time
- Closing a claim