A voice AI demo is designed to impress. The agent sounds natural, answers a scripted question smoothly, and the room nods. None of that tells you whether the system will still be accurate in November when your programs, deadlines, and staff have all changed, or whether it will write a lead back into your SIS, or what happens when a student gets upset. The questions below are the ones that separate a running system from a good demo. Ask them before you sign, and hold every vendor, including us, to the same standard.
This is written from the operator’s side of the table. The goal is not to make the evaluation adversarial. It is to make sure you buy something that works in production, not just in a sales call, because a voice program that underperforms can affect how prospective students experience your institution.
1. Integration: does it write back, or just read?
Many tools can read from your systems. The ones that save real work write back to them. Ask specifically whether the agent updates your student information system after a call, or whether it just logs a transcript someone has to re-key later.
- Does it write back to our SIS, not just read from it? Two-way is the difference between saved work and doubled work.
- Which CRM and telephony platforms are already integrated? Ask for live, proven integrations, not roadmap items.
- How long does integration actually take? Get a real number tied to a real environment like yours.
2. Compliance: can it survive procurement and legal?
Student data carries obligations, and your procurement and legal teams will check. Ask the compliance questions early so a deal does not stall late. This is also where a serious vendor is precise rather than vague. For the full picture on the education-records side, see the companion piece on what FERPA requires from a voice AI vendor.
- How do you handle student data, and where does it live? Encryption in transit and at rest, and clarity on data residency.
- How are calls recorded, stored, and logged? You want an audit trail, and a clear retention policy.
- What is your posture on FERPA, TCPA, and SOC 2? Ask what is certified versus in process, and expect an honest answer, not a checkmark.
3. Escalation: what happens when a student needs a person?
The most important thing a voice agent does is know when to stop and hand off. A system that traps a frustrated or distressed student is worse than no system at all. Probe the escalation design hard.
- What triggers a handoff to a human? Distress, repeated confusion, an explicit request, or an out-of-scope intent should all qualify.
- Does the human get full context on transfer? The student should never have to repeat themselves.
- What is the escalation SLA? How fast does a student reach a person once the agent decides to route them?
4. Ownership: who builds and tunes it after launch?
This is the question that most often gets skipped and most often causes regret. A voice agent is not set-and-forget. It needs tuning as your programs and language change. Ask who does that work: you, or the vendor. A managed service versus a self-serve platform is precisely this distinction, and for most enrollment offices, owning ongoing tuning on top of the day job is the reason a self-serve tool quietly stops working.
- Who builds the call flows and knowledge base? Your team, or theirs?
- Who tunes it after launch, and how often? Ongoing tuning is what keeps accuracy from decaying.
- What does support look like at your peak? When registration or FAFSA season hits, who is on the hook?
5. Proof: has it actually run in higher ed?
A demo proves the tool can talk. Proof is whether it has run in a real institution like yours. Ask for it plainly, and ask to speak to someone who has lived through a deployment.
- Do you have live higher-ed deployments? Not adjacent industries, higher ed specifically.
- Can we speak to a reference we can actually call? A named reference beats a logo wall.
- What went wrong in a deployment, and how did you fix it? An honest answer here tells you more than any success story.
Two questions vendors hope you forget
Beyond the five categories, two questions can reveal problems that are easy to miss, and they are the ones a weak vendor may avoid answering clearly. The first is about accuracy over time. Ask: how do you keep the agent from confidently saying something wrong six months after launch, when our deadlines and programs have changed? A good answer describes a tuning and review process and a way to ground the agent in approved knowledge. A bad answer waves at the model being smart. Accuracy is not a launch-day property. It has to be maintained over time, and the vendor should treat it that way.
The second is about failure. Ask: what happens when the agent does not know the answer? The right behavior is to say so and route to a person, not to guess. A system that invents a plausible-sounding answer to a financial aid or eligibility question can create real risk. A confident wrong answer at scale can be more damaging than a slower human response. Push the vendor to show you exactly what the agent does at the boundary of its knowledge, and make sure that behavior is designed to be cautious.
A note on the demo itself
When you do run the demo, test the edges, not the center. Ask the agent something off-script. Interrupt it. Give it a messy, real question a student would actually ask. See how it escalates. The polished path is easy; how a system behaves at the edges is what your students will experience on a bad day. The same rigor applies whether you are evaluating voice or text, which is worth remembering if a vendor pushes a chatbot instead of voice: hold both to the identical standard.
How to run the evaluation itself
Structure the process so the answers are comparable. Send the same written question set to every vendor before the demo, so you are reading like-for-like rather than reacting to whoever presents most smoothly. Get compliance and IT in the room early, not at the end, because a tool that cannot clear your security review is a non-starter no matter how good the demo sounds. And insist on at least one reference call with an institution similar to yours, where you ask what the first ninety days were actually like, not whether they are happy in the abstract.
Finally, weigh the answers by what will matter in production, not in the sales cycle. Integration, ongoing ownership, and escalation behavior determine whether the system still works in month six. Voice quality and demo polish are table stakes that every serious vendor can clear. If you score the evaluation on the things that decay, accuracy over time, who maintains it, how it fails, you will pick the tool that survives contact with your real call volume rather than the one that presented best.
How BrainCX answers these questions
Since we are telling you to ask, it is only fair to say where we stand. BrainCX writes back to the SIS and integrates with major CRM and telephony platforms; operates under HIPAA and TCPA with SOC 2 Type II in process; escalates on distress, confusion, request, or out-of-scope intent with full context passed to the human; and is delivered as a managed service, so we build and tune it, not you. The details live on the BrainCX platform overview. Ask every vendor these questions, then compare the answers side by side.
Frequently asked questions
1. What should you ask a voice AI vendor before buying?
Cover five areas: integration (does it write back to your SIS, not just read), compliance (data handling, recording, FERPA/TCPA/SOC 2 posture), escalation (what triggers a human handoff and the SLA), ownership (who tunes it after launch), and proof (real higher-ed deployments and callable references).
2. Why does SIS write-back matter?
Because reading data only logs a transcript someone must re-key. Writing back updates the student record automatically after a call, which is the difference between saved work and doubled work.
3. What is the most overlooked question?
Who tunes the agent after launch. A voice agent needs ongoing tuning as programs and language change. If that falls on your team, a self-serve tool can quietly stop working; a managed service keeps it accurate.
4. How should we test the demo?
Test the edges, not the center. Ask off-script questions, interrupt, give a messy real-world question, and watch how it escalates. Edge behavior is what students experience on a bad day.
5. Should we hold chatbot and voice vendors to the same standard?
Yes. Integration, compliance, escalation, ownership, and proof matter regardless of channel. Apply the identical checklist to any vendor.
Buy a running system, not a good demo
Bring these questions to every vendor conversation and compare the answers honestly. Talk to the BrainCX solutions team to see how we answer them, or explore how BrainCX serves trust-dependent industries.