How we test AI receptionists

Every service gets the same fictional business, the same instructions and the same six phone calls. We record every call and publish what happened.

1. Same business, same instructions

We set up each service as the same small dental office: identical opening hours, services, prices, an after-hours emergency instruction and a rule for when to hand a caller to a person. Only the provider changes. See the exact test profile.

2. Six scripted calls

A test caller places these calls to each service's phone number. Each call has a written pass list.

1. Book an appointment

The caller: You are calling a small dental office. You want a cleaning next Tuesday afternoon. Give your name as Jordan Miller and your phone number as 312-555-0142 (say all ten digits) when asked. Accept the first reasonable time offered.

• Offers or confirms a specific day and time
• Collects the caller's name
• Collects or confirms a callback number
• Repeats the booking back before ending

2. Ask about prices

The caller: You are calling to ask how much a standard cleaning costs and whether they take Delta Dental insurance. If the agent doesn't know, ask whether someone can call you back.

• Answers from the business information it was given
• Does not invent a price it wasn't given
• Offers a callback or next step when it doesn't know

3. After-hours emergency

The caller: It is late evening. You have severe tooth pain and swelling and want to know what to do tonight.

• Recognizes the urgency
• Gives the emergency instruction or number it was given
• Does not give medical advice beyond its instructions
• Takes a message for the morning

4. Spanish-speaking caller

The caller: Speak only Spanish. You want to know the office hours on Saturday.

• Understands the question
• Answers in Spanish or offers a Spanish option
• Gives the correct Saturday hours

5. Interrupts and changes their mind

The caller: Start booking for Thursday morning, then interrupt the agent mid-sentence and change to Friday at 3pm.

• Stops talking when interrupted
• Updates the booking to Friday 3pm
• Confirms the final time correctly

6. Asks for a person

The caller: You are frustrated after a billing mistake and ask to speak to a real person right away.

• Stays polite
• Follows the transfer or message instruction it was given
• Captures the reason for the call

3. What we measure

• Pickup and response time: rings before answering and the pause before each reply, measured from the call recording.
• Task success: each item on the pass list, marked pass or fail from the transcript.
• Accuracy: any price, hour or detail it invented that wasn't in its instructions.
• Follow-through: whether the booking or message actually reached us (email, text or calendar).

4. What we publish

For each service: the recordings, the transcripts, the pass list results and the date of the test. When a provider changes its product, we test again and keep the old result with its date.

Independence

We pay for or use the standard free trial of every service we test. Providers don't see results before publication and can't pay to change them. How we make money.