We built measurable framework to compare how well each voice assistant could understand, respond to, and follow up on a wide range of real-world questions.

We asked 50 questions to each tool: 15 commerce, 20 informational, and 15 reasoning prompts.

We graded the voice assistants on their understanding of the question, need for clarifications, helpfulness, clarity, and conciseness of the response, could it contextually handle a follow-up question, and the speed of response.

Since all voice assistants were fully capable of understanding each question without the need for clarifications, we used the following scoring system:

• 2 points: Was the answer helpful, clear, and concise? 
• 2 points: Was the voice assistant able to handle a follow-up question?
• 1 point: Were the responses fast? (i.e., responded within 5 seconds)

If the assistant crushed it, it was awarded 5 points. If it completely failed, 0 points. The range of scores ended up falling between 1-5 points.

On a traditional grading scale, Siri would receive an F, ChatGPT through Siri would receive a B, and ChatGPT gets an A.

Siri on its own is stuck in the last decade. It functions more like a voice-activated Google search than a true AI assistant. Rarely did Siri provide a clear, useful answer, either via voice or text. Within the scope of this experiment, the model offered little value beyond simple device commands. It still does well with tasks like “Call Mom” or “What’s the weather this weekend?” or anything that taps into its deep integration with iOS, but it struggles with anything that requires reasoning or conversational follow-up.

ChatGPT’s integration into Siri is better, but the experience is sluggish. On average, Siri’s handoff to ChatGPT took 5–10 seconds, with some answers delayed by as much as 20 seconds. In a world of instant results, that lag makes a difference.

ChatGPT voice, by contrast, feels like the future. The model mimics human rhythm and tone, even including natural pauses and filler words like “um” and “uh.” You can interrupt it mid-sentence, change direction, or go deeper and it keeps up. I found myself engaging in unscripted conversation, testing boundaries and learning more without thinking about it. It’s by far the most capable and life-like model we’ve tested.

We all know that Apple is behind in AI and needs to nail the revamped Siri. We believe they should take as much time as necessary to get this right. A rushed rollout risks another high-profile miss, something Apple can’t afford in the AI race.

The refreshed Siri, initially demoed at WWDC 2024, was expected in spring 2025, but is now delayed until spring 2026. That delay suggests Apple knows what’s at stake. Unlike ChatGPT, Siri isn’t just a chat interface. It’s deeply embedded in your device, accessing everything from your location to your calendar to your messages.

That makes Siri more powerful in theory, but far more complex to build. Apple has to thread the needle between cutting-edge AI and airtight privacy. If they can do both, Siri could leapfrog today’s models. But it has to be more than a voice for web search. Siri needs to think, converse, and assist like an intelligent partner.

Disclaimer

Appendix: Full list of questions and follow-ups: