Siri: Your wish is its command, some of the time

By Salvador Rodriguez

June 29, 2012 12 AM PT

Apple’s voice assistant, Siri, was put under the microscope this week with a tough test and the arrival of a new opponent, and it did not fare well at all.

Siri, the signature feature of the Apple iPhone 4S that was introduced in October, has received a lot of criticism from consumers, with many saying it does not do many of the things Apple shows it doing in its commercials.

But those were only anecdotes until now, thanks to an analyst who has given Siri a rigorous test that shows she is currently doing poorly at her job.

Gene Munster of Piper Jaffray reportedly sent a note Thursday to clients explaining that he had asked Siri a series of 1,600 questions to get some hard data on how well it works and asked the same questions of the Google search engine for a comparison. Munster asked Siri 800 of the questions in a quiet room and asked the others out on the streets of Minneapolis.

His results showed Siri understood 83% of the questions asked during noisy situations and 89% in quiet situations. She answered 62% of the questions asked in noisy conditions correctly and 68% in the quiet room.

By comparison, questions typed into Google were understood every time, and Google replied accurately 86% of the time.

“In order to become a viable mobile search alternative Siri must match or surpass Google’s accuracy of B+ and move from a grade D to a B or higher,” Munster said, according to Fortune.

With Apple’s announcements earlier this month that it would be beefing up Siri come iOS 6, Munster said there was hope for the voice assistant.

“We expect Siri to improve meaningfully while reducing its reliance on Google from 60% to 48%,” he said.

Right now, Siri gets 60% of her answers from Google, 20% from Yelp, 14% from WolframAlpha, 4% from Yahoo and 2% from Wikipedia, according to Munster.

To show how clunky Siri can be at times, Munster gave some examples. Siri could not correctly identify Munster’s location, could not give accurate directions from Boston to New York, and, when asked which team NFL quarterback Peyton Manning plays for, gave an answer to the previous question Munster had asked.

Siri said she couldn’t answer the question “Where is Elvis buried?” because she thought she was being asked about a person named “Elvis Buried.”

But that isn’t all the bad news for Siri.

Earlier this week, Google showed off its improved Voice Search for its upcoming Android operating system, Jelly Bean, which will launch next month, and TechnoBuffalo compared it with Siri in a video that you can watch below.

From that test, it’s even easier to see Siri is not the champ in the market she and Apple arguably created.

Voice Search was much quicker than Siri, and sometimes Siri made the process take extra long by making you have to answer a second question rather than just providing information the way Voice Search did. It is an embarrassing video for Siri, and an extra bonus for some users, depending on their preference, will be that the improved Voice Search also has of a less robotic-sounding voice than Siri.