July 29, 2013 : SAN FRANCISCO — In Hollywood, there are umbrella holders. Outside corner offices, there are people who know exactly how much cream to pour in the boss’s coffee. In British castles, royals have their valets.
And then there is Silicon Valley, where mind-reading personal assistants come in the form of a cellphone app.
A range of start-ups and big companies like Google are working on what is known as predictive search — new tools that act as robotic personal assistants, anticipating what you need before you ask for it. Glance at your phone in the morning, for instance, and see an alert that you need to leave early for your next meeting because of traffic, even though you never told your phone you had a meeting, or where it was.
How does the phone know? Because an application has read your e-mail, scanned your calendar, tracked your location, parsed traffic patterns and figured out you need an extra half-hour to drive to the meeting.
The technology is the latest development in Web search, and one of the first that is tailored to mobile devices. It does not even require people to enter a search query. Your context — location, time of day and digital activity — is the query, say the engineers who build these services.
 |
Google Now, a search app that tries to anticipate what users want, as seen on a Galaxy Nexus. |
 |
| Heidi Schumann for The New York Times Baris Gultekin, a product management director at Google who helped invent Now. |
Many technologists agree that these services will probably become mainstream, eventually incorporated in alarm clocks, refrigerators and bathroom mirrors. Already, an app called Google Now is an important part of Google’s Internet-connected glasses. As a Glass wearer walks through the airport, her hands full of luggage, it could show her an alert that her flight is delayed.
Google Now is “kind of blowing my mind right now,” said Danny Sullivan, a founding editor of Search Engine Land who has been studying search for two decades. “I mean, I’m pretty jaded, right? I’ve seen all types of things that were supposed to revolutionize search, but pretty much they haven’t. Google Now is doing that.”
But for some people, predictive search — also in services like Cue, reQall, Donna, Tempo AI, MindMeld and Evernote — is the latest intrusion into our lives, another disruption pinging and buzzing in our pockets, mining our digital lives for personal information and straddling the line between helpful and creepy.
“To the question of creepiness, the answer is it depends who you ask,” said Andrea M. Matwyshyn, an assistant professor at the Wharton School of the University of Pennsylvania, who studies the legal implications of technology. “What works for a group of 30-something engineers in Silicon Valley may not be representative of the way that 60-year-old executives in New York tend to use their phones.”
Many software programmers have dreamed of building a tool like this for years. The technology is emerging now because people are desperate for ways to deal with the inundation of digital information, and because much of it is stored in the cloud where apps can easily access it.
“We can’t go on with eight meetings and 200 e-mails a day,” said N. Rao Machiraju, co-founder and chief executive of reQall, which sells its technology to other companies to make their own personal assistant apps. “We have a technology that isn’t waiting for you to ask it a question, but is anticipating what you need and when is the best time to deliver that.”
The services guess what you want to know based on the digital breadcrumbs you leave, like calendar entries, e-mails, social network activity and the places you take your phone. Many use outside services for things like coupons, news and traffic.
Google Now, which came to some Android phones a year ago and iPhones in April, tells you when it is time to leave for a dinner reservation. That is because it noticed an OpenTable e-mail in your Gmail in-box, knows your location from your phone’s GPS and checked Google Maps for traffic conditions.
A couple days before you travel, it will show you weather in your destination, and when you arrive, currency exchange information and the time back home. Ask aloud that Google Now remind you to pick up milk next time you step in a grocery store, and an alert will appear when you are at Safeway.
Successful predictive search, though, is as complicated as real life. If you are in London on business, which an app would know from the events on your calendar, you probably want a PDF related to work. But if you are there on vacation, you might want directions to Big Ben.
“By the time you search, something’s already failed,” said Phil Libin, chief executive of Evernote, a note-taking app that actively shows previous entries related to current circumstances.
Many of the apps use machine learning to get to know people over time.
ReQall’s service, for instance, can block calls from interrupting you during meetings. But one day, the young son of reQall’s co-founder, Sunil Vemuri, was sick at home with Mr. Vemuri’s father, who was urgently trying to reach him with a medication question. Because he called more than once, and reQall knew the two had the same last name and spoke often, the app interrupted the meeting.
The goal is to move beyond logistical help to sending you anything you might need to know. Google in May added book, movie and music recommendations, for instance.
“You can just imagine several years down the road, if that personal assistant was an expert in every field known to humankind,” said Amit Singhal, Google’s senior vice president for search.
Ads are not far off. “The better we can provide information, even without you asking for it, the better we can provide commercial information people are excited to be promoting to you,” Larry Page, Google’s chief executive, told analysts in April.
Some skeptics say pushing ads and other unwanted information could be annoying or even a violation of privacy. If you watch a movie trailer on YouTube, for instance, Google Now may send local showtimes when the film arrives in your city. But what if you hated the trailer?
“People could find the interface disruptive rather than helpful,” Ms. Matwyshyn said.
Baris Gultekin, a product management director at Google who helped invent Now, said the company was aware of that risk and was “very conservative” with what it showed people.
Daniel Gross, co-founder of another personal assistant app, Cue, said that was why it had started with alerts in which a person had already signaled interest, by creating a calendar entry, for example.
“It’s a really tricky problem, because on one hand you really want to give someone the best experience you possibly can,” he said. “And on the other hand, you don’t want someone to have this uncanny valley type of moment: ‘Oh my gosh, this feels too good.’ ”