Gmail users are set to benefit from Google's machine learning research with Smart Reply. The system will use a deep neural network to not only analyze incoming emails for what information is required to form an appropriate response, but to propose three likely replies, with the end result enabling mobile users to respond quickly to emails.
Google's research blog details the initial challenge and the science that went into creating the technology. Crucial is a concept called sequence-to-sequence learning, already used in Google translation and a chatbot the search giant released earlier this year.
In sequence-to-sequence learning, two neural networks fuse both understanding a language and synthesizing language. The decoding network creates a thought vector by transcribing each word individually into a number, based on its context within the rest of the text. This grants the network the "idea" of the email.
The encoding network then generates potential responses, obviously not knowing which the human user might be partial to, but all of them making sense in the context of the decoded message and presenting suitable alternatives.
Researchers encountered some challenges with this latter network. First, Smart Reply might generate perfectly reasonable responses, but they would all be essentially the same in terms of content, just rephrased.
Smart Reply was also had a propensity to respond with "I love you", "sounds good" or "thanks" when it wasn't sure of the correct response, so it resorted to common responses as a fallback. This problem was solved by normalizing the likeliness of a response in comparison to how probable it had been in the past. Thus Smart Reply will no longer attempt to hit on your boss.
Before technologists had the computer power to run neural networks, a chatbot or other artificial language system might rely on cleverly coded rules of grammar and syntax, rather than understanding what its conversational partner was saying. Google researcher and blog author Greg Corrado explains why this would break down for the Gmail's Smart Reply.
He describes the "tremendous diversity" in which people communicate, and points out that machine learning could capture the style and tones of different writing styles. No matter how anticipatory the rules would be, emails would range far beyond what programmers could realistically expect.
He also points out that this feature is being created and fine-tuned without a human ever reading an email. So while you won't have to worry that Smart Reply is being refined off your email to your grandma, you can probably anticipate a few wonky responses while the neural nets learn.
Unlike Google's research into training networks to play Atari games, or reverse engineering image recognition into "Inceptionism" art, Smart Reply will be available for iOS and Android Inbox users to play with later this week.
Source: Google