Meet Meena, Google’s first conversational AI
A chatbot that can imitate human conversation
Who is Meena?
Meena is an Artificial Intelligence (AI) chatbot created and considered by Google as the best digital assistant so far. It is a multi-turn open-
One of Meena’s conversations.
domain, with a neural network of 2.6 billion parameters. It is claimed to be better than any other AI chatbot available in the market and is capable of having human like conversations. Meena is considered a conversational AI. This term is given to AI models that can understand human language, speak and complete tasks given in simple words. In the past we’ve seen other similar AI assistants that can also complete given tasks and answer factual questions like Alexa and Siri but talking and carrying out a conversation are two extremely different tasks.
Meena’s capabilities measured by SSA:
To check the skills and capacity of a conversational agent, Google released a system of measurement called Sensibleness and Specificity Average (SSA). Sensibleness and Specificity Average (SSA) is a new metric for open-domain chatbots that captures basic, but important attributes for human conversation. SSA is simple, and yields more consistent results by tackling two fundamentals of a conversation, if it makes sense and if it’s specific. Human evaluators were asked if what the system says makes sense. If there’s anything confusing, illogical, out of context, or factually incorrect, the response would receive a low score when it came to sensibleness. The next evaluated criteria is specificity. Once human evaluators have deemed a response as one that makes sense, they are then asked to judge whether the response is specific to the given context.
Meena’s pun.
Meena is able to carry out any conversation and it is also able to make up jokes to brighten up the mood. The pun about horses going to Hayvard for college isn’t the best we’ve heard, but it certainly feels human. Meena can carry on a conversation at a level that feels far more like talking to another person than any existing chatbot.
So how is Meena different from other AI assistants?
Holding a human-like conversation is still very much impossible for the smart assistants available today but for Meena this seems like a pretty easy task. To compute SSA, a crowd-sourced free-form conversation with the chatbots, was carried out on Meena and other well-known open-domain chatbots, notably, Mitsuku, Cleverbot, DialoGPT and XiaoIce. In order to ensure consistency between evaluations,
Final SSA results from the chatbot experiment carried out by Google.
each conversation started with the same greeting, “Hi!”. For each utterance, the crowd workers answer the two questions stated above, “does it make sense?” and “is it specific?” Meena scored 79%, which is barely below the human score of 86%. Other AI developers have their own ways of measuring performance, but Meena crushed open-domain chatbots like Mitsuku and Cleverbot in the SSA test. According to Google, Meena is far closer to being able to carry on conversations like a human than other advanced chatbots.
How is Google’s Meena going to influence upcoming AI assistants?
Another conversation carried out by Meena.
Open-ended conversation that covers a wide range of topics is hard and most chatbots can’t keep up. At some point most say things that make no sense or reveal a lack of basic knowledge about the world. A chatbot that avoids such mistakes will go a long way toward making AIs feel more human, and make characters in video games more lifelike. A chatbot that can carry on an open-ended conversation is potentially very powerful and will help in the advancements of other AI assistants. Even with their current limitations, voice assistants are already reducing loneliness of older people, according to a British study. Google’s investment in Meena is only one of many efforts to further humanise computer interactions, improve foreign language learning, and make interactive video game characters more realistic and enjoyable.
About AIBC Summit:
AIBC Summit is the
emerging tech destination for leading brands in the sector, held in both Malta and Manila. The event brings together thought leaders in the blockchain and AI space. Join us at the first edition of Manila AIBC Summit, the
future tech event of the year, set to take place on the 08-09 June 2020.
