Facebook Open Sources a Chatbot That Can Discuss Any Topic

The new version expands the capabilities of its predecessor building a much more natural conversational experience.





Image Credit: Facebook AI Research

 

I recently started a new newsletter focus on AI education and already has over 50,000 subscribers. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

Image

 

Last year, Facebook AI Research(FAIR) open sourced BlenderBot 1.0, the largest open domain chatbot ever built. BlenderBot is able to engage in a large variety of conversations across nearly any topic while displaying human-like characteristics such as empathy and personable levels of engagement. The first version of BlenderBot was one of the first chatbots to combine empathy, personality and knowledge in a single system.

Last week, the FAIR team open sourced a new version of BlenderBot which includes several key improvements to its predecessor. More specifically, BlenderBot 2.0 incorporates two main innovations relative to the previous version:

  1. Improved long term memory capabilities.
  2. Ability to search the internet for real time knowledge.

Transformer architectures such as GPT-3 or BERT have revolutionized natural language processing(NLP) systems but they still have marked limitations. Among them, long term memory stands out as one of the major drawback of large transformer models. In general, transformer architectures are only able to build short term memory structures that are highly dependent on the pretrained knowledge. For instance, GPT-3 is super knowledgeable about movies and TV series from a few years ago but is not up to date with current shows.



Image Credit: Facebook AI Research

 

To address this limitation, BlenderBot 2.0 complements the training knowledge with the ability of triggering internet searches for relevant knowledge that is then persisted in its long-term memory store. Architecturally, BlenderBot 2.0 augments its transformer model with an additional neural network that triggers search queries based on a specific conversational context. BlenderBot then combines with search results with the existing conversational knowledge in order to articulate a final response.



Image Credit: Facebook AI Research

 

The result of the BlenderBot 2.0 architecture is a chatbot that can combine natural conversations with real time knowledge while also accessing the context from previous conversations. The techniques used in BlenderBot 2.0 can inspire a new generation of conversational systems that adapts transformer architectures to the requirements of real world systems.

 
Original. Reposted with permission.

Related: