Wednesday, October 20Digital Marketing Journals

NLP based Chatbot in PyTorch. Bonus Flask and JavaScript deployment | by Victoria Maslova | Oct, 2021


Victoria Maslova

Among the various ways you can improve customer satisfaction, chatbots are a powerful solution to help the customer base. Chatbots are affordable, help scale your business, fully customizable, help your customers find the right products/services, and help build trust for your business. To prove this I’ll go through following content:

  1. What is a machine learning chatbot?
  2. Why chatbots are important in different business spheres?
  3. Build you own NLP based chatbot using PyTorch.
  4. Deploy chatbot in Javascript and Flask.

A chatbot (Conversational AI) is an automated program that simulates human conversation through text messages, voice chats, or both. It learns to do that based on a lot of inputs, and Natural Language Processing (NLP).

For the sake of semantics, chatbots and conversational assistants will be used interchangeably in this article, they sort of mean the same thing.

Business Insider reported that the global chatbot market was expected to grow from $2.6 billion in 2019 to $9.4 billion in 2024, forecasting a compound annual growth rate of 29.7%. The same report also suggested that the highest growth in chatbot implementation would be in the retail and ecommerce industries, due to the increasing demand of providing customers with seamless omnichannel experiences.

That alone should be enough to convince you that chatbots are the way to handle customer relationships moving forward, but they will also continue to grow as internal tools for enterprise tools and nearly every industry will adopt the technology if it hasn’t already.

Below are the key reasons why more and more businesses are adopting the chatbot strategy and how they are a win-win formula to acquire & retain customers.

  • Reduce customer waiting time — 21% of consumers see chatbots as the easiest way to contact a business. Bots are a smarter way to ensure that customers receive the immediate response that they are looking for without making them wait in a queue.
  • 24×7 availability — Bots are always available to engage customers with immediate answers to the common questions asked by them. The top potential benefit of using chatbots is 24-hour customer service.
  • Better customer engagement — Conversational bots can engage customers round the clock by starting proactive conservation and offering personalized recommendations that boost customer experience.
  • Save customer service costs — Chatbots will help businesses save more than $8 billion per year. Bots can be easily scaled which saves customer support costs of hiring more resources, infrastructure costs, etc.
  • Automate lead qualification & sales — You can automate your sales funnel with chatbots to prequalify leads and direct them to the right team for further nurturing. Being able to engage customers instantly increases the number of leads and conversion rates.

1. How Conversational AI can Automate Customer Service

2. Automated vs Live Chats: What will the Future of Customer Service Look Like?

3. Chatbots As Medical Assistants In COVID-19 Pandemic

4. Chatbot Vs. Intelligent Virtual Assistant — What’s the difference & Why Care?

There are many platform where developers, data scientists, and machine learning engineers can create and maintain chatbots like Dialogflow and Amazon Lex. But my goal in this article to show you how to create a chatbot from scratch to help you understand concepts of Feed-Forward Networks for Natural Language Processing.

Let’s get started!

You can easily find a complete code in my GitHub repo.

Here is a short plan that I want to follow to build a model.

  1. Theory + NLP concepts (Stemming, Tokenization, bag of words)
  2. Create training data
  3. PyTorch model and training
  4. Save/load model and implement the chat

We will build chatbot for Coffee and Tea Supplier needs to handle simple questions about hours of operation, reservation options and so on.

A chatbot framework needs a structure in which conversational intents are defined. One clean way to do this is with a JSON file, like this.

Chatbot intents

Each conversational intent contains:

  • a tag (a unique name)
  • patterns (sentence patterns for our neural network text classifier)
  • responses (one will be used as a response)

So our NLP pipeline looks like this

  • Tokenize
  • Lower + stem
  • Exclude punctuation characters
  • Bag of Words

We create a list of documents (sentences), each sentence is a list of stemmedwords and each document is associated with an intent (a class). Full code is in this file.

Then we need to set a training data and hyperparameters.

After all needed preprocessing steps we create a model.py file to define FeedForward Neural Network.

Feedforward neural networks are artificial neural networks where the connections between units do not form a cycle. Feedforward neural networks were the first type of artificial neural network invented and are simpler than their counterpart, recurrent neural networks. They are called feedforward because information only travels forward in the network (no loops), first through the input nodes, then through the hidden nodes (if present), and finally through the output nodes.

Be careful! In the end we don’t need an activation function because later we will use cross-entropy loss and it automatically apply an activation function for us.

Why we use ReLU?

They are simple, fast to compute, and don’t suffer from vanishing gradients, like sigmoid functions (logistic, tanh, erf, and similar). The simplicity of implementation makes them suitable for use on GPUs, which are very common today due to being optimised for matrix operations (which are also needed for 3D graphics).

After defining a CrossEntropy Loss and Adam we implement backward and optimizer step.

What do all these lines mean?

We set zero_grad() to optimizer because in PyTorch, for every mini-batch during the training phase, we need to explicitly set the gradients to zero before starting to do backpropragation (i.e., updation of Weights and biases) because PyTorch accumulates the gradients on subsequent backward passes.

Calling .backward() mutiple times accumulates the gradient (by addition) for each parameter. This is why you should call optimizer.zero_grad() after each .step() call. Note that following the first .backward call, a second call is only possible after you have performed another forward pass.

optimizer.step is performs a parameter update based on the current gradient (stored in .grad attribute of a parameter) and the update rule.

Finally, after running train.py script what a wonderful result we got!

And in the last part we need to save our model. Here the way I did it easily.

I decided to go further and create this amazing visualization of ChatBot.

All my HTML, CSS and JavaScript scripts you will find in my GitHub repo.

Enjoy!

Now, as you are aware of what a chatbot is and how important bot technology is for any kinds of business. You will certainly agree that bots have drastically changed the way businesses interact with their customers.

Chatbot technologies will become a vital part of customer engagement strategy going forward. Near to future bots will advance to enhance human capabilities and human agents to be more innovative, in handling strategic activities.

Leave a Reply