As an investor/trader, I wanted to access the information of the stock quickly and buy/sell quickly. However, often trading is not the only job we do right?. So what if all the information are in the one place for us to capture the right opportunity?
If you are ever thinking of creating a consolidated page or bot, you will find endless medium/ other articles how to develop it. or “create a bot in 10 minutes” or “create a bot without coding”. However, have you ever used a bot on a daily or frequent basis which provides some info or trades it for you? The answer will be NO for many of them. So I decided to build one for my needs.
Many wanted to use these types of bots to make money and often they will not have a perfect bot that works for their needs. I have been in the stock market for some time, trust me predicting stock prices is really really hard. You need to be extremely good at neural networks/modeling with good quality of data and software engineering skills to deploy in realtime. So to create a bot you need to be modeler, quant person, software engineer, and data collector. These are different skill sets. If you are a full-time trader or on (High Frequency Trading) HFT and interested in all the above skillsets, then you can certainly invest time and explore. According to my knowledge for normal retail investors, human judgment is the best one to make money in the market.
If you see the US stock market, it is back to its peak and tech stocks have soared like anything. If you have simply invested your money on specific stocks you could have easily received more than 200% return. A couple of examples are GameStop(GME), AMC, Lemonade and the list goes on. Please check out their historical prices at the time of writing this article. But how do you know about those stocks? And, how do I get the information? How do I check the pulse of the stocks?
When the market was starting to crawl back, I asked these questions to myself like a normal retail investor. For an investor, if he has all the information at the correct time he will be able to buy/sell those stocks. Rather than having a bot that can predict the prices of stocks, it should provide the below information.
- Show me the price of stocks
- Get me the latest news on my stocks
- Provide me the sentiment on news articles
- Show the community stock sentiment from Twitter
- The popularity of the stocks(Positively or negatively)
- Institutional activity on the stocks
- Finally, buy and sell from users commands or triggers. No auto trading.
So I decided to build a bot with these capabilities which are not fancy but provide me the required info so I can get information even when I am in my office hours via WhatsApp. I often miss to login into my brokerage account during office hours.
After trying all fancy chatbots, I decided to use a framework called Rasa. It is a chatbot framework that understands intents and I can code my own logic in python on the server-side. Let me know in the comments if you wanted to know more about RASA.
Here is the quick detour tutorial on RASA
You can create a new virtual environment and install rasa as per their documentation. Once completed, if you execute below command it will create a structure.
Create NLU intents
The intent is nothing but what the user is aiming for. For example — if the user says “News of TSLA” the intent can be classified as to get the price.
We need to create different intents. As mentioned before, RASA needs some sample sentences to get the intent and stock ticker symbol.
## intent:stock_news- what is the news of [TSLA](stock_name)- Latest information about [TSLA](stock_name)
You can add more examples to make the bot better. However, we will be using NLP TRANSFORMER models with the bot to understand the context.
Stories define the sample interaction between the user and chatbot in terms of intent and action taken by the bot. Like in the example above bot got the intent of price. Get the sentiment of the stock after checking the price.
You need to add conversation tracks with the user. This is where the real-world conversation will help. More conversation stories you have, the bot will perform better.
1.Chatbot Trends Report 2021
2. 4 DO’s and 3 DON’Ts for Training a Chatbot NLP Model
3. Concierge Bot: Handle Multiple Chatbots from One Chat Screen
4. An expert system: Conversational AI Vs Chatbots
Entity is to extract useful information from the user input. From the example above “Price of TSLA” the entities extracted would be the stock ticker symbol.
If there are different entities, everything needs to be created and added to domain.yml file.
Actions are basically the operations performed by the bot either asking for some more details to get all the entities or integrating with some APIs or querying the database to get/save some information.
Until now all the configurations are in yml format. If you need any type of custom code this is where it is written.
There are other groundwork that needs to be performed. We need to gather all the stocks and create an entity file. Also, the domain.yml needs to have all the intents, entities, and any predefined responses.
The real strength of the bot lies in the data which it provides back to the user. Here we need an API which performs different tasks
- Provides stock quote
- Gets the news
- Provides us the news sentiment
- Gets us the twitter tweet and sentiment
- Provide institutional info on stock
Each one of it a unique segment of its own. It took so much time for me to collect and create a REST API. I have hosted the API here. I am just going to consume the API in our bot. If there is enough interest I will cover behind the API in a separate post.
All the above-mentioned info will be delivered via a rest API with multiple endpoints. These endpoints will be called from actions.py file
After all our setup, below is the final structure of the chatbot code.
In rasa there is config.yml file which contains the pipeline of steps which you need to perform. If a user enters some text, below are some the things which you need to perform
- Split by whitespace — Splitting the text
- Tokenization — Convert string to number
- Parse any symbols or unwanted characters- like commas, etc
- Classifier — Assign one of the intents from the domain file
- Policies — Action to take at every step in the conversation
Rasa provides different options on all these components and sections. Checkout https://rasa.com/docs/rasa/nlu/components/ and https://rasa.com/docs/rasa/core/policies/
I have chosen to create a custom policy with a two-stage fallback policy.
Once you have complete the setup, then issue the below command. It will train the model.
This is a critical point of our bot. When a user interacts with the bot it should create a WebSocket connection provide back the response. This is where rasa shines. It has multiple API integrations, it can be connected with Facebook messenger, telegram, Twilio, or a webchat. All the related keys need to be entered in endpoints.yml
Currently, I have integrated the bot via normal message and WhatsApp business account. Whatsapp business account requires additional approvals from Whatsapp and Facebook business. Once all the approval is complete, you can finally use the bot. It is not mandatory to deploy in Whatsapp. It can be deployed on any channel even with a text message the bot can be accessed.
Backend API services are deployed in AWS EC2 instances.
I have been using this bot and web tool which I developed for a while and I was showing it to my friend. He suggested this product will be useful for many of the folks who want quick information about stocks. So for wider access, I am in process of creating a scalable app with underlying data with affordable subscription.
Please follow me here for more updates and other articles.
Currently I am in process of writing a book with the integration of stock market and python in a book. If you are interested please subscribe here to get notified about it.