ProBot: An online aid to procurement

Abstract. The activity of procurement is one such activity that is not only crucial for any organization from the fiscal point of view but also involves a great deal of lucubration both on human and software fronts. In this paper, we discuss about a chatbot developed to automate this tedious process. FinderBuddy is a procurement bot i.e. it is a chatbot designed to facilitate the procurement process in industries. (Here, FinderBuddy was designed for a simple office building where the bulk of work is on desk work. Hence the supplies in the database are predominantly stationery items and office supplies).It enables all the important features like issuing of desired items from the inventory, procuring best deals of materials from suppliers, etc. Apart from these, it also offers advanced features like processing natural language input (apart from accepting queries in a well-defined syntax), accepting voice input, delivering audio output, displaying items running short in the inventory, updating quantities of items in the inventory, tracking pre placed orders, changing orders still in processing, cancelling orders, reordering past orders, checking availability of various items in the inventory, review of past orders, spell check etc. Thus the chat bot provides all possible features to make the procurement process lucid, smooth and user friendly with least human intervention. The design involves crucial use of database concepts, Natural Language Processing, Web Scraping and Speech-to-Text (and vice versa) conversion. The implementation is carried out in Python and SQL using varied APIs.

Keywords- Procurement, chatbot, Natural Language Processing, Web Scraping, Text mining

1 Introduction

The activity of procurement is one such activity that is crucial for any organization from the fiscal point of view but it also involves a great deal of lucubration both on the software and human fronts. Searching for the best deals of raw materials and managing items in a commercial firm requires great deal of effort and vigilance from the part of the manager as his decisions involve huge monetary investments. A handy solution for this was the maintenance of databases but they too needed human presence to supervise them and hence were prone to errors. With the current surge of Artificial Intelligence, chatbots are almost becoming intrinsic in handling various interaction activities of businesses like taking orders from customers, customer complaint redressal, digital assistants etc. Carrying forward this idea, a chatbot can also be designed to regulate the procurement and inventory control activities of a firm. Such a bot can simply ask the user to input his needs (be it in text or speech form) and accordingly process them to judge if they can be fulfilled by the current available supply and obviously meet those needs if feasible. Otherwise, the bot can revert to the net, look for the best deal for that commodity amongst all potential suppliers and present the most economically sound deal suggestion to the user. Other secondary tasks like updating stock lists, keeping track of short items, canceling

orders, tracking orders etc. can also be handled easily by such a bot. thus relieving the staff of many tedious tasks and increasing efficiency at the same time. The primary motto was to construct an error free efficient system that can accept user input and provide responses to the user such that the user gets the experience of interacting with human staff personnel while using the chatbot. Such a chatbot, if deployed in the current industrial setup can help to speed up procurement proceedings while saving on the user effort.

Thus in this paper we present a procurement chatbot of the fore mentioned kind, called the FinderBuddy.(Here, FinderBuddy was designed for a simple office building where the bulk of work is on desk work. Hence the supplies in the database are predominantly stationery items and office supplies).. The bot has been developed in Python using the ChatterBot API and it uses SQL and MongoDB at the backend as databases. The bot is capable of issuing items from the inventory as per the order. It can also access the internet, use it to gather information about the best deals of materials from various suppliers and then order supplies according to the most economically optimum deal. All these changes are also updated in the bot’s database. The bot can also process voice input and deliver audio output. It can also process queries in colloquial formats(apart from a well defined syntax) and is largely tolerant to spelling mistakes. The bot can also be used to track or cancel orders, change orders, review past orders ( or even reorder them as it is, without entering the details time and again), check the availability of various items in the inventory, keep a check on items running short etc. These features are implemented using concepts like Web Scraping, Natural Language Processing, Text-to-Speech Conversion( and vice versa) and database concepts. The software tools used for implementation include BeautifulSoup, Google Speech Recognition API, MongoDB, SQLite, Tkinter etc.

2 Basic Concepts

In this section of the paper, we will provide a brief overview of all the basic concepts that have been applied together in consonance to achieve the smooth functioning of the bot. The section presents the broad concept, construes its meaning in plain terms and demonstrates its point and degree of application in the bot.

2.1 Web Scraping

Web scraping (web harvesting or web data extraction) is data scraping used for extracting data from websites.Web Scraping[1]software may access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. We studied two Python frameworks for web scraping-BeautifulSoup and Scrapy.

a) BeautifulSoup-Beautiful Soup is a Python package which creates a parse tree for parsed pages that can be used to extract data from HTML, which is useful for web scraping.

b) Scrapy-Scrapy is a Python framework for large scale web scraping. It gives all the tools needed to efficiently extract data from websites, process them as wanted, and store them in a preferred structure and format.

After doing sufficient study and analysis of both, we tried doing scraping of required websites. using both one by one. The results are as under-

BeautifulSoup-The structure of the code of the required websites was analyzed and appropriate BeautifulSoup was written in Python by setting the required parameters and filters. The module thus developed was tried both independently and upon integratingwith the rest of the chatbot. In both the cases, it offered efficient results with reasonable accuracy and speed.Scrapy-Scrapy involves making separate crawlers for each website and running them separately. Thus, after making the required spiders for each of the websites, they were tried both independently and upon integrating with the rest of the chatbot. Though the spiders worked efficiently upon being run independently, however upon integrating with the rest of the chatbot, the single threaded nature of each spider and the fact of running many spiders together along with the single threaded nature of Tkinter created a serious lag of speed.Hence, after considering the above facts, we arrived at the conclusion that BeautifulSoup was a better option since it could fetch all the information we need with considerable speed and less memory overheadThe bot makes use of web scraping to scrape popular sites. The required item is rendered as a keyword and with the help of urllib package of Python, a search is initiated on the selected sites (the urls of these sites are already present in the code). The results fetched by the site are suitably scraped and seminal details like the price, dimensions, brand name etc of the five most suitable options are returned to the user.

2.2 Text to speech and Speech to text

An important feature of the bot under analysis is its ability to accept voice input and deliver audio output because this gives the user a feeling of having a real time conversation.

a) Converting Speech to TextThe bot needs to accept queries from the user in speech form and extract relevant details from it like the name, quantity and other specifications of the product. Thus, for converting speech to text, Google’s SpeechRecognition API can be used.SpeechRecognition 3.7.1 is the Library for performing speech recognition, with support for several engines and APIs, online and offline.Furthermore, the computer system needs to be equipped with a microphone to enable the user to use this feature.b) Converting text to speech

The bot needs to convert text output to speech to return it to the user. For this, the python library pyttsx was used.Pyttsx is a cross platform text to speech wrapper.

2.3 Natural language Processing

It needs to be made sure that the client can interact with the bot in the most colloquial way, i.e. like he would talk naturally to any human assistant. Human beings interact with each other using proper sentences comprising of grammatical elements like nouns, pronouns, verbs, tenses etc. but till now the bot had been configured to handle queries in a particular syntax( like #change_order/31/4). But analyzing these kind of queries is difficult for a program because they lack a certain syntax( or format) . Hence to extract useful information from such queries, we need to do Natural Language Processing.

Natural Language Processing is a branch of artificial intelligence that deals with analyzing, understanding and generating the languages that humans use naturally in order to interface with computers in both written and spoken contexts using natural human languages instead of computer languages.Hence, we undertook a study of basics of NLP and came up with an algorithm involving POS tagging, lemmatization and singularization to process user queries. The algorithm so constructed has been designed to consider all possible synonyms and tenses of the concerned query word.

a) POS TaggingIn NLP, each word of a sentence is considered a token. Automatic assignment of descriptors to the given tokens is called Tagging. The descriptor is called a tag. The process of assigning one of the parts of speech(like nouns, verbs, adverbs etc.) to the given word is called Parts Of Speech(POS) tagging. It is commonly referred to as POS tagging. Example:Word: Paper, Tag: NounWord: Go, Tag :VerbWord: Famous, Tag: AdjectiveNote that some words can have more than one tag associated with them. For example, chair can be noun or verb depending on the context.Hence, in a given sentence we can separate out tokens and label them as per their grammatical significance like noun, adjective, verb etc. and this information can be used to frame suitable queries and process them.For example, in a query like-“I need black pens”In this query, we can do POS tagging likeWord :I :PronounWord :need :VerbWord :black :AdjectiveWord:pens:

Hence, we can filter out the pronoun and use the verb to make out what action needs to be carried out. The noun indicates the commodity and the adjective indicates the specification of the commodity in question.

b) LemmatizationAt times the user might enter different forms of verbs in his query like requires, required etc. But to frame the correct query, we need just the root form of the word i.e. ‘require’. Thus the derivation of the root form of the word is undertaken in lemmatization. It is this root form which is used to make out the nature of the query. Hence, an entered verb is first reduced to its root form (or lemmatized) and then this root form is matched against the various mentioned keywords in the code so as to determine the associated function and execute it.

c) SingularizationThe user might enter the plural form of a noun as the ‘commodity’ for example, the user might enter the word ‘pens’ but this will not match the database since the database has an entry by the name ‘pen’. To resolve this issue, we need to singularize the item name before passing

it on. In case the item name is already in singular form, it is passed on as it is otherwise its singular form is passed.

d) Dealing with SynonymsTo trigger the execution of the language module of the bot, the user’s query needs to contain any of the specified keywords. Only then is the natural language module executed and all of the abovementioned processing occurs on the query. But it is quite possible that the user may enter a synonym of the keyword, another tense of the keyword (other than the one specified), plural form of the keyword or a combination of any of these conditions. Hence, the gateway function needs to contain the specification of all of these cases. But no programmer can manually recall and enter all of these. Hence, lists of all possible synonyms of all keywords were created and these lists were appended with all possible tense and singular/plural forms of the words already present in them. For example, a separate list for ‘need’ query comprising words like needed, required, requiring etc. ; another list for ‘change’ query comprising words like changing, altered, modify etc. Each of these lists was used to specify the condition for the execution of the associated function. For instance, any word in the ‘need’ list would initiate the processing of the need query to issue needed items to the user. Other than these lists, a superlist was created by adding all these lists and this superlist was attached to the gateway function to ensure the triggering of this natural language module in case any of the words in this superlist were present in the query.

The synonyms of each keyword were generated using the wordnet package of nltk in Python.

The tenses and singular/plural were generated using basic rules of grammar.

Since all languages of human communication are largely ambiguous, a hundred percent accuracy rate is impossible to achieve with any natural language processing system. However, here we have tried to attain the maximum possible efficiency in least possible time by considering all possible scenarios that we could think of

2.4 Database Concepts

Since the problem domain of this bot is largely concerned with the proper maintenance of an inventory, the bot heavily banks of database concepts like select query(to issue required items, display price or delivery date of items etc.), update query( to maintain stocks of items), delete query(to cancel orders) etc. The syntax for programming of these queries is dependent on the database management system (DBMS) used.

3 Tools Used

To implement the concepts discussed above, a variety of technologies were worked upon and integrated in this bot to achieve the end product. They are described as under.

3.1 Python

Python[2]is a programming language like C, PHP, etc. Its interpreted nature(giving quick execution time), multi platform support, presence of high quality packages and large community support---all these reasons formed the foundation of our decision to code this bot in

Hence the main code, along with the adapters and the APIs are all in Python.

3.2 SQLite

SQLite[3]is an embedded SQL database engine. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file. The database file format is cross-platform - a database can be easily copied between 32-bit and 64-bit systems. These features make SQLite a popular choice as an Application File Format.

SQLite is used to track the item details and order details in this bot. The two databases used are as follows:

item_list:It is used to track the details of all item present in an inventory.order_list: It is used to track the details of all order placed by user.

3.3 MongoDB

MongoDB[4]is an open-source document database that provides high performance, high availability, and automatic scaling.

The MongoDatabaseAdapteris an interface that allows Bot to store statements in a MongoDB database.

These statements are the ones used by the user during a conversation. These, along with the suitable responses are stored along with a suitable degree of confidence, so that the response with the highest confidence value can be invoked if the user enters the same statement again.

3.4 Google Speech Recognition API

Google[5]has a highly efficient Speech Recognition API. This API converts spoken text (text spoken over the microphone) into written text (Python strings), briefly Speech to Text. The user can simply speak in a microphone and Google API will translate this into written text. The API has excellent results for English language, however, the accuracy of the results is dependent on the accent. The version used here gives accurate results for British English accent.

The bot needs to accept queries from the user in speech form and extract relevant details from it like the name, quantity and other specifications of the product and the action to be taken.3.5 Tkinter

Tkinter is the standard GUI library for Python and it provides various controls, such as buttons, labels and text boxes used in a GUI application. These controls are commonly called widgets.

Tkinter API has been to make interactive and user friendly GUI for the procurement bot. It is used to accept input and output the response generated by bot and is well aided with buttons, scroll bars etc.

4 Design

The overall design of our project is shown below. It shows the working of the project that is how it is taking input and how it is processed. Also, all essential adapters that are used in the project are discussed below. But first of all, lets understand the meaning of adapter. In our application, an adapter is like an API which is a set of functions that help us to complete the task given to the bot.We divided adapters into four categories: input, output, storage and logic.

Input adapters are designed to permit the bot to have a method of receiving input from a given source. The aim of designing the input adapter is to take input from a source and convert it into an acceptable format that our application can process. This format is the Statement object found in ChatterBot’s conversation module. For our convenience, we are using an adapter that takes input from terminal and process it accordingly.

Output adapters are designed to allow the bot to return a result in response to input given by the user as a Statement object. A simple adapter that allows the bot to communicate through the terminal is Output terminal adapter. It allows a user to type into their terminal to communicate with the bot.

Storage adapter is an interface that allows interacting with different storage backend. For our corpus storage, MongoDB is used. The MongoDatabaseAdapter is an interface that allows the bot to store statements in a MongoDB database.

Logic adapters are designed to determine the logic for how our bot selects a response to a given input statement. We have implemented so many adapters under this category. We will be discussing all of them below.

a) The BestMatch adapter selects a response based on the best-known match to a given statement. The best match adapter uses a function to compare the input statement to known statements. Once it finds the closest match to the input statement, it uses another function to select one of the known responses to that statement. It uses the concept of Levenstein distance.

b) The InventoryAdapter adapter will be invoked when a user has entered queries related to order such as to place an order, to review an order, to cancel an order, etc. This adapter gets to interact with databases of the bot and provide a response based on the result from the database. This adapter first creates a connection with both databases and interacted with the corresponding database accordingly. For e.g if it is regarding placing an order then it first interacts with the item_list database to check the availability of an item, if it is available, then the order is placed and corresponding order details are saved in the order_list database for future references.

c) The FindDealAdapter adapter is designed to handle the queries related to finding the best deal of any item. This adapter scraps the web with the use of BeautifulSoup API. It searches the particular item given in

query and find the best deals of that item from various websites and provide the top five suggestions as a response.

d) The ChoiceAdapter adapter helps in getting the choice entered by a user after showing the suggestions of items by the bot. The user can choose the item from the suggestions that he wants to buy. After choosing an item, firstly it will be added to the item_list database for future purchases and that item will also be ordered simultaneously along with storing the details in the order_list database.

e) The LanguageAdapter adapter parses the input or query given by a user and retrieves the quantity, name, colour of the item by using the technique of Natural Language Processing. This adapter first does some preprocessing step such as POS tagging, lemmatization and singularization. After getting relevant information about the item, it uses the above-discussed adapters to complete the task. Also, it also processes the natural queries in the same way.

5 Implementation

The application is very user friendly and interactiveand uses a GUI interface implemented in Python using Tkinter API to communicate with the user. Below are the screenshots of the procurement bot to describe its working and results.

5.1 Results

Various screens of the project are as follows:

Figure 1 Introductory screen of Bot

This is the introductory screen of the bot which guides the user about the right methodology to use the bot to gain maximum advantage from it

Figure 2 Bot processing query related to order

Figure 3 Bot processing query related to find best deals

Figure 4 Bot processing query related to fetch order details

Figure 2 is showing the use of inventory_adap.Whenever a user enters a query related to order like cancelling order, updating order, reviewing order and so on, backend program queries the database to process that query. In above example, user has asked for 4 black pens, then inventory_adap will first search database and looks for available quantity. If that much quantity is available then order will be placed successfully. Otherwise bot notify about less quantity.

Figure 3 shows the use of find_deal_adap. Here user is buying chocolates so find_deal_adap will be invoked and it searches the internet for best deals of chocolates. After getting best deals from internet, bot displays top five deals from the internet and now user can select from these options.Here, choosing an option will be handled by choice_adap, so in this example, user want fourth option so that item will be added in the inventory of items if it is not availabel and also ordered for the user.

In Figure 4, user want to know about the details of order. Here inventory_adap is used to fetch all details about that order. Queries related to reviewing order, cancelling order, updating order and so on will be handled by the same adapter.

6 Evaluation Cases

Query Appropriate Response Response of the botHelp me The bot must lucidly

explain its various features and explain to the user how to use these features.

The bot lucidly explains its various features and explain to the user how to use these features.

#place_order /<product name>/quantity/color(if any)

The bot must issue the item if it is available. Otherwise, it must notify the user of the product being unavailable.

The bot issues the item if it is available. Otherwise, it notifies the user of the product being unavailable.

#change_order /<order-id>/<new quantity>

The bot must change the entered order if the order id is valid. Else it must notify the user that the entered order id is invalid

The bot changes the entered order if the order id is valid. Else it notifies the user that the entered order id is invalid

#review_order /<order-id> The bot must display the details of entered order if the order id is valid. Else it must notify the user that the entered order id is invalid

The bot displays the details of entered order if the order id is valid. Else it notifies the user that the entered order id is invalid

#cancel_order /<order-id> The bot must cancel the entered order if the order id is valid. Else it must notify the user that the entered order id is invalid

The bot cancels the entered order if the order id is valid. Else it notifies the user that the entered order id is invalid

#check The bot must analyze the inventory and return the names of the items that are about to run out

The bot responds by indicating a possible spelling error in the query.

Check The bot must analyze the inventory and return the names of the items that are about to run out

The bot analyzes the inventory and return the names of the items that are about to run out

I want 4 pencils The bot must issue the pencil (in the required quantity) if it is available. Otherwise, it must notify the user of the product being unavailable.

The bot issues the pencil (in the required quantity) if it is available. Otherwise, it notifies the user of the product being unavailable.

I want twenty four pencils The bot must issue the pencil (in the required quantity) if it is available. Otherwise, it must notify the user of the product being unavailable.

The bot indicates the quantity as being unavailable, despite of it being there in the inventory. Thus, the bot fails in cases where the quantity is entered in words.

I need 10 black markers The bot must issue the marker (in the required

The bot issues the marker (in the required quantity) if it

quantity) if it is available. Otherwise, it must notify the user of the product being unavailable.

is available. Otherwise, it must notify the user of the product being unavailable.

I want 10 blue pens The bot must issue the marker (in the required quantity) if it is available. Otherwise, it must notify the user of the product being unavailable.

The bot indicates the quantity as being unavailable, despite of it being there in the inventory.

I want to change my last order to 4

The bot must modify the quantity in the most recent order to the one mentioned in the query

The bot modifies the quantity in the most recent order to the one mentioned in the query

Modify order 34 to 5 The bot must modify the quantity in the order with the order id mentioned to the one mentioned in the query. Else it must notify the user if no such order is available.

The bot modifies the quantity in the order with the order id mentioned to the one mentioned in the query.

Alter order 100 to 4 The bot must modify the quantity in the order with the order id mentioned to the one mentioned in the query. Else it must notify the user if no such order is available.

The bot notifies the user since there is no order with this order id

I wish to cancel my first order

The bot must cancel the first order placed by the user

The bot cancels the first order placed by the user

Delete order 32 The bot must cancel the entered order if the order id is valid. Else it must notify the user that the entered order id is invalid

The bot notifies the user that no such order is available

Review order 45 The bot must display the details of entered order if the order id is valid. Else it must notify the user that the entered order id is invalid

The bot notifies the user that no such order is available

I want to view my last order

The bot must display the details of the last order of the user

The bot displays the details of the last order of the user

Buy me some chocolates The bot must fetch the best deals of chocolate from the net and present them to the user.

The bot fetches the best deals of chocolate from the net and presents them to the user.

Find 10 best books The bot must fetch The bot fetches details of

details of 10 most suitable books from the net.

only 6 most suitable books from the net.

Can you reorder my previous order, please?

The bot must reorder the user’s last order from the inventory

The bot reorders the user’s last order from the inventory

Find the total value of order 34

The bot must find the total value of the order with the entered order id and present it to the user, if the order id is valid. Else it must notify the user that the order id is invalid

The bot redirects the user to the different ways in which he can frame the query to find the total value of an order.

Total order 34 The bot must find the total value of the order with the entered order id and present it to the user, if the order id is valid. Else it must notify the user that the order id is invalid

The bot finds the total value of order 34 and presents it to the user

What is the expected delivery date of order 32 ?

The bot must find the delivery date of the order with the entered order id and present it to the user, if the order id is valid. Else it must notify the user that the order id is invalid

The bot notifies the user that there is no order with the order id 32.

Add 5 more blue pens The bot must increment the quantity of blue pens in the inventory by 5

The bot interprets this as a total based query and tries to find out the total value of order 5, if it exists

Increment blue pens to 5 The bot must increment the quantity of blue pens in the inventory by 5

The bot runs into an error.

Find available quantity of black marker

The bot must return the available quantity of the product from the inventory to the user. If no such product is available, then the bot must notify the user

The bot returns the quantity of available black markers to the user.

Check short items The bot must analyze the inventory and return the names of the items that are about to run out

The bot analyzes the inventory and return the names of the items that are about to run out

7 Conclusion

As noted from the above sections, the bot has been implemented with all the basic functionalities expected from it. It can be used for procurement of supplies

from online suppliers, issue items from the inventory, track/review/change orders etc. It is also efficient in handling speech input however minor software changes need to be done in the speech module depending upon the region of use. As of now, it is fit for use for users with British or American English accent. It can also deliver output in audio format and it is also largely tolerant to spelling errors. The bot can process all queries typed in a specified format. It also has reasonable accuracy in dealing with natural language input but certain borderline cases still need to be integrated in the natural language processing module to ensure total accuracy. Furthermore, the user interface of the bot is interactive and can be easily construed.

8 Future Scope

The bot still needs refining in certain areas which we aim to undertake in near future. For example, the bot can be enabled to predict the demand of a product for the coming months using the data of the usage of that particular product during the past months. It can also be configured to cluster products in the database into different categories for increasing user comfort and decreasing retrieval time. The user interface can also be enhanced using Django.

References

1. Web Scraping. https://en.wikipedia.org/wiki/Web_scraping 2. The Python language. http://www.scipy-lectures.org/intro/language/python_language.html3. About SQLite. https://www.sqlite.org/about.html4. Introduction to MongoDB. https://docs.mongodb.com/manual/introduction/5. Speech Recognition using Google Speech API. https://pythonspot.com/en/speech-

recognition-using-google-speech-api/6. Tkinter. https://en.wikipedia.org/wiki/Tkinter 7. ChatterBot API. http://chatterbot.readthedocs.io/en/stable/ 8. https://www.nltk.org/