Embed Size (px)
Citation preview
Towards a Multimodal System for PrecisionAgriculture using IoT and Machine Learning
Note This paper is accepted in the 12th ICCCNT 2021 conference at IIT Kharagpur India The final version of this paper will appear in the conference proceedings
Satvik Garg1 Pradyumn Pundir1 Himanshu Jindal1 Hemraj Saini1 Somya Garg2
1Jaypee University of Information Technology Solan India2Deloitte Consulting LLP New York USA
Email satvikgarg27 pundirpradyumn25 himanshu19jgmailcom hemraj1977yahoocoin somgargdeloittecom
AbstractmdashPrecision agriculture system is an arising idea thatrefers to overseeing farms utilizing current information and com-munication technologies to improve the quantity and quality ofyields while advancing the human work required The automationrequires the assortment of information given by the sensorssuch as soil water light humidity temperature for additionalinformation to furnish the operator with exact data to acquireexcellent yield to farmers In this work a study is proposed thatincorporates all common state-of-the-art approaches for precisionagriculture use Technologies like the Internet of Things (IoT) fordata collection machine Learning for crop damage predictionand deep learning for crop disease detection is used The datacollection using IoT is responsible for the measure of moisturelevels for smart irrigation n p k estimations of fertilizers forbest yield development For crop damage prediction variousalgorithms like Random Forest (RF) Light gradient boostingmachine (LGBM) XGBoost (XGB) Decision Tree (DT) and KNearest Neighbor (KNN) are used Subsequently Pre-TrainedConvolutional Neural Network (CNN) models such as VGG16Resnet50 and DenseNet121 are also trained to check if the cropwas tainted with some illness or not
Index TermsmdashPrecision Agriculture Pre-Trained CNN Mul-timodal system Internet of Things Machine Learning
I INTRODUCTION
As per the report by the National Crime Record Bureau(NCRB) the number of farmers died in 2019-2020 is ap-proximately 10281 [1] Despite the popular image of farmerssuicide in agriculture has become more common Numerousworkers still adopt the regular methods in farming whichbrings about low yield and efficiency A study conductedby the Centre for Study of Developing Societies (CSDS)[2] found that 76 of the farmers want to quit farming Italso reports that 74 of farmers didnrsquot get basic farming-related information such as fertilizer doses from officials ofthe agriculture department Hence the precision agriculturesystem assists and helps the farmers with robotizing andupgrading them to improve rural profitability and contributeto making farming systems smart [3] The introduction of newtechnologies such as IoT-based devices would definitely havea positive impact For the most part large-scale setups arehard to execute in real-world scenarios This paper involvesusing existing advancements like for instance sensor-basedmodules that are demanding and easy to carry out
An IoT-based smart irrigation system is designed whichmaintains a balance in the water level of the field It turnsthe water pump rsquoonrsquo when the field is dry and turns it rsquooffrsquowhen it is wet Also considering one of the major problemsin todayrsquos world is water scarcity [4] this system will helpmanaging water levels and only irrigates fields when needed
Crop mainly required three macronutrients n (nitrogen) p(phosphorus) k (potassium) Shortage of nutrients can causedeficiency and affect croprsquos health Fertilizers are used toprovide nutrients to the soil it is a natural or chemicalsubstance Overuse of fertilizers can have a negative impactnot only on the crops but also on the groundwater So afertilizers dose recommendation system can help to increasecrop productivity and controls the overuse of fertilizers Usingfirebase data can be stored with help of an IoT-based fertilizersdose recommendation system One can easily monitor theresults from the previously collected data in the firebasedatabase
Deep learning is becoming popular and common sincethe introduction of ImageNet by Alex Krizhevsky et al [5]Various Pre-trained CNN models are available such as VGG16VGG19 ResNet50 InceptionV3 which can help in image-related classification tasks This work also adopts the idea oftransfer learning [6] which helps to detect the abnormalities incrop images Supervised machine learning techniques are alsoemployed to predict crop damage These models can definitelyassist farmers in the early prediction of diseases and damagesin crops or plants automatically
The idea behind presenting a paper like this was to comeup with a study that gathers all state-of-the-art approaches onthe multimodal systems for precision agriculture use Whenlooking at research papers and talking to researchers in thearea of precision agriculture we learned that there is a slightdisconnect between what researchers believe state-of-the-artand what actually is state-of-the-art Our biggest contributionfor this paper is not an approach but a study of approacheswith an aim to help fellow researchers leverage our findings topush the state-of-the-art further The following are the notablecontributions of this study
bull To the best of our knowledge this is one of the initialattempts to present beginner-friendly extensive work re-lated to precision agriculture systems
arX
iv2
107
0489
5v1
[cs
LG
] 1
0 Ju
l 202
1
bull Well-defined figures including flowcharts and circuitdiagrams are additionally included to better understandthe proposed IoT systems
bull We successfully leverage deep learning and machinelearning methods that help oversee new researchers tofollow the examination work from traditional and non-conventional approaches
bull A birdrsquos eye view on current techniques are providedthat will assist readers further push the state-of-the-artsystems
The remaining sections are compiled as follows Relatedworks where we discuss the developments made in the smartprecision-based farming system The Methodology sectioncontains the methods adopted in this project The resultsegment provides the evaluation of machine and deep learningmodels using different metrics and last is the conclusionsection
II RELATED WORKS
In this section an birdrsquos eye view of studies and researchwork is provided that has been conducted in precision-basedagriculture from all over the world
Soil is a significant part in the field of agriculture becauseoverall yield development is dependent on the soil so itssampling is needed to make further critical decisions Theprimary goal of the soil examination is to check the strengthof a land ie whether a field is nutrition deficient or notDepending upon the soil conditions and weather soil tests canbe done accordingly [7] The factors that can be determinedby analyzing soil are soil types irrigation moisture levels etcThese elements provide an overview regarding the syntheticphysical and natural status of the soil Currently there arevarious toolkits and sensors available to check the soil qualityThese toolkits help to monitor the different soil behaviour forexample water-holding limit strength and so forth A testingkit is developed by Agrocares [8] that may test up to 100 soilsamples per day Additionally distinguishing polluted soil byutilizing IoT innovations can further shields the field fromoverfertilization and yield loss [9]
Drought is also one of the serious issues that farmers faceRemote sensing is being used to deal with these problemswhich helps to analyze water content in the soil Soil Moistureand Ocean Salinity (SMOS) satellites were launched in 2009The author in [10] utilizes SMOS L2 to compute the soilwater deficit index (SWDI) In [11] the researchers useda moderate resolution imaging spectroradiometer (MODIS)to gather information about the land degradation risk In[12] sensor and a vision-based self-governing robot namedAgribot has been proposed that can help in planting seedsTo determine the seed flow rates a sensor which is equippedwith LEDs is used [13] The sign data is utilized to gauge thestream rate
About 3 of freshwater is accessible on the earth of whicharound 66 is frozen as icy masses and polar ice covers[14] In brief the whole world relies on 05 of total waterVarious traditional methods are currently employed by farmers
like drip irrigation and sprinkler which are ineffective andcrop production is badly affected by them Crop efficiencystress index (CWSI) based smart irrigation is proposed in[15] Every sensor is associated to gather the estimations andit further sends the information to the advanced calculationcentre to analyze farm data through various intelligent softwareapplications
Various IoT-based fertilizers approaches are being used toestimate the nutrient requirements like Normalized DifferenceVegetation Index (NDVI) It utilizes satellite pictures to checkcrop status [16] and is dependent on the impression of apparentand close infrared light from vegetation used to decide cropwellbeing Advances like GPS exactness [17] variable rateinnovation (VRT) [18] are added to the intelligent systemThere are other sensors that also help to gather data regardingplant health and pest situations like IoT-based automated traps[19] It is used to count and characterize insect types An IoT-based automated robot can locate and deal with pest problems
Machine and Deep learning methods have also been lever-aged by researchers for the prediction tasks like yield predic-tion object classification multimodal frameworks [20] [32]Arun Kumar et al [21] utilized ANN for regression analysis toanticipate crop yields dependent on yield efficiency Authors in[22] used time series forecasting methods to analyze and pre-dict the weather patterns Factors such as environmental soilweather and abiotic features are adopted in [23] to classify andforesee the groundnut yield using Random Forest SVM andKNN Authors in [24] accentuate the utilization of a minimalexpense UAV framework with a vision-based arrangement forthe isolation of fundamental harvests from weed It targetslessening the spread of herbicides and pesticides on crops tosafeguard their quality Strategies like harvest location are alsoutilized where the picture of the yield is specifically maskedout of the background that incorporates soil and differentitems
III METHODOLOGY
In this work we implemented the smart irrigation systemsmart fertilizer dose recommendation system crop diseasedetection system and crop damage prediction system Thissection is divided into its various subsections for detaileddiscussion
A Irrigation System
In smart irrigation method soil moisture sensor gathers theinformation of the moisture content present in soil and sends itto the Node MCU It checks the condition whether to turn onor off the water pump by a program designed in Arduino IDESoil moisture sensor gives the values in the range of ADC thatvaries from (0 to 1023) using
AnalogOutput = ADCV alue1023 (1)
Moisture = 100minus (Analogoutput lowast 100) (2)
The proposed flow of events is given in figure 1 Generallymoisture below 50 is considered to be dry conditions which
Fig 1 Proposed flow of events smart irrigation system
Fig 2 Web Portal for Irrigation system
suggests the need to irrigate the field and above 50 to bewet If the value of the moisture content received from soilmoisture sensor is less than the threshold value of moisturecontent required for the crop it means that there is a needto irrigate the field Node MCU sends a signal to the relaymodule to turn on the water pump which results in an increasein the water levels in the soil until it reaches the thresholdvalue of moisture required for the crop Once it reaches thethreshold value the relay module will automatically turn offthe water pump Node MCU also sends data to the firebasedatabase Firebase provides an online database to store datathat can be extracted by the web portal It can easily monitorthe information about the moisture content of the field Theweb portal also shows the information whether the water pumpis on or off We can also manually turned on and off the waterpump through the web portal as shown in figure 2
The circuit diagram for smart irrigation system is given inFigure 3 The SIG pin of moisture sensor provides the analogsignals that is attached to A0 pin of the Node MCU and readsthe analog values VCC pin of the soil moisture sensor isconnected to 3v3 pin of Node MCU for the power supplyThe ground pins of both are connected with each other to useit as (0V) reference to all other electronic parts Motor pumpis attached to the NodeMCU with ground and a Vin pin isusedIn the Relay module the Node MCU connection withthe ground pins of both the devices are connected with eachother Relay module power is connected with D1 pin of theNode MCU and provides the data from the relay module and
its signal pin is connected to Vin of the Node MCU for thepower supply
Fig 3 Circuit Diagram Smart Irrigation System
B Fertilizer System
In smart fertilizer dose recommendation system as shownin figure 4 we first dug an n-p-k sensor into the soil to gatherthe data on the nitrogen phosphorus potassium content of thesoil The n-p-k sensor sends the n-p-k values to the NodeMCUwith the help of max 485 modbus NodeMCU sends the datato the firebase through a program design in Arduino IDE Thevalues can be easily accessed by a web portal given in figure 5to perform basic calculations that helps suggesting the quantityof fertilizers for a specific crop type For instance the n-p-kvalue of the soil given by the n-p-k sensor is (10 5 10 kgha)and the requirement is to grow wheat on this land The n-p-k requirement of wheat is (1002060 kgha) The target isto calculate the doses of fertilizers through urea muriate ofpotash (MOP) and decomposed organic phosphorus (DOP)The difference in n-p-k values is (90 15 50) which impliesthat we need 90 kg of nitrogen 15 kg of phosphorus and 50kgof potassium
Fig 4 Proposed flow of events smart fertilizer system
MOP = 60potassium (3)
Urea = 46nitrogen (4)
DAP = (18nitrogen+ 46phosphorus) (5)
From 3 4 and 5 we get1) 100 kg MOP rarr 60kg Potassium2) 50kg potassium rarr (10060) 50 of MOP = 83 kg of
MOP3) 100 kg DOP rarr 18kg of nitrogen and 46kg of phospho-
rus4) 15kg phosphorus rarr (10046) 15 of DOP = 326kg of
DOP5) DOP also has Nitrogen 326kg of DOP rarr (18100)
326 nitrogen = 586kg of nitrogen6) Now we require (90 - 586) kg of nitrogen = 84kg of
nitrogen7) 100kg of urea rarr 46kg of Nitrogen8) 84kg of nitrogen rarr (10046) 84kg of Urea = 1826
kg of ureaFor better wheat productivity we need 83kg of MOP 326kgof DOP and 1826 kg of urea on this type of soil
Fig 5 Web Portal for Fertilizer Recommendation
The circuit diagram is presented in Figure 6 R0 and d1 pinsof Modbus are connected to d2 and d3 of node MCU whichsends the data collected by the n-p-k sensor to node MCU viad2 and d3 pins The brown wire is VCC which needs a 9v-24vpower supply The Ground pin of NodeMCU is connected tothe Ground pin of Modbus (black Wire) Ground pin maintainsa reference level to all the other IOT connections (ie 0v) Bluewire which is the B pin is connected to the B pin of MAX485and Yellow wire is pin A connected to A pin of MAX485which sends data from n-p-k to modbus
C Crop Disease Detection System
In crop disease detection we employed plant village dataset[25] containing approximately 54000 images distributed in38 classes In this work images of only six crops namelypotatoes tomatoes corn peach apple grapes are utilized forclassification The dataset distibution of crop types used in thiswork is given in Table I The images originally are of size256256 and are cropped to 6464 An image augmentation
Fig 6 Circuit Diagram smart fertilizer system
pipeline using Image Data Generator is used to automaticallyaugment the original images incorporating methods like fliprotate zoom in zoom out blur rotate etc The augmentationis only performed for the training set to increase the accuracyand to avoid data leakage problems Prior augmentation theimages were divided using the train test split method Weperformed a standard 602020 split in which 60 percent oftotal images belong to the train set and 20 percent to the testset The validation set is also created to help train the modeleffectively which constitutes 20 percent of the total images
TABLE IDISTRIBUTION OF IMAGES
Crop Type Classes Images Train Validation TestTomato 10 18160 10896 3632 3632Potato 3 2152 1291 431 430Apple 4 3171 1902 635 634Peach 2 2657 1594 532 531Grapes 4 4062 2437 813 812Corn 4 3852 2371 771 770
The concept of transfer learning is used in this project totransfer the weight of the already trained models ie Pre-Trained CNN (Convolutional Neural Network) models [6] Ithelps in reducing the computational power and speeds up theperformance which shows indications of quicker and improvedoutcomes In this study we utilize only three models namelyResNet50 VGG16 and DenseNet121 [26] [27] [28] forcomparative analysis Our proposed pipeline for preparing themodel contains three areas
bull Feature Generation - Separating the main features bycalibrating the models through preparing the three Stateof the Art (SOTA) Pre-trained CNN structures Theextracted features after this phase act as an input to themodification phase
bull Modification - Incorporated a concat layer which is aconcatenation of features of three layers namely Max-Pooling2D AveragePooling2D flatten layer In additiona dropout layer with a dropout rate of 05 has additionallybeen fused
bull Output - The output generated from the modificationphase goes to the dense layer The activation function
used is sigmoid For compilation of model adam opti-mizer is used with a learning rate of 00002 The batchsize is 32 and the model is trained for 30 epochs
D Crop Damage Prediction System
In crop damage prediction the task is to use and employfeature engineering concepts with machine learning algorithmsto predict the category of crop damage We used rsquomachinelearning in agriculturersquo dataset from the Analytics Vidhyawebsite [33] This dataset has 88858 rows and 10 columnswhich is divided into 7525 using train test split The descrip-tion of the features are given in Figure 7
Fig 7 Dataset Description
It is important to clean and visualize data before imple-menting machine learning algorithms This dataset initiallycontains null values replaced by -999 using fillna method Italso contains columns like Estimated insects counts as shownin Figure 8 with some time series related pattern that suggeststo generate features containing different lag values using shiftand rolling method A window size of 5 is used to extract themean of five observations with a lag of 1 and 2
Fig 8 Estimated Insects Count
We used five machine learning algorithms namely RandomForest LGBM KNN Decision Tree and XGBoost [29][30] [31] [9] for classification Decision Trees is a simplesupervised learning algorithm built on the concepts of treesThe tree has two components namely leaves and nodes Theleaves are the choices or the ultimate results whereas the nodesrepresent the points where the data is divided according to afixed parameter for further classification Random Forest is anensemble-based supervised machine learning algorithm Thebagging method is used to train the ensemble of decision treeswhich help to build the forest In simple terms Random Forestassembles numerous decision trees and combines them to get amore exact and stable expectation K-nearest neighbors (KNN)
depends on the possibility of the likeness of similarities Itpredicts the values of new points dependent on how closelythe new information is identified with the upsides of thegiven training set The KNN doesnrsquot make any assumptionsidentified with the preparation information and it for the mostpart inputs all the data values for training
Extreme gradient boosting (XGBoost) and light gradientboosting (LGBM) are based on the concept of gradient boost-ing This technique leverage weak models utilizing ensemblemethods that can develop new models to decrease the lossfunction estimated by gradient descent However there isa subtle difference these two models in terms of split ForXGBoost histogram based filters are used while for LGBMgradient-based one-side sampling (GSOS) method is used todistinguish the best split It has been observed that histogrambased filters technique is computationally costly comparedwith GSOS which recommends that LGBM is more effectivethan XGBoost
E Metrics
In this research we primarily perform two prediction tasksto classify crop damages and disease detection Since both fallunder the category of classification the models are evaluatedby metrics namely precision recall f1 score and accuracy
Precision =Tp
Tp + Fp(6)
Recall =Tp
Tp + Fn(7)
Accuracy =Tp + Tn
Tp + Tn + Fp + Fn(8)
F1score = 2P recisionRecall
Precision+Recall(9)
where Tp Tn Fp Fn represents True positive True negativeFalse positive False negative
IV RESULTS
Table II shows the evaluation of metrics for crop diseasedetection We performed the disease classification on six cropstomato potato apple peach grapes and xorn using threePre-trained CNN models VGG16 ResNet50 Densenet121All models performed well and there is a slight differencebetween the accuracy of the models For tomato corn andpeach Densenet121 outperforms other models in terms of allevaluation metrics VGG16 also performed state of the artresults for potato apple and grapes However it marginallyunderperformed in classification for peach and tomato Theresults forecasted from ResNet50 manifests that aside fromclass apple it failed to outperform other models For betterinterpretation and visualization bar plot is pictured in figure9 showing precision of models on given classes
Analysis of evaluation metrics are also discussed for cropdamage prediction As we can see from Table III the high-est accuracy achieved is 94 by LGBM and it completelybeats other algorithms Random Forest and XGBoost have
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
bull Well-defined figures including flowcharts and circuitdiagrams are additionally included to better understandthe proposed IoT systems
bull We successfully leverage deep learning and machinelearning methods that help oversee new researchers tofollow the examination work from traditional and non-conventional approaches
bull A birdrsquos eye view on current techniques are providedthat will assist readers further push the state-of-the-artsystems
The remaining sections are compiled as follows Relatedworks where we discuss the developments made in the smartprecision-based farming system The Methodology sectioncontains the methods adopted in this project The resultsegment provides the evaluation of machine and deep learningmodels using different metrics and last is the conclusionsection
II RELATED WORKS
In this section an birdrsquos eye view of studies and researchwork is provided that has been conducted in precision-basedagriculture from all over the world
Soil is a significant part in the field of agriculture becauseoverall yield development is dependent on the soil so itssampling is needed to make further critical decisions Theprimary goal of the soil examination is to check the strengthof a land ie whether a field is nutrition deficient or notDepending upon the soil conditions and weather soil tests canbe done accordingly [7] The factors that can be determinedby analyzing soil are soil types irrigation moisture levels etcThese elements provide an overview regarding the syntheticphysical and natural status of the soil Currently there arevarious toolkits and sensors available to check the soil qualityThese toolkits help to monitor the different soil behaviour forexample water-holding limit strength and so forth A testingkit is developed by Agrocares [8] that may test up to 100 soilsamples per day Additionally distinguishing polluted soil byutilizing IoT innovations can further shields the field fromoverfertilization and yield loss [9]
Drought is also one of the serious issues that farmers faceRemote sensing is being used to deal with these problemswhich helps to analyze water content in the soil Soil Moistureand Ocean Salinity (SMOS) satellites were launched in 2009The author in [10] utilizes SMOS L2 to compute the soilwater deficit index (SWDI) In [11] the researchers useda moderate resolution imaging spectroradiometer (MODIS)to gather information about the land degradation risk In[12] sensor and a vision-based self-governing robot namedAgribot has been proposed that can help in planting seedsTo determine the seed flow rates a sensor which is equippedwith LEDs is used [13] The sign data is utilized to gauge thestream rate
About 3 of freshwater is accessible on the earth of whicharound 66 is frozen as icy masses and polar ice covers[14] In brief the whole world relies on 05 of total waterVarious traditional methods are currently employed by farmers
like drip irrigation and sprinkler which are ineffective andcrop production is badly affected by them Crop efficiencystress index (CWSI) based smart irrigation is proposed in[15] Every sensor is associated to gather the estimations andit further sends the information to the advanced calculationcentre to analyze farm data through various intelligent softwareapplications
Various IoT-based fertilizers approaches are being used toestimate the nutrient requirements like Normalized DifferenceVegetation Index (NDVI) It utilizes satellite pictures to checkcrop status [16] and is dependent on the impression of apparentand close infrared light from vegetation used to decide cropwellbeing Advances like GPS exactness [17] variable rateinnovation (VRT) [18] are added to the intelligent systemThere are other sensors that also help to gather data regardingplant health and pest situations like IoT-based automated traps[19] It is used to count and characterize insect types An IoT-based automated robot can locate and deal with pest problems
Machine and Deep learning methods have also been lever-aged by researchers for the prediction tasks like yield predic-tion object classification multimodal frameworks [20] [32]Arun Kumar et al [21] utilized ANN for regression analysis toanticipate crop yields dependent on yield efficiency Authors in[22] used time series forecasting methods to analyze and pre-dict the weather patterns Factors such as environmental soilweather and abiotic features are adopted in [23] to classify andforesee the groundnut yield using Random Forest SVM andKNN Authors in [24] accentuate the utilization of a minimalexpense UAV framework with a vision-based arrangement forthe isolation of fundamental harvests from weed It targetslessening the spread of herbicides and pesticides on crops tosafeguard their quality Strategies like harvest location are alsoutilized where the picture of the yield is specifically maskedout of the background that incorporates soil and differentitems
III METHODOLOGY
In this work we implemented the smart irrigation systemsmart fertilizer dose recommendation system crop diseasedetection system and crop damage prediction system Thissection is divided into its various subsections for detaileddiscussion
A Irrigation System
In smart irrigation method soil moisture sensor gathers theinformation of the moisture content present in soil and sends itto the Node MCU It checks the condition whether to turn onor off the water pump by a program designed in Arduino IDESoil moisture sensor gives the values in the range of ADC thatvaries from (0 to 1023) using
AnalogOutput = ADCV alue1023 (1)
Moisture = 100minus (Analogoutput lowast 100) (2)
The proposed flow of events is given in figure 1 Generallymoisture below 50 is considered to be dry conditions which
Fig 1 Proposed flow of events smart irrigation system
Fig 2 Web Portal for Irrigation system
suggests the need to irrigate the field and above 50 to bewet If the value of the moisture content received from soilmoisture sensor is less than the threshold value of moisturecontent required for the crop it means that there is a needto irrigate the field Node MCU sends a signal to the relaymodule to turn on the water pump which results in an increasein the water levels in the soil until it reaches the thresholdvalue of moisture required for the crop Once it reaches thethreshold value the relay module will automatically turn offthe water pump Node MCU also sends data to the firebasedatabase Firebase provides an online database to store datathat can be extracted by the web portal It can easily monitorthe information about the moisture content of the field Theweb portal also shows the information whether the water pumpis on or off We can also manually turned on and off the waterpump through the web portal as shown in figure 2
The circuit diagram for smart irrigation system is given inFigure 3 The SIG pin of moisture sensor provides the analogsignals that is attached to A0 pin of the Node MCU and readsthe analog values VCC pin of the soil moisture sensor isconnected to 3v3 pin of Node MCU for the power supplyThe ground pins of both are connected with each other to useit as (0V) reference to all other electronic parts Motor pumpis attached to the NodeMCU with ground and a Vin pin isusedIn the Relay module the Node MCU connection withthe ground pins of both the devices are connected with eachother Relay module power is connected with D1 pin of theNode MCU and provides the data from the relay module and
its signal pin is connected to Vin of the Node MCU for thepower supply
Fig 3 Circuit Diagram Smart Irrigation System
B Fertilizer System
In smart fertilizer dose recommendation system as shownin figure 4 we first dug an n-p-k sensor into the soil to gatherthe data on the nitrogen phosphorus potassium content of thesoil The n-p-k sensor sends the n-p-k values to the NodeMCUwith the help of max 485 modbus NodeMCU sends the datato the firebase through a program design in Arduino IDE Thevalues can be easily accessed by a web portal given in figure 5to perform basic calculations that helps suggesting the quantityof fertilizers for a specific crop type For instance the n-p-kvalue of the soil given by the n-p-k sensor is (10 5 10 kgha)and the requirement is to grow wheat on this land The n-p-k requirement of wheat is (1002060 kgha) The target isto calculate the doses of fertilizers through urea muriate ofpotash (MOP) and decomposed organic phosphorus (DOP)The difference in n-p-k values is (90 15 50) which impliesthat we need 90 kg of nitrogen 15 kg of phosphorus and 50kgof potassium
Fig 4 Proposed flow of events smart fertilizer system
MOP = 60potassium (3)
Urea = 46nitrogen (4)
DAP = (18nitrogen+ 46phosphorus) (5)
From 3 4 and 5 we get1) 100 kg MOP rarr 60kg Potassium2) 50kg potassium rarr (10060) 50 of MOP = 83 kg of
MOP3) 100 kg DOP rarr 18kg of nitrogen and 46kg of phospho-
rus4) 15kg phosphorus rarr (10046) 15 of DOP = 326kg of
DOP5) DOP also has Nitrogen 326kg of DOP rarr (18100)
326 nitrogen = 586kg of nitrogen6) Now we require (90 - 586) kg of nitrogen = 84kg of
nitrogen7) 100kg of urea rarr 46kg of Nitrogen8) 84kg of nitrogen rarr (10046) 84kg of Urea = 1826
kg of ureaFor better wheat productivity we need 83kg of MOP 326kgof DOP and 1826 kg of urea on this type of soil
Fig 5 Web Portal for Fertilizer Recommendation
The circuit diagram is presented in Figure 6 R0 and d1 pinsof Modbus are connected to d2 and d3 of node MCU whichsends the data collected by the n-p-k sensor to node MCU viad2 and d3 pins The brown wire is VCC which needs a 9v-24vpower supply The Ground pin of NodeMCU is connected tothe Ground pin of Modbus (black Wire) Ground pin maintainsa reference level to all the other IOT connections (ie 0v) Bluewire which is the B pin is connected to the B pin of MAX485and Yellow wire is pin A connected to A pin of MAX485which sends data from n-p-k to modbus
C Crop Disease Detection System
In crop disease detection we employed plant village dataset[25] containing approximately 54000 images distributed in38 classes In this work images of only six crops namelypotatoes tomatoes corn peach apple grapes are utilized forclassification The dataset distibution of crop types used in thiswork is given in Table I The images originally are of size256256 and are cropped to 6464 An image augmentation
Fig 6 Circuit Diagram smart fertilizer system
pipeline using Image Data Generator is used to automaticallyaugment the original images incorporating methods like fliprotate zoom in zoom out blur rotate etc The augmentationis only performed for the training set to increase the accuracyand to avoid data leakage problems Prior augmentation theimages were divided using the train test split method Weperformed a standard 602020 split in which 60 percent oftotal images belong to the train set and 20 percent to the testset The validation set is also created to help train the modeleffectively which constitutes 20 percent of the total images
TABLE IDISTRIBUTION OF IMAGES
Crop Type Classes Images Train Validation TestTomato 10 18160 10896 3632 3632Potato 3 2152 1291 431 430Apple 4 3171 1902 635 634Peach 2 2657 1594 532 531Grapes 4 4062 2437 813 812Corn 4 3852 2371 771 770
The concept of transfer learning is used in this project totransfer the weight of the already trained models ie Pre-Trained CNN (Convolutional Neural Network) models [6] Ithelps in reducing the computational power and speeds up theperformance which shows indications of quicker and improvedoutcomes In this study we utilize only three models namelyResNet50 VGG16 and DenseNet121 [26] [27] [28] forcomparative analysis Our proposed pipeline for preparing themodel contains three areas
bull Feature Generation - Separating the main features bycalibrating the models through preparing the three Stateof the Art (SOTA) Pre-trained CNN structures Theextracted features after this phase act as an input to themodification phase
bull Modification - Incorporated a concat layer which is aconcatenation of features of three layers namely Max-Pooling2D AveragePooling2D flatten layer In additiona dropout layer with a dropout rate of 05 has additionallybeen fused
bull Output - The output generated from the modificationphase goes to the dense layer The activation function
used is sigmoid For compilation of model adam opti-mizer is used with a learning rate of 00002 The batchsize is 32 and the model is trained for 30 epochs
D Crop Damage Prediction System
In crop damage prediction the task is to use and employfeature engineering concepts with machine learning algorithmsto predict the category of crop damage We used rsquomachinelearning in agriculturersquo dataset from the Analytics Vidhyawebsite [33] This dataset has 88858 rows and 10 columnswhich is divided into 7525 using train test split The descrip-tion of the features are given in Figure 7
Fig 7 Dataset Description
It is important to clean and visualize data before imple-menting machine learning algorithms This dataset initiallycontains null values replaced by -999 using fillna method Italso contains columns like Estimated insects counts as shownin Figure 8 with some time series related pattern that suggeststo generate features containing different lag values using shiftand rolling method A window size of 5 is used to extract themean of five observations with a lag of 1 and 2
Fig 8 Estimated Insects Count
We used five machine learning algorithms namely RandomForest LGBM KNN Decision Tree and XGBoost [29][30] [31] [9] for classification Decision Trees is a simplesupervised learning algorithm built on the concepts of treesThe tree has two components namely leaves and nodes Theleaves are the choices or the ultimate results whereas the nodesrepresent the points where the data is divided according to afixed parameter for further classification Random Forest is anensemble-based supervised machine learning algorithm Thebagging method is used to train the ensemble of decision treeswhich help to build the forest In simple terms Random Forestassembles numerous decision trees and combines them to get amore exact and stable expectation K-nearest neighbors (KNN)
depends on the possibility of the likeness of similarities Itpredicts the values of new points dependent on how closelythe new information is identified with the upsides of thegiven training set The KNN doesnrsquot make any assumptionsidentified with the preparation information and it for the mostpart inputs all the data values for training
Extreme gradient boosting (XGBoost) and light gradientboosting (LGBM) are based on the concept of gradient boost-ing This technique leverage weak models utilizing ensemblemethods that can develop new models to decrease the lossfunction estimated by gradient descent However there isa subtle difference these two models in terms of split ForXGBoost histogram based filters are used while for LGBMgradient-based one-side sampling (GSOS) method is used todistinguish the best split It has been observed that histogrambased filters technique is computationally costly comparedwith GSOS which recommends that LGBM is more effectivethan XGBoost
E Metrics
In this research we primarily perform two prediction tasksto classify crop damages and disease detection Since both fallunder the category of classification the models are evaluatedby metrics namely precision recall f1 score and accuracy
Precision =Tp
Tp + Fp(6)
Recall =Tp
Tp + Fn(7)
Accuracy =Tp + Tn
Tp + Tn + Fp + Fn(8)
F1score = 2P recisionRecall
Precision+Recall(9)
where Tp Tn Fp Fn represents True positive True negativeFalse positive False negative
IV RESULTS
Table II shows the evaluation of metrics for crop diseasedetection We performed the disease classification on six cropstomato potato apple peach grapes and xorn using threePre-trained CNN models VGG16 ResNet50 Densenet121All models performed well and there is a slight differencebetween the accuracy of the models For tomato corn andpeach Densenet121 outperforms other models in terms of allevaluation metrics VGG16 also performed state of the artresults for potato apple and grapes However it marginallyunderperformed in classification for peach and tomato Theresults forecasted from ResNet50 manifests that aside fromclass apple it failed to outperform other models For betterinterpretation and visualization bar plot is pictured in figure9 showing precision of models on given classes
Analysis of evaluation metrics are also discussed for cropdamage prediction As we can see from Table III the high-est accuracy achieved is 94 by LGBM and it completelybeats other algorithms Random Forest and XGBoost have
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
Fig 1 Proposed flow of events smart irrigation system
Fig 2 Web Portal for Irrigation system
suggests the need to irrigate the field and above 50 to bewet If the value of the moisture content received from soilmoisture sensor is less than the threshold value of moisturecontent required for the crop it means that there is a needto irrigate the field Node MCU sends a signal to the relaymodule to turn on the water pump which results in an increasein the water levels in the soil until it reaches the thresholdvalue of moisture required for the crop Once it reaches thethreshold value the relay module will automatically turn offthe water pump Node MCU also sends data to the firebasedatabase Firebase provides an online database to store datathat can be extracted by the web portal It can easily monitorthe information about the moisture content of the field Theweb portal also shows the information whether the water pumpis on or off We can also manually turned on and off the waterpump through the web portal as shown in figure 2
The circuit diagram for smart irrigation system is given inFigure 3 The SIG pin of moisture sensor provides the analogsignals that is attached to A0 pin of the Node MCU and readsthe analog values VCC pin of the soil moisture sensor isconnected to 3v3 pin of Node MCU for the power supplyThe ground pins of both are connected with each other to useit as (0V) reference to all other electronic parts Motor pumpis attached to the NodeMCU with ground and a Vin pin isusedIn the Relay module the Node MCU connection withthe ground pins of both the devices are connected with eachother Relay module power is connected with D1 pin of theNode MCU and provides the data from the relay module and
its signal pin is connected to Vin of the Node MCU for thepower supply
Fig 3 Circuit Diagram Smart Irrigation System
B Fertilizer System
In smart fertilizer dose recommendation system as shownin figure 4 we first dug an n-p-k sensor into the soil to gatherthe data on the nitrogen phosphorus potassium content of thesoil The n-p-k sensor sends the n-p-k values to the NodeMCUwith the help of max 485 modbus NodeMCU sends the datato the firebase through a program design in Arduino IDE Thevalues can be easily accessed by a web portal given in figure 5to perform basic calculations that helps suggesting the quantityof fertilizers for a specific crop type For instance the n-p-kvalue of the soil given by the n-p-k sensor is (10 5 10 kgha)and the requirement is to grow wheat on this land The n-p-k requirement of wheat is (1002060 kgha) The target isto calculate the doses of fertilizers through urea muriate ofpotash (MOP) and decomposed organic phosphorus (DOP)The difference in n-p-k values is (90 15 50) which impliesthat we need 90 kg of nitrogen 15 kg of phosphorus and 50kgof potassium
Fig 4 Proposed flow of events smart fertilizer system
MOP = 60potassium (3)
Urea = 46nitrogen (4)
DAP = (18nitrogen+ 46phosphorus) (5)
From 3 4 and 5 we get1) 100 kg MOP rarr 60kg Potassium2) 50kg potassium rarr (10060) 50 of MOP = 83 kg of
MOP3) 100 kg DOP rarr 18kg of nitrogen and 46kg of phospho-
rus4) 15kg phosphorus rarr (10046) 15 of DOP = 326kg of
DOP5) DOP also has Nitrogen 326kg of DOP rarr (18100)
326 nitrogen = 586kg of nitrogen6) Now we require (90 - 586) kg of nitrogen = 84kg of
nitrogen7) 100kg of urea rarr 46kg of Nitrogen8) 84kg of nitrogen rarr (10046) 84kg of Urea = 1826
kg of ureaFor better wheat productivity we need 83kg of MOP 326kgof DOP and 1826 kg of urea on this type of soil
Fig 5 Web Portal for Fertilizer Recommendation
The circuit diagram is presented in Figure 6 R0 and d1 pinsof Modbus are connected to d2 and d3 of node MCU whichsends the data collected by the n-p-k sensor to node MCU viad2 and d3 pins The brown wire is VCC which needs a 9v-24vpower supply The Ground pin of NodeMCU is connected tothe Ground pin of Modbus (black Wire) Ground pin maintainsa reference level to all the other IOT connections (ie 0v) Bluewire which is the B pin is connected to the B pin of MAX485and Yellow wire is pin A connected to A pin of MAX485which sends data from n-p-k to modbus
C Crop Disease Detection System
In crop disease detection we employed plant village dataset[25] containing approximately 54000 images distributed in38 classes In this work images of only six crops namelypotatoes tomatoes corn peach apple grapes are utilized forclassification The dataset distibution of crop types used in thiswork is given in Table I The images originally are of size256256 and are cropped to 6464 An image augmentation
Fig 6 Circuit Diagram smart fertilizer system
pipeline using Image Data Generator is used to automaticallyaugment the original images incorporating methods like fliprotate zoom in zoom out blur rotate etc The augmentationis only performed for the training set to increase the accuracyand to avoid data leakage problems Prior augmentation theimages were divided using the train test split method Weperformed a standard 602020 split in which 60 percent oftotal images belong to the train set and 20 percent to the testset The validation set is also created to help train the modeleffectively which constitutes 20 percent of the total images
TABLE IDISTRIBUTION OF IMAGES
Crop Type Classes Images Train Validation TestTomato 10 18160 10896 3632 3632Potato 3 2152 1291 431 430Apple 4 3171 1902 635 634Peach 2 2657 1594 532 531Grapes 4 4062 2437 813 812Corn 4 3852 2371 771 770
The concept of transfer learning is used in this project totransfer the weight of the already trained models ie Pre-Trained CNN (Convolutional Neural Network) models [6] Ithelps in reducing the computational power and speeds up theperformance which shows indications of quicker and improvedoutcomes In this study we utilize only three models namelyResNet50 VGG16 and DenseNet121 [26] [27] [28] forcomparative analysis Our proposed pipeline for preparing themodel contains three areas
bull Feature Generation - Separating the main features bycalibrating the models through preparing the three Stateof the Art (SOTA) Pre-trained CNN structures Theextracted features after this phase act as an input to themodification phase
bull Modification - Incorporated a concat layer which is aconcatenation of features of three layers namely Max-Pooling2D AveragePooling2D flatten layer In additiona dropout layer with a dropout rate of 05 has additionallybeen fused
bull Output - The output generated from the modificationphase goes to the dense layer The activation function
used is sigmoid For compilation of model adam opti-mizer is used with a learning rate of 00002 The batchsize is 32 and the model is trained for 30 epochs
D Crop Damage Prediction System
In crop damage prediction the task is to use and employfeature engineering concepts with machine learning algorithmsto predict the category of crop damage We used rsquomachinelearning in agriculturersquo dataset from the Analytics Vidhyawebsite [33] This dataset has 88858 rows and 10 columnswhich is divided into 7525 using train test split The descrip-tion of the features are given in Figure 7
Fig 7 Dataset Description
It is important to clean and visualize data before imple-menting machine learning algorithms This dataset initiallycontains null values replaced by -999 using fillna method Italso contains columns like Estimated insects counts as shownin Figure 8 with some time series related pattern that suggeststo generate features containing different lag values using shiftand rolling method A window size of 5 is used to extract themean of five observations with a lag of 1 and 2
Fig 8 Estimated Insects Count
We used five machine learning algorithms namely RandomForest LGBM KNN Decision Tree and XGBoost [29][30] [31] [9] for classification Decision Trees is a simplesupervised learning algorithm built on the concepts of treesThe tree has two components namely leaves and nodes Theleaves are the choices or the ultimate results whereas the nodesrepresent the points where the data is divided according to afixed parameter for further classification Random Forest is anensemble-based supervised machine learning algorithm Thebagging method is used to train the ensemble of decision treeswhich help to build the forest In simple terms Random Forestassembles numerous decision trees and combines them to get amore exact and stable expectation K-nearest neighbors (KNN)
depends on the possibility of the likeness of similarities Itpredicts the values of new points dependent on how closelythe new information is identified with the upsides of thegiven training set The KNN doesnrsquot make any assumptionsidentified with the preparation information and it for the mostpart inputs all the data values for training
Extreme gradient boosting (XGBoost) and light gradientboosting (LGBM) are based on the concept of gradient boost-ing This technique leverage weak models utilizing ensemblemethods that can develop new models to decrease the lossfunction estimated by gradient descent However there isa subtle difference these two models in terms of split ForXGBoost histogram based filters are used while for LGBMgradient-based one-side sampling (GSOS) method is used todistinguish the best split It has been observed that histogrambased filters technique is computationally costly comparedwith GSOS which recommends that LGBM is more effectivethan XGBoost
E Metrics
In this research we primarily perform two prediction tasksto classify crop damages and disease detection Since both fallunder the category of classification the models are evaluatedby metrics namely precision recall f1 score and accuracy
Precision =Tp
Tp + Fp(6)
Recall =Tp
Tp + Fn(7)
Accuracy =Tp + Tn
Tp + Tn + Fp + Fn(8)
F1score = 2P recisionRecall
Precision+Recall(9)
where Tp Tn Fp Fn represents True positive True negativeFalse positive False negative
IV RESULTS
Table II shows the evaluation of metrics for crop diseasedetection We performed the disease classification on six cropstomato potato apple peach grapes and xorn using threePre-trained CNN models VGG16 ResNet50 Densenet121All models performed well and there is a slight differencebetween the accuracy of the models For tomato corn andpeach Densenet121 outperforms other models in terms of allevaluation metrics VGG16 also performed state of the artresults for potato apple and grapes However it marginallyunderperformed in classification for peach and tomato Theresults forecasted from ResNet50 manifests that aside fromclass apple it failed to outperform other models For betterinterpretation and visualization bar plot is pictured in figure9 showing precision of models on given classes
Analysis of evaluation metrics are also discussed for cropdamage prediction As we can see from Table III the high-est accuracy achieved is 94 by LGBM and it completelybeats other algorithms Random Forest and XGBoost have
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
DAP = (18nitrogen+ 46phosphorus) (5)
From 3 4 and 5 we get1) 100 kg MOP rarr 60kg Potassium2) 50kg potassium rarr (10060) 50 of MOP = 83 kg of
MOP3) 100 kg DOP rarr 18kg of nitrogen and 46kg of phospho-
rus4) 15kg phosphorus rarr (10046) 15 of DOP = 326kg of
DOP5) DOP also has Nitrogen 326kg of DOP rarr (18100)
326 nitrogen = 586kg of nitrogen6) Now we require (90 - 586) kg of nitrogen = 84kg of
nitrogen7) 100kg of urea rarr 46kg of Nitrogen8) 84kg of nitrogen rarr (10046) 84kg of Urea = 1826
kg of ureaFor better wheat productivity we need 83kg of MOP 326kgof DOP and 1826 kg of urea on this type of soil
Fig 5 Web Portal for Fertilizer Recommendation
The circuit diagram is presented in Figure 6 R0 and d1 pinsof Modbus are connected to d2 and d3 of node MCU whichsends the data collected by the n-p-k sensor to node MCU viad2 and d3 pins The brown wire is VCC which needs a 9v-24vpower supply The Ground pin of NodeMCU is connected tothe Ground pin of Modbus (black Wire) Ground pin maintainsa reference level to all the other IOT connections (ie 0v) Bluewire which is the B pin is connected to the B pin of MAX485and Yellow wire is pin A connected to A pin of MAX485which sends data from n-p-k to modbus
C Crop Disease Detection System
In crop disease detection we employed plant village dataset[25] containing approximately 54000 images distributed in38 classes In this work images of only six crops namelypotatoes tomatoes corn peach apple grapes are utilized forclassification The dataset distibution of crop types used in thiswork is given in Table I The images originally are of size256256 and are cropped to 6464 An image augmentation
Fig 6 Circuit Diagram smart fertilizer system
pipeline using Image Data Generator is used to automaticallyaugment the original images incorporating methods like fliprotate zoom in zoom out blur rotate etc The augmentationis only performed for the training set to increase the accuracyand to avoid data leakage problems Prior augmentation theimages were divided using the train test split method Weperformed a standard 602020 split in which 60 percent oftotal images belong to the train set and 20 percent to the testset The validation set is also created to help train the modeleffectively which constitutes 20 percent of the total images
TABLE IDISTRIBUTION OF IMAGES
Crop Type Classes Images Train Validation TestTomato 10 18160 10896 3632 3632Potato 3 2152 1291 431 430Apple 4 3171 1902 635 634Peach 2 2657 1594 532 531Grapes 4 4062 2437 813 812Corn 4 3852 2371 771 770
The concept of transfer learning is used in this project totransfer the weight of the already trained models ie Pre-Trained CNN (Convolutional Neural Network) models [6] Ithelps in reducing the computational power and speeds up theperformance which shows indications of quicker and improvedoutcomes In this study we utilize only three models namelyResNet50 VGG16 and DenseNet121 [26] [27] [28] forcomparative analysis Our proposed pipeline for preparing themodel contains three areas
bull Feature Generation - Separating the main features bycalibrating the models through preparing the three Stateof the Art (SOTA) Pre-trained CNN structures Theextracted features after this phase act as an input to themodification phase
bull Modification - Incorporated a concat layer which is aconcatenation of features of three layers namely Max-Pooling2D AveragePooling2D flatten layer In additiona dropout layer with a dropout rate of 05 has additionallybeen fused
bull Output - The output generated from the modificationphase goes to the dense layer The activation function
used is sigmoid For compilation of model adam opti-mizer is used with a learning rate of 00002 The batchsize is 32 and the model is trained for 30 epochs
D Crop Damage Prediction System
In crop damage prediction the task is to use and employfeature engineering concepts with machine learning algorithmsto predict the category of crop damage We used rsquomachinelearning in agriculturersquo dataset from the Analytics Vidhyawebsite [33] This dataset has 88858 rows and 10 columnswhich is divided into 7525 using train test split The descrip-tion of the features are given in Figure 7
Fig 7 Dataset Description
It is important to clean and visualize data before imple-menting machine learning algorithms This dataset initiallycontains null values replaced by -999 using fillna method Italso contains columns like Estimated insects counts as shownin Figure 8 with some time series related pattern that suggeststo generate features containing different lag values using shiftand rolling method A window size of 5 is used to extract themean of five observations with a lag of 1 and 2
Fig 8 Estimated Insects Count
We used five machine learning algorithms namely RandomForest LGBM KNN Decision Tree and XGBoost [29][30] [31] [9] for classification Decision Trees is a simplesupervised learning algorithm built on the concepts of treesThe tree has two components namely leaves and nodes Theleaves are the choices or the ultimate results whereas the nodesrepresent the points where the data is divided according to afixed parameter for further classification Random Forest is anensemble-based supervised machine learning algorithm Thebagging method is used to train the ensemble of decision treeswhich help to build the forest In simple terms Random Forestassembles numerous decision trees and combines them to get amore exact and stable expectation K-nearest neighbors (KNN)
depends on the possibility of the likeness of similarities Itpredicts the values of new points dependent on how closelythe new information is identified with the upsides of thegiven training set The KNN doesnrsquot make any assumptionsidentified with the preparation information and it for the mostpart inputs all the data values for training
Extreme gradient boosting (XGBoost) and light gradientboosting (LGBM) are based on the concept of gradient boost-ing This technique leverage weak models utilizing ensemblemethods that can develop new models to decrease the lossfunction estimated by gradient descent However there isa subtle difference these two models in terms of split ForXGBoost histogram based filters are used while for LGBMgradient-based one-side sampling (GSOS) method is used todistinguish the best split It has been observed that histogrambased filters technique is computationally costly comparedwith GSOS which recommends that LGBM is more effectivethan XGBoost
E Metrics
In this research we primarily perform two prediction tasksto classify crop damages and disease detection Since both fallunder the category of classification the models are evaluatedby metrics namely precision recall f1 score and accuracy
Precision =Tp
Tp + Fp(6)
Recall =Tp
Tp + Fn(7)
Accuracy =Tp + Tn
Tp + Tn + Fp + Fn(8)
F1score = 2P recisionRecall
Precision+Recall(9)
where Tp Tn Fp Fn represents True positive True negativeFalse positive False negative
IV RESULTS
Table II shows the evaluation of metrics for crop diseasedetection We performed the disease classification on six cropstomato potato apple peach grapes and xorn using threePre-trained CNN models VGG16 ResNet50 Densenet121All models performed well and there is a slight differencebetween the accuracy of the models For tomato corn andpeach Densenet121 outperforms other models in terms of allevaluation metrics VGG16 also performed state of the artresults for potato apple and grapes However it marginallyunderperformed in classification for peach and tomato Theresults forecasted from ResNet50 manifests that aside fromclass apple it failed to outperform other models For betterinterpretation and visualization bar plot is pictured in figure9 showing precision of models on given classes
Analysis of evaluation metrics are also discussed for cropdamage prediction As we can see from Table III the high-est accuracy achieved is 94 by LGBM and it completelybeats other algorithms Random Forest and XGBoost have
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
used is sigmoid For compilation of model adam opti-mizer is used with a learning rate of 00002 The batchsize is 32 and the model is trained for 30 epochs
D Crop Damage Prediction System
In crop damage prediction the task is to use and employfeature engineering concepts with machine learning algorithmsto predict the category of crop damage We used rsquomachinelearning in agriculturersquo dataset from the Analytics Vidhyawebsite [33] This dataset has 88858 rows and 10 columnswhich is divided into 7525 using train test split The descrip-tion of the features are given in Figure 7
Fig 7 Dataset Description
It is important to clean and visualize data before imple-menting machine learning algorithms This dataset initiallycontains null values replaced by -999 using fillna method Italso contains columns like Estimated insects counts as shownin Figure 8 with some time series related pattern that suggeststo generate features containing different lag values using shiftand rolling method A window size of 5 is used to extract themean of five observations with a lag of 1 and 2
Fig 8 Estimated Insects Count
We used five machine learning algorithms namely RandomForest LGBM KNN Decision Tree and XGBoost [29][30] [31] [9] for classification Decision Trees is a simplesupervised learning algorithm built on the concepts of treesThe tree has two components namely leaves and nodes Theleaves are the choices or the ultimate results whereas the nodesrepresent the points where the data is divided according to afixed parameter for further classification Random Forest is anensemble-based supervised machine learning algorithm Thebagging method is used to train the ensemble of decision treeswhich help to build the forest In simple terms Random Forestassembles numerous decision trees and combines them to get amore exact and stable expectation K-nearest neighbors (KNN)
depends on the possibility of the likeness of similarities Itpredicts the values of new points dependent on how closelythe new information is identified with the upsides of thegiven training set The KNN doesnrsquot make any assumptionsidentified with the preparation information and it for the mostpart inputs all the data values for training
Extreme gradient boosting (XGBoost) and light gradientboosting (LGBM) are based on the concept of gradient boost-ing This technique leverage weak models utilizing ensemblemethods that can develop new models to decrease the lossfunction estimated by gradient descent However there isa subtle difference these two models in terms of split ForXGBoost histogram based filters are used while for LGBMgradient-based one-side sampling (GSOS) method is used todistinguish the best split It has been observed that histogrambased filters technique is computationally costly comparedwith GSOS which recommends that LGBM is more effectivethan XGBoost
E Metrics
In this research we primarily perform two prediction tasksto classify crop damages and disease detection Since both fallunder the category of classification the models are evaluatedby metrics namely precision recall f1 score and accuracy
Precision =Tp
Tp + Fp(6)
Recall =Tp
Tp + Fn(7)
Accuracy =Tp + Tn
Tp + Tn + Fp + Fn(8)
F1score = 2P recisionRecall
Precision+Recall(9)
where Tp Tn Fp Fn represents True positive True negativeFalse positive False negative
IV RESULTS
Table II shows the evaluation of metrics for crop diseasedetection We performed the disease classification on six cropstomato potato apple peach grapes and xorn using threePre-trained CNN models VGG16 ResNet50 Densenet121All models performed well and there is a slight differencebetween the accuracy of the models For tomato corn andpeach Densenet121 outperforms other models in terms of allevaluation metrics VGG16 also performed state of the artresults for potato apple and grapes However it marginallyunderperformed in classification for peach and tomato Theresults forecasted from ResNet50 manifests that aside fromclass apple it failed to outperform other models For betterinterpretation and visualization bar plot is pictured in figure9 showing precision of models on given classes
Analysis of evaluation metrics are also discussed for cropdamage prediction As we can see from Table III the high-est accuracy achieved is 94 by LGBM and it completelybeats other algorithms Random Forest and XGBoost have
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
TABLE IIEVALUATION OF METRICS FOR CROP DISEASE DETECTION
Crop Type Model Prec Rec F1 AccVGG16 093 094 094 0945
Tomato ResNet50 096 096 096 0965Densenet121 096 097 097 0973
VGG16 094 096 095 0981Potato ResNet50 089 097 092 0960
Densenet121 087 096 090 0955VGG16 099 098 098 0981
Apple ResNet50 098 096 097 0970Densenet121 097 095 096 0970
VGG16 097 098 097 0986Peach ResNet50 100 099 099 0996
Densenet121 100 099 099 0996VGG16 097 097 097 0969
Grape ResNet50 096 096 096 0948Densenet121 097 096 096 0963
VGG16 093 093 093 0946Corn ResNet50 093 093 093 0943
Densenet121 095 095 095 0965
Fig 9 Bar Plot showing precision for Crop Disease Detection
very subtle difference in their predictions KNN is the worstperforming algorithm forestalling less than 5 percent precisionfor minority classes The differences in the results of thevarious classes suggest that the algorithms failed to managethe issue of class imbalance In general tree based classifiersoutperformed the standard machine learning algorithm Abar plot is pictured in Figure 10 shows the precision ofmachine learning algorithms used for crop damage predictionon various sub classes
V CONCLUSION
A multimodal precision farming system is implemented inthis project which consists of intelligent fertilizer irrigationcrop disease and damage prediction which help reduce theefforts and labors in the agriculture sector In this workwe intend to provide the methodology with circuit diagramflowchart and theoretical aspects for IoT systems Our workis well-organized easy to read contains well-defined figuresand diagrams This would definitely help new researchersto get a better understanding of the problem statement andfurther push the state of the art systems Subsequently weleverage multimodal approach to train an image classifica-
TABLE IIIEVALUATION OF METRICS FOR CROP DAMAGE PREDICTION
Model Class Prec Rec F1 Acc0 095 099 097
RF 1 078 072 075 0932 057 010 0160 097 099 098
LGBM 1 082 078 080 0942 044 020 0280 096 091 093
DT 1 050 059 054 0852 019 032 0240 095 098 096
XGB 1 072 070 072 0922 060 006 0110 085 097 090
KNN 1 023 006 009 0842 005 001 001
Fig 10 Bar Plot showing Precision for Crop Damage Prediction
tion and machine learning models related to crop diseasedetection and damages In multiclass image classificationwe employed three Pre-Trained CNN architectures namelyResNet50 VGG16 DenseNet121 In crop damage predictionLightGBM beats XGBoost and Random Forest by a slightmargin However the aformentioned models arenrsquot able toproduce quality results for minority classes Oversamplingtechniques like SMOTE and ADASYN are required to curbthe dominance of majority class Hyperparameter optimizationtechniques such as evolutionary algorithms are missed in thisresearch due to time constraints and less computational powerIt will be a part of the future work
REFERENCES
[1] Accessed Mar15 2021 httpsscrollinlatest973744farmer-suicides-centre-again-says-no-data-but-blames-states-uts-for-not-providing-it-to-ncrbsimtext=The20latest20NCRB20data20showedoverall20figure20in20the20country
[2] Accessed Mar15 2021 httpswwwdowntoearthorginnewsindias-deepening-farm-crisis-76-farmers-want-to-give-up-farming-shows-study-43728
[3] Shafi U Mumtaz R Garcıa-Nieto J Hassan S A Zaidi S A Ramp Iqbal N (2019) Precision agriculture techniques and practices Fromconsiderations to applications Sensors 19(17) 3796
[4] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
[5] Krizhevsky A Sutskever I amp Hinton G E (2012) Imagenet clas-sification with deep convolutional neural networks Advances in neuralinformation processing systems 25 1097-1105
[6] Tan C Sun F Kong T Zhang W Yang C amp Liu C (2018October) A survey on deep transfer learning In International conferenceon artificial neural networks (pp 270-279) Springer Cham SpringerCham
[7] Dinkins C Jones C Interpretation of Soil Test Reports for AgricultureMontana State University Extensions 2013
[8] Accessed Mar 15 2021 [Online]httpswwwagrocarescomenproductslab-in-the-box
[9] Song Y Y amp Ying L U (2015) Decision tree methods applicationsfor classification and prediction Shanghai archives of psychiatry 27(2)130
[10] Martınez F J Gonzalez- Z A Sanchez N Gumuzzio A Herrero-J CM rdquoSatellite soil moisture for agricultural drought monitoringAssessment of the SMOS derived Soil Water Deficit Indexrdquo RemoteSensing of Environment Vol 177 May 2016
[11] Vagen et al T-G Vagen LA Winowiecki JE Tondoh LT DestaT Gumbricht rdquoMapping of soil properties and land degradation risk inAfrica using MODIS reflectancerdquo Geoderma vol 263 2016 pp 216-225
[12] P V Santhi N Kapileswar V K R Chenchela and C H V S PrasadrdquoSensor and vision based autonomous AGRIBOT for sowing seedsrdquoInternational Conf on Energy Communication Data Analytics and SoftComputing Chennai 2017
[13] H Karimi H Navid B Besharati H Behfar I Eskandari rdquoA practicalapproach to comparative design of non-contact sensing techniques forseed flow rate detectionrdquo Computers and Electronics in Agriculture Vol142 Part A 2017 Pages 165-172
[14] Sasmal J (2014) Foodgrains Production in IndiandashHow Serious isthe Shortage of Water Supply for Future Growth Indian Journal ofAgricultural Economics 69(902-2016-66846) 229-242
[15] USDA httpwwwusdagov(accessedmarch152021)[16] Hongli Liu Xi Wang Jin Bing-kun rdquoStudy on Ndvi Optimization of
Corn Variable Fertilizer Applicatorrdquo Agricultural Engineering Sep-Dec2018 Vol 56 Issue 3 p193-202 10
[17] Shi J Yuan X Cai Y et al GPS Solut 2017 21 405 httpsdoiorg101007s10291-016-0532-2
[18] A F Colaco J P Molin rdquoVariable rate fertilization in citrus a longterm studyrdquo Precision Agriculture April 2017 Vol 18 Issue 2
[19] Accessed June 23 2019 [Online] Semios Integrated pest managementhttpsemioscomipm
[20] Kulkarni O N Patil V Parikh S B Arora S amp Atrey P K(2020 December) Can You All Look Here Towards Determining GazeUniformity In Group Images In 2020 IEEE International Symposiumon Multimedia (ISM) (pp 100-103) IEEE Computer Society
[21] Kumar A Kumar N amp Vats V (2018) Efficient crop yield predictionusing machine learning algorithms International Research Journal ofEngineering and Technology 5(06) 3151-3159
[22] Crane-Droesch A (2018) Machine learning methods for crop yieldprediction and climate change impact assessment in agriculture Envi-ronmental Research Letters 13(11) 114003
[23] Shah V amp Shah P (2018) Groundnut Crop Yield Prediction UsingMachine Learning Techniques Int J Scient Res Comput Sci EngInform Technol 3(5) 1093-1097
[24] Lottes P Khanna R Pfeifer J Siegwart R amp Stachniss C (2017May) UAV-based crop and weed classification for smart farmingIn 2017 IEEE International Conference on Robotics and Automation(ICRA) (pp 3024-3031) IEEE
[25] Mohanty S P Hughes D P amp Salathe M (2016) Using deep learningfor image-based plant disease detection Frontiers in plant science 71419
[26] He K Zhang X Ren S amp Sun J (2016) Deep residual learning forimage recognition In Proceedings of the IEEE conference on computervision and pattern recognition (pp 770-778)
[27] Simonyan K amp Zisserman A (2014) Very deep convolu-tional networks for large-scale image recognition arXiv preprintarXiv14091556
[28] Huang G Liu Z Van Der Maaten L amp Weinberger K Q (2017)Densely connected convolutional networks In Proceedings of the IEEEconference on computer vision and pattern recognition (pp 4700-4708)
[29] Pal M (2005) Random forest classifier for remote sensing classifica-tion International journal of remote sensing 26(1) 217-222
[30] Guo G Wang H Bell D Bi Y amp Greer K (2003 November)KNN model-based approach in classification In OTM Confederated In-ternational Conferencesrdquo On the Move to Meaningful Internet Systemsrdquo(pp 986-996) Springer Berlin Heidelberg
[31] Rahman S Irfan M Raza M Moyeezullah Ghori K Yaqoob S ampAwais M (2020) Performance analysis of boosting classifiers in recog-nizing activities of daily living International journal of environmentalresearch and public health 17(3) 1082
[32] Parikh S B Khedia S R amp Atrey P K (2019 September) Aframework to detect fake tweet images on social media In 2019 IEEEFifth International Conference on Multimedia Big Data (BigMM) (pp104-110) IEEE
[33] Accessed Mar 15 2021 httpsdatahackanalyticsvidhyacomcontestjanatahack-machine-learning-in-agriculture
