Automated Visual Yield Estimation in Vineyards · 2014-10-17 · Journal of Field Robotics 31(5), ... Nuske et al.: Automated Visual Yield Estimation ... cant differences are in terms

Automated Visual Yield Estimation in Vineyards• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Stephen Nuske, Kyle Wilshusen, Supreeth Achar, Luke Yoder, Srinivasa Narasimhan, and Sanjiv SinghRobotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, Pennsylvania 15213e-mail: [email protected]

Received 5 September 2013; accepted 30 June 2014

We present a vision system that automatically predicts yield in vineyards accurately and with high resolution.Yield estimation traditionally requires tedious hand measurement, which is destructive, sparse in sampling,and inaccurate. Our method is efficient, high-resolution, and it is the first such system evaluated in realisticexperimentation over several years and hundreds of vines spread over several acres of different vineyards.Other existing research is limited to small test sets of 10 vines or less, or just isolated grape clusters, with tightlycontrolled image acquisition and with artificially induced yield distributions. The system incorporates camerasand illumination mounted on a vehicle driving through the vineyard. We process images by exploiting the threeprominent visual cues of texture, color, and shape into a strong classifier that detects berries even when they areof similar color to the vine leaves. We introduce methods to maximize the spatial and the overall accuracy of theyield estimates by optimizing the relationship between image measurements and yield. Our experimentation isconducted over four growing seasons in several wine and table-grape vineyards. These are the first such resultsfrom experimentation that is sufficiently sized for fair evaluation against true yield variation and real-worldimaging conditions from a moving vehicle. Analysis of the results demonstrates yield estimates that captureup to 75% of spatial yield variance and with an average error between 3% and 11% of total yield. C© 2014 WileyPeriodicals, Inc.

1. INTRODUCTION

Yield predictions in vineyards are important for managingvines to optimize growth and eventual fruit quality. Forinstance, if an overly large crop is forecast, fruit may beremoved during the season to achieve certain fruit qualitygoals. This practice of crop thinning is much more effectivewhen based on an accurate yield estimate. Yield forecastsalso prepare a grower for the harvest operation, for shippingtheir crop, storing their crop, and also selling their crop onthe market. Typical yield predictions are performed usingknowledge of historical yields and weather patterns alongwith measurements manually taken in the field. The cur-rent industry practice for predicting harvest yield is labor-intensive, expensive, inaccurate, spatially sparse, destruc-tive, and riddled with subjective inputs. Typically, the pro-cess for yield prediction is for workers to sample a certainpercentage of the vineyard and extrapolate these measure-ments to the entire vineyard. Agronomic studies have es-tablished that large spatial variability in vineyards acrossmultiple countries and growing conditions exists (Taylor,Tisseyre, Bramley, Reid, & Stafford, 2005). However, thesample size is often too small in comparison to the spa-tial variability across a vineyard, and as a result the yieldpredictions are inaccurate and spatially coarse.

[Author list amended on 23 September 2014 after error in first onlinepublication on 11 August 2014: The following authors have beenadded: Srinivasa Narasimhan, Robotics Institute, Carnegie MellonUniversity.]

There is a gap between the methods available to predictthe yield in a vineyard and the needs of vineyard managersto make informed decisions for their vineyard operations.Using carefully designed illumination and a camera sys-tem (see Figure 1) paired with novel algorithms that auto-matically detect the fruit within the imagery, our methodcan make dense predictions of harvest yield efficiently andnondestructively. We overcome difficulties in imaging anddesign our camera and illumination setup to optimize forlow motion blur, increased depth-of-focus, and low illumi-nation power for fast-recycle times permitting high-framerates. This design maintains high image quality at high ve-hicle velocities and enables deployment at large scales, fea-tures overlooked in many existing visual yield estimationstudies. Our results demonstrate that we can automaticallydetect and count grapes to forecast yield efficiently withhigh precision and accuracy.

We also overcome the challenges distinguishing thegrape berries from similarly colored leaves, in nonuniformlighting, and variable scales due to variable berry size orvariable distance of the fruit from the camera. Our approachcan be distinguished from a number of existing studieson the detection of grapes, which are based upon one ofthree different types of visual cues of grape berries appear-ance: either color (Diago et al., 2012; Dunn & Martin, 2004),shape (Rabatel & Guizard, 2007), or texture (Grossetete et al.,2012). Each of the three cues has advantages in certain con-ditions and, all have some limitations in others. Color onits own is not suitable for distinguishing green grapes on a

Journal of Field Robotics 31(5), 837–860 (2014) C© 2014 Wiley Periodicals, Inc.View this article online at wileyonlinelibrary.com • DOI: 10.1002/rob.21541

838 • Journal of Field Robotics—2014

Figure 1. Example camera image and image of a vehicle in a vineyard. Automatically detecting the grape crop within such imageryrequires an algorithm that can distinguish the shape and texture/shading of the green fruit against the cluttered and similarlycolored leaf background.

Journal of Field Robotics DOI 10.1002/rob

Nuske et al.: Automated Visual Yield Estimation • 839

background of green leaves. The grape shape can be diffi-cult to identify in cluttered images with leaves and otherspurious contours. Grape texture can be less distinguishingunder certain illumination conditions.

We take a different approach. Rather than focusing ona single cue, we develop a robust and versatile algorithmto detect grape berries under a variety of conditions byexploiting all three main visual cues that grape berries have.Our approach is to detect candidate hypotheses of wheregrapes may be located in images using a shape transform,and then classify these candidate locations using texture andcolor descriptions within a robust classification frameworkthat exploits all the visual cues.

We develop a model that relates image measurementsto yield and optimize the model for two key goals:

1. accurate estimation of the spatial distribution of yield,2. accurate estimation of the overall yield.

Preliminary results of our approach were reportedin Nuske, Achar, Bates, Narasimhan, & Singh (2011) andNuske, Gupta, Narasihman, & Singh (2012). We extend ourprior work in the following ways:

� Present a study of three different visual texture descrip-tions and evaluate each on a variety of datasets (Sections3.2 and 5.3.2).

� Introduce a novel algorithm for berry keypoint detectionthat is invariant to berry size (Section 3.1.2).

� Expand our model relating image measurements to yieldpredictions and demonstrate how to optimize for accu-racy (Section 4).

� Report on field experiments now spanning four growingseasons of multiple varieties and growing systems, total-ing tens of acres of vines, all validated with true yieldmeasurements (Section 5).

The experimental results are the first visual vineyardyield estimation results that are collected over hundredsof vines and also encompassing multiple growing seasons.This is a substantially larger scale than other existing work—by comparison, the next largest study in terms of datasetsize was a study using a static camera of 10 vines (Diagoet al., 2012). Their study artificially increased the size of thedataset by defruiting vines sequentially, which increasesthe distribution of yields but does not capture the true vari-ance in leaf and vine occlusions and cluster-to-cluster oc-clusions. Our system stands apart from existing approachesin the complete design that solves all of the true imagingand visual detection challenges. Our method is also the firstto present a way to eliminate the double-counting prob-lem from overlapping imagery and also the challenge ofgeometrically referencing the measurements by estimatingcamera position along the row. Unlike other work, our ex-periments are conducted from a moving vehicle in a com-pletely un-isolated fashion such that we replicate all issues

that would be faced by a real deployment. No other studyso far has evaluated all the challenges and variables thatmust be considered in a rigorous study, including variancein visual detection performance, differences in berry-size,detection of green juvenile berries, the variable distance ofthe camera to the fruit zone, estimating location along thevineyard rows, and the problems of leaf/vine occlusion andcluster-on-cluster occlusion.

2. RELATED WORK

Viticultural studies (Clingeleffer, Dunn, Krstic, & Martin,2001) have revealed that current industry practices to fore-cast yield are inaccurate because of sampling approachesthat tend to adjust toward historical yields and include sub-jective inputs. The calculation of final cluster weight fromweights at veraison (i.e., onset of color development) usesfixed multipliers from historic measurements (Wolpert &Vilas, 1992). The multipliers are biased toward healthiervines, thus discriminating against missing or weak vines,and multipliers for cluster weights vary widely by vine-yard, season, and variety. In general, the approaches haverequired manual samples of mean berry size, mean clustercount, or mean cluster size (Serrano, Roussel, Gontier, Du-fourcq, 2005), or a combination of the three, and then anextrapolation to yield across a vineyard block.

Sensor-based technologies using trellis tension mon-itors, multispectral sensors, terahertz-wave imaging, andvisible-light image processing have all been proposed foryield estimation in vineyards. A dynamic yield estimationsystem based on trellis tension monitors has been demon-strated (Blom & Tarara, 2009), but it requires a permanentinfrastructure to be installed. Further, variation in yield isnot the only cause of variations in trellis tension. Ambienttemperature and vine size also affect trellis tension, resultingin a loss of accuracy when relating tension to yield. Infor-mation obtained from multispectral images has been usedto forecast yields with good results, but it is limited to vine-yards with uniformity requirements (Martinez-Casasnovas& Bordes, 2005). Related multispectral imagine work hasbeen applied to detecting almonds with a combination fromRGB and ir images (Hung, Nieto, Taylor, Underwood, &Sukkarieh, 2013a). A proof of concept study by Federici,Wample, Rodriguez, & Mukherjee (2009) has shown that ter-ahertz imaging can detect the curved surfaces of grapes, andit also has the potential to detect through an occluding thincanopy. The challenge for this approach is to achieve fastscan rates to be able to deploy the scanner on a mobile plat-form while also having sufficient wavelength/resolution toboth detect as well as penetrate.

Conventional RGB camera imagery has been proposedin a number of works as a means for fruit detection andyield estimation. Jimenez et al. (Hung, Underwood, Nieto,& Sukkarieh, 2000) provide a summary of fruit detectionwork. Singh et al. (2010), Zhou, Damerow, Sun, and Blanke(2012), Hung, Underwood, Nieto, & Sukkarieh (2013b), and



Tabb, Peterson, and Park (2006) present methods for de-tecting and classifying fruit pixels in imagery collectedin apple orchards. Yang, Dickinson, Wu, and Lang (2007)demonstrated a similar approach to recognize tomatoesfor harvesting. A method presented by Berenstein, Shahar,Shapiro, and Edan (2010) used a combination of color andedge features for a coarse estimate of where grape clustersand leaves reside in the images for the purpose of precisionspraying. A new method that uses 3D image reconstruc-tions for fruit, leaf, and stem detection, demonstrated byDey, Mummert, and Sukthankar (2012), could classify fruit,leaves, and shoots based on 3D reconstructions generatedfrom image sequences, which, unlike our work, is sensitiveto slight wind while imaging.

The most closely related works to ours in vineyard yieldestimation with image analysis can be divided by whichtype of visual feature is used to identify the grapes withinimages: color, shape, or texture.

Color discrimination has been demonstration by Dunn& Martin (2004). This approach was attempted on Shirazpost-veraison (red-grapes with full color development) inshort row segments. A similar color-based method thatdetects fruit close to harvest after the onset of color-development is presented by Diago et al. (2012) and Marden,Liu, & Whitty (2013). These simple color-based methodsare not applicable for the majority of real-world exampleswhere the fruit appears over a background of similarly col-ored leaves, as is the case in green grape varieties and inall varieties before veraison. The contour and shape of theberry is another method used to detect the fruit (Rabatel& Guizard, 2007). The contour is more broadly applicableto detect green grapes on green leaves, although partiallyvisible grapes and cluttered images filled with a variety ofcontours on leaves and stems make it difficult to completelydistinguish the grapes using contour and shape alone. Thevisual texture on the surface of the grape, especially whenimaged with an illumination source, is also a viable cuefor detection as exploited by Grossetete et al. (2012), whodemonstrate a hand-held device that can be used to mea-sure the size of isolated clusters. Similar approaches detectthe shading on apples (Wang, Nuske, Bergerman, & Singh,2012) and oranges (Swanson et al., 2010). All three types ofcues—color, shape, or texture—have seen some success indetecting grapes, although none is convincing as a sole cuefor detecting the fruit. In our work, we demonstrate how toexploit all three of these cues to create a strong classificationalgorithm that separates berries from leaves and surpassesthe ability that can be achieved with one of these visual cuesalone.

Finally, in performing a comparison of our work toall prior visual yield estimation studies, the most signifi-cant differences are in terms of the scale of deployment andalso rigorous evaluation of all real-world considerations.The only other works performed in which experiments in-cluded measurements evaluated against total vine yield are

Diago et al. (2012) and Dunn & Martin (2004). These exam-ples used limited sets of vines in their experimentation—10vines and 1 vine, respectively. Both of these examples useddefruiting to artificially increase the size of the datasets,which caused their experimental dataset to not be represen-tative of the true variance in vine occlusions and cluster-to-cluster occlusions. Also, they did not consider individualberries, they only considered raw pixel count, which is sen-sitive to the berry size, the distance from the vine, the focallength, image resolution, as well as berry count (the desiredmeasurement). Also, they used a white sheet or white wallbackdrop to block out background disruptions. Further, todraw more differences, our work is unique in that it is asystem that can be deployed from a moving vehicle thoughthe use of automatic image registration to deal with doublecounting or undercounting of fruit. We also consider de-tection of green juvenile fruit on a green leaf background.There are other works that do look at detecting green berriesamong green leaves, such as Grossetete et al. (2012) and Ra-batel & Guizard (2007), but their experiments are restrictedto a small set of isolated clusters and do not evaluate to-tal vine yield. Our work is the first system and study ofall these real-world considerations, and evaluation is per-formed over hundreds of vines and four growing seasons.

3. BERRY DETECTION

We deploy a sideways-facing camera on a small vineyardutility vehicle; see an illustration in Figure 6. The imagescapture the vines and are processed with our algorithm todetect and estimate the crop yield.

Yield in a vineyard is a combination of the followingcrop components: the number of clusters, the number ofberries per cluster, and the berry weight. Given the vari-ance in yield of a set of vines, this variance can be brokendown into the three yield components as follows (Clingel-effer et al., 2001):

1. Variance in number of clusters per vine—contributes ap-proximately 60% of the total yield variance.

2. Variance in number of berries per cluster—contributesapproximately 30% of the total yield variance.

3. Variance in berry size—contributes approximately 10%of the total yield variance.

These three yield components combine to describe all100% of variance in harvest yield. Current practice is to takesamples of each of these components to compute an averageand compute the final yield. We take an approach to estimatethe first two of these items together in one measurement—that of the number of berries per vine. The reason for this isthat it is difficult, especially late in the season, to delineatethe boundaries of clusters within images. However, it ispossible to count the total number of berries seen, hencecombining the two components—number of clusters per



Figure 2. Our yield estimation process is divided into three facets: detector-calibration to the left, berry-detection shown in thecenter, and visibility-calibration to the right. The first facet is detector calibration, which uses a set of training images (we use a setof 10 images), where grape locations are manually marked. The training images are processed to find grape keypoints (Section 3.1),features are extracted at these keypoints (Section 3.2), and features are labeled as grape or nongrape features based on theirproximity to manually marked locations. The labeled feature-descriptors are then stored in a randomized KD-forest. Detectionperformance is then characterized using the training set; this allows for the identification of the optimal voting ratio (Section 4.2)and also the computation of a detection calibration function to account for trends in detection performance (Section 4.2.1). Aftertraining, the core part of our approach is berry detection. Here, grape keypoints are found in each image, and then features areextracted at the keypoints and classified as grape and nongrape using the randomized KD-forest with the optimal voting ratio[Eq. (1)]. After detections are computed, the output is corrected using the function in Eq. (10) to estimate visible fruit. The finalpart of our process is visibility calibration, which uses yield data (either from prior harvest or sparse in-season samples) to form acalibration function [Eq. (17)] to estimate total yield from our dense image measurements.

vine and berries per cluster—into one measurement: berriesper vine.

Our approach does not attempt to measure berryweight. However, we account for 90% of the harvest yieldvariation with berries per vine (Clingeleffer et al., 2001).Furthermore, instead of taking a small sample and extrap-olating, we aim to estimate nondestructively the specificyield at high resolution across the entire vineyard. Hence,we will not introduce sampling errors into the process.

The overall process is presented in Figure 2, which issplit into three phases: berry-detection, detector-calibration,and visibility-calibration.

Our algorithm, as first proposed in Nuske et al. (2011)and expanded here, is split into the following stages:

(1) Detect potential berry locations (Section 3.1).(2) Classify and group the potential locations that have sim-

ilar appearance to grape berries (Section 3.2).



3.1. Detecting Image Keypoints—Potential BerryLocations

The first step of our algorithm is to find image keypoints thatare potential grape berry locations; see Figure 3(b). Thesekeypoints will later be classified as either berry or not-berry.

There are two reasons why we detect keypoints. Thefirst is to identify distinct grape-berries, which is impor-tant to extract measurements that are invariant to the stageof the berry development. Other work, such as Dunn &Martin (2004) and Diago et al. (2012), detect grape pixelsonly. However, the number of pixels that belong to fruitwill change as the grapes get larger, and therefore the mea-surements will not be invariant to berry development. Thesecond reason is to reduce computation by only consideringa set of keypoints; this reduces the amount of the image thatis subsequently processed.

We present two keypoint detection algorithms. The firstis a radial symmetry algorithm [introduced by Loy & Zelin-sky (2003)] that uses the circular shape of the berry as a cuefor detection. The second is a novel maximal point detectionalgorithm that searches for the maximal point of shading inthe center of grapes that have been illuminated by a flash.The two algorithms are introduced in the following twosubsections and evaluated later in Section 5.3.1.

3.1.1. Radial Symmetry Transform

One approach we use is to search for points that have peaksin a radial symmetry transform from Loy & Zelinsky (2003).The radial symmetry transform requires us to know theradii of the berries as seen in the image ahead of time. Theberry radii (in pixels) are dependent on the focal lengthof the camera, the actual berry size, and the distance fromthe camera. The focal length is kept fixed in our tests andthe vehicle maintains a relatively constant distance fromthe vines. There is still variation in the radius in which theberries appear in the image from differing berry sizes andalso some variation in location within the vine. We accountfor this variation by searching for radially symmetric pointsover a range of possible radii, finding points that exhibit themost radial symmetry.

3.1.2. Invariant Maximal Detector

We have developed an alternative keypoint detection algo-rithm that searches for peaks in intensity corresponding togrape centers. These peaks are found by finding local max-ima in three-by-three kernels, and the peaks are validatedby an iterative growing procedure that identifies peaks withsymmetrical shading created by a flash. We continue in thissection to describe this procedure in more detail.

Lighting upon grapes is controlled, as flashes are po-sitioned parallel to the camera’s optical axis to illuminatethe grapes (discussed in Section 5.1). This leads to grapesthat have a strong specular reflectance at the center of the

Figure 3. Example images showing the functioning of our vi-sual berry detection algorithm. Input image is seen in (a). Key-points, shown in (b), are the potential berry locations. In (c),points marked in blue have been classified as having an ap-pearance similar to a berry, and in (d) berries that neighborother berries are clustered together.



Figure 4. Examples demonstrating the invariant maximal initial keypoint detector. Part (a) demonstrates the seeded growth intoregions of decreasing intensity over many iterations. Parts (b), (c), and (d) show the input image, the initial maximal points, andthe final keypoints that pass the seeded growing algorithm, respectively.

berry. From this point of specular reflectance, pixel intensitydecreases steadily toward the edges of the grape. If we canfind this point of specular reflectance, we can find a set ofkeypoints that are potential berries.

There are other algorithms in the literature that detectthe shading on fruit produced by a flash, such as Grosseteteet al. (2012), Swanson et al. (2010), and our previous work(Wang et al., 2012). However, these approaches do not con-sider fruit of substantially different sizes, and they wouldrequire some reconfiguration to function on fruit of differentsize, or images with different resolution.

We present an approach that does not need a size pa-rameter as input and can function on berries of substantiallydifferent sizes, from 10 pixels in diameter up to well over100 pixels in diameter.

The identification of potential grape centers in imagesis as follows. Image noise is first eliminated through a Gaus-sian pyramid downsampling operation. Then, an initial setof local regional max within their immediate neighborhoodis identified; see Figure 4(c).

Filtering of this set is then achieved through an iter-ative region growing method, operating on each regionalmaxima. Regional maxima serve as the seed points to region

growing. Through an iterative process, shown in Figure 4,these seed points are grown and evaluated for symmetryand decreasing intensity profile.

At each iteration, pixels of decreasing intensity adjacentto the regional max are included in the region. This adds aring of pixels darker than the maxima but brighter than afraction of the maxima intensity (we use 85% as a thresholdfor each ring). Once the new “ring” of darker pixels hasfinished growing, a new iteration begins by growing a newring of pixels, which includes darker pixels than the previ-ous ring but brighter than a fraction of the previous ring’sintensity. Each image point can only belong to one grownregion. This ensures that after an initial region is grown, anadjacent region cannot contain the same pixels, eliminatingoverlapping maxima. This process repeats until a set num-ber of rings has been added (three intensity levels) or oneof the following shape criteria has failed:

� First, the region’s centroid should be within a range ofthe region’s maxima point.

� Second, the shape of the region should not become elon-gated.



A region that has three levels of decreasing intensityand passes these two shape criteria is considered a possiblegrape keypoint. Finally, a mask image of pixels that belongto regions is maintained, such that each image point canonly belong to one grown region. This ensures that after aninitial region is grown, an adjacent region cannot containthe same pixels, eliminating overlapping maxima.

3.2. Classifying Appearance of CandidateKeypoints as Berry/Not-berry

The next stage in our algorithm is to classify the detectedkeypoints into grapes or not-grapes; see Figure 3(c). We takean image patch around each detected center and compute acombination of both color and texture features from that im-age patch. For the color features, we form a six-dimensionalvector from the three RGB channels and the three L*a*bcolor channels.

Our initial work in Nuske et al. (2011) presented the useof Gabor filters to classify the grape’s texture. In this paper,we evaluate three different texture features from three broadclasses of features, and we will study the applicability ofeach type of feature to a set of different imaging conditions.In particular, we choose three of the broad categories:

1. Filter banks: used as far back as the 1980s (Laws, 1980)in image texture classification tasks. We use Gabor filterswith four scales and six orientations and combine to forma 24-dimensional feature vector, as per the original work(Nuske et al., 2011).

2. Local texture features: proposed in the late 1990s, theydescribe histograms of gradients (HoG), or similar vari-ations, in a support region around an image location.Here, we use a SIFT descriptor (Lowe, 2004) computedin a support region centered at our candidate centers.Since we are using our specific berry keypoint detec-tion algorithms as presented in the previous section, wedo not use the SIFT keypoint detection step. The rawdescriptors total 128 dimensions, which we decimateinto its most discriminative axes using PCA into a 32-dimensional vector.

3. Binary relations: a recent class of image features thatcompile sets of pairwise binary intensity relations be-tween pixels or regions surrounding an image location.We use the Fast Retinal Descriptor (FREAK) (Alahi, Or-tiz, & Vandergheynst, 2012), which is designed to mimicthe human retinal construction and is noted to achievehigher performance rates than other recent binary fea-tures [BRIEF (Calonder, Lepetit, Strecha, & Fua, 2010)].Similar to the process used for SIFT descriptors, we per-form dimensionality reduction from the 200 raw dimen-sions into its most discriminative axes using PCA into a32-dimensional vector.

We study these three types of features in the contextof three different illumination conditions with which wehave collected image data in vineyards: natural illumina-tion, flash illumination, and cross-polarized flash illumina-tion. Results of the study are presented in Section 5.

We concatenate the color and texture features to forma set of candidate image features, Ic, computed at the berrykeypoint candidates. To classify the candidate features, weuse an a priori constructed randomized KD-forest (Lepetit,Lagger, & Fua, 2005) from a set of training samples, T, ex-tracted from a subset of images. We manually define berrycenters in the training images that correspond to the posi-tive examples of the appearance of berries, Tp. For negativesamples, Tn, we compute features at our detected keypointsthat do not align with a manually defined berry center.

Then for each candidate feature Ic(i) we extract a setof nearest nodes Tc(i), from the randomized KD-forest. Wethen vote on the class of each interest point by comput-ing a ratio of the number of positively labeled features,Tr = |Tc(i)∈Tp|

|Tc(i)| , to the total size of the candidate feature set.We define a threshold, τr , applied to the voting ratio, whichis adjusted to control how conservative or liberal the clas-sification is. Our resulting set of classified berries is bd ∈ Ic,where

Tr > τr . (1)

We group these detected berries into clusters less thana threshold in distance from each other. We remove clustersof detections that are smaller than an area threshold, whichremoves spurious detections; see Figure 3(d).

3.3. Combining Berry Detections in ImageSequences

Since we are collecting data from a moving platform andwish to produce yield estimates that are assigned to specificlocations in the vineyard, it is not sufficient merely to beable to detect grapes in an image. First, we need to havestate information for the data collection vehicle to be ableto back-project the grape detections from each image to aposition along the vineyard row. Second, we need to usethe registered grape locations to avoid double counting ofgrapes between consecutive images.

To gain state estimation for the data collection vehicle,we use a second camera facing backward from the vehi-cle angled at approximately 45◦ down toward the groundfor positioning. The downward-facing camera is a stereopair and we use a visual odometry algorithm (Kitt, Geiger,& Lategahn, 2010) to track the position of the vehicle as itmoves along the row. With state information data for the ve-hicle, individual grapes can be registered to locations on thelocal fruit wall. The problem is that there can be significantoverlap between consecutive images. This overlap needsto be accounted for during grape registration to prevent asingle grape from being counted multiple times.



The process to identify individual grapes is as follows.First, the detections from each image from a vineyard roware back-projected onto the local fruit wall. The entire lengthof the row is then partitioned into short segments (around0.5 m in length). A segment that contains projected grapedetections from more than one image is a region in whichcamera views overlapped. The detections from the imagewith the most detections lying in the segment are retained,and detections in the segment from the other images arediscarded. This heuristic is used to choose the image wherethe effect of occlusions was minimum. Taking the maximumimage measurement rather than attempting to merge all im-age measurements together avoids the issue of attemptingto do fine registration while fruit is moving from wind or thevehicle pulling at the vine. Each segment now has a visibleberry count associated with it, and this can be aggregatedover rows, vines, or sets of vines.

4. RELATING IMAGE-BASED MEASUREMENTS TOYIELD PREDICTIONS

The previous section describes how to detect berries withinimages. This section describes how to take these image mea-surements and form an estimate of fruit yield; Figure 2 ex-plains the procedure.

Viticulturists have long studied the process of predict-ing the size of the harvest yield, and they have developedmodels of the various yield components (Clingeleffer et al.,2001). In the most basic form, the weight of the harvest (Wh)can be expressed as a product of the number of berries (Nb)and the mean weight of the berries (Wb),

Wh = NbWb. (2)

Our approach to predict the yield at harvest time is fo-cused on measuring the number of berries (Nb). The numberof berries is stable shortly after berry set (once the periodknown as shatter is passed) and it accounts for 90% of thevariation in yield (Clingeleffer et al., 2001); varying berryweight is responsible for the remaining 10%. Our berrycount forms a part of a yield forecasting function, f (·), whichoutputs an estimate (Nb) of the actual berry count:

Nb = f (Ndb ). (3)

4.1. Visible Berries and Estimating Self-occlusions

In our prior work (Nuske et al., 2011) we used the visibleberry count as a prediction of the cluster size, assuming thedetected berry count is proportional to the total berry count:

Nb ∝ Ndb . (4)

In the Results section (Section 5.4), we study the visibleberry count in controlled experiments, and also, in an at-tempt to improve the measurement of the occluded berriesin a cluster, we propose two potential modifications.

The first alternative measurement we propose is to takethe convex hull formed by all the visible berries in the clus-ter. Assuming the cluster has uniform density and an aver-age thickness of the grape cluster to be D, we multiply thearea A to this fixed cluster depth, and we normalize withthe average berry radius Rb,

Nb ∝ DA

R2b

. (5)

The second alternative is to predict the size of a clusterusing a 3D ellipsoid model. A grape cluster’s volume can beapproximated with an ellipsoid cutting off the image planeas an ellipse. We find the best-fit ellipse for the berry centerlocations with same normalized second central moments.Given the semiaxes of the ellipse in pixels R1 and R2, withR1 ≥ R2, the volume of the corresponding ellipsoid wouldbe proportional to the volume occupied by the berries (Bv) inthe cluster. Using the average berry radius (Rb) of the cluster,we can calculate the total number of berries occupied by thecluster:

Vc ∝ 43πr1r2

2,

Nb = Vc

/ (43πr3

b

). (6)

We study these three approaches to measuring grapecluster size in controlled laboratory tests in the Results sec-tion (Section 5.4).

4.2. Optimizing for Yield Estimate Accuracy

Ultimately, the goals are to achieve accurate yield estimates.In this section, we isolate the metrics of interest and presenta means to optimize our algorithm to improve spatial yieldestimation accuracy.

There are two requisites for Nb in terms of accuracy:

1. Accurate estimate of overall yield. Minimizing the errorterm: ey .

2. Accurate estimate of spatial yield. Minimizing the errorterm: es .

ey =∑

(N ib) − ∑

(Nib)∑

(Nib)

, (7)

es = �(N ib − Ni

b)2

�[Nib − μ(Ni

b)]2, (8)

where Nib is the number of berries on the ith vine.

We will revisit details regarding ey and es later, but webegin by introducing a function f that captures the relation-ship between the detected fruit in the camera image Nd

b tothe total fruit Nb.



4.2.1. Detection-Calibration

There are errors in the visual detection process that mustbe modeled. In Nuske et al. (2011), the performance of thedetection algorithm was analyzed to find that the algorithmdoes not detect some berries visible to the camera, and to alesser extent there are some occasions when the algorithmfalsely reports a berry where there was not one.

We form a measurement of detection using a sampleof images where the berry centers have been manuallymarked, enabling us to measure three metrics: true posi-tives (TP)—the number of berries detected that were actualberries; false positives (FP)—the number of false berry de-tections; and false negatives (FN)—the number of berriesvisible in the image that were not detected. Our measure,κd , exactly defines the rate of visible berries that are detectedby the system,

κd = TPTP + FN

. (9)

We take κd , which is the true positive rate, togetherwith the false positives (FP) expressed as a number perlinear length of vineyard, and it can be applied to the levelof detected fruit Nd

b to estimate the amount of visible fruit:

Nvb = fd (Nd

b , κd , FP) = (Ndb − FP)κd

. (10)

Importantly, unlike κd , the number of false positivesis decoupled from the total number of visible fruit. Thisdecoupled behavior was discovered empirically throughour analysis and readily understood and explained, sincevines with little or no fruit will induce a similar amount offalse positives as vines with larger amounts of fruit.

We use this function to correct for variable detectionperformance arising from the differing appearance of thevines and variable imaging and lighting configurations.

Referring back to our error, we wish to minimize, es ,in Eq. (8), which is in fact directly related to the correlationmeasure r2 = 1 − es ; the portion of the error term we caremost about is the squared error term of the predicted yieldcompared to the true yield: �(N i

b − Nib)2.

Our model of detected to visible to total fruit is a linearsystem, and the terms can all be calibrated in closed form re-gression. Assuming there is sufficiently accurate calibrationdata, the squared error term will only be nonzero under thepresence of some unmodeled variance in our model. It isexpected that there is some nonzero variance in our occlu-sion term κω, where we use the notation for the the standarddeviation of this term as σκω , which reforms the equation as

Nb = fv(Nvb , κω, σκω ) = Nv

b

κω + g(σκω ), (11)

where g() is a zero-mean Gaussian function. At present, inthe absence of a known method to estimate this variance,σ (κω), we leave this as an unmodeled error.

However, there is another source of variance, which isthe deviation within our detection function σ (fd (·)). Thisvariance in the detection function is possible to minimize,which in turn will optimize es . Using hand-labeled trainingimages to analyze our berry detection system, we find thatthere are two different behaviors in the detection function,one in which the mean of the true positive berries output byour system is linearly related to the visible fruit, and anotherin which we find false positives to be modeled as a variable(β) unrelated to the visible fruit:

TP = Nvb [α + g(σκd

)], (12)

FP = β[1 + g(σβ )]. (13)

Thus, we can rewrite our relationship to visible fruit asfollows:

Nvb = Nd

b

α + g(σα) + β[1 + g(σβ )]. (14)

g(σα) and g(σβ ) are the two sources of variance in the detec-tion function, and in a well-calibrated system they will bedirectly proportional to the squared error term:

�(N ib − Ni

b)2 ∝ μ(Nvb )g(σα)

μ(TP)+ μ(FP)g(σβ )

μ(Nvb )

. (15)

We have now identified the terms responsible for spa-tial accuracy, and we can develop an optimization strategy.The key variable in our berry classification algorithm is thethreshold on the randomized KD-forest voting ratio, τr fromEq. (1). We use this as the variable to optimize against com-puting the minimum of our error term:

arg minτr

(μ(Nv

b )g(σα)μ(TP)

+ μ(FP)g(σβ )μ(Nv

b )

). (16)

The error term and its components are graphed to-gether in Figure 5. The τr parameter is adjusted to balancethe variance in the false positives in the system and the vari-ance in true positives, and the optimal point is captured bythe error term. The minimal point on the error term curveis used to ultimately reflect the maximal R2 value againsta vineyard dataset. Later in the Results section, this opti-mization is studied for different texture feature descriptors(Section 5.5).

4.2.2. Visibility Calibration

Once a corrected estimate of visible fruit is derived, we forma second function, fv(), relating the visible estimate of fruitto the total fruit as follows:

Nb = fv(Nvb , κω) = 1

κω

Nvb . (17)



Figure 5. Spatial yield accuracy, R2, can be maximized by minimizing the detection system’s error metric. Specifically, therandomized KD-forest voting threshold τr is optimized against normalized σα and normalized σβ from the error term from Eq. (16).

This relationship requires yield data to form an estimateof visibility; κω. As discussed in detail in Nuske et al. (2012),the yield data can be collected from previous harvest dataor from in-season samples. Although fd () varies by datasetand imaging type, we find the visibility function fv() isconsistent over seasons for a particular vineyard.

Together the two functions fd () and fv() form the twokey relationships—from image measurement to visible fruitand from visible fruit to yield prediction; for an overviewof the procedure, see Figure 2.

5. RESULTS

Experiments are conducted at four different vineyards overfour growing seasons, including the following varietals:Traminette, Riesling, Flame Seedless, Chardonnay. In eachvineyard and season, the image-measurements are eval-uated against carefully collected harvest yield measure-ments. The yield estimation is studied for both spatial ac-curacy and overall accuracy, and we demonstrate how tooptimize the algorithm to maximize spatial yield accuracyand also to analyze the accuracy for different methods tocalibrate the relation between image measurements andyield.

We analyze the berry detection algorithm over thedifferent grape varieties and illumination conditions, as-sessing the performance of different texture feature de-scriptions by quantifying which descriptor performs bet-ter in which lighting condition and which generalizes thebest across datasets without the need for retraining andtuning.

5.1. Equipment Setup

There are significant challenges in designing an imagingsystem that adheres to a number of considerations, includ-ing the lighting power, lighting distribution, and specular reflec-tions from the vine leaves, the depth-of-focus appropriate tothe vineyard fruiting-zone size plus the variable position ofthe camera with respect to the fruiting zone, the frame-rate ofthe camera with respect to desired operating velocity, motionblur, which is related to the exposure duration, the vehiclevelocity, and vibrations from the engine and the terrain un-dulations, the recycle time of the flashes, heat accumulation inthe flashes, position-estimation and image-registration, image-resolution to enable detections of small grape berries early inthe season, and image-quality to enable detection of the fruit.

The camera is mounted about 0.9 and 1.5 m from thefruiting zone, depending on the size of the fruiting zone forthe particular vineyard. Illumination is placed directly tothe side of the camera to reduce shadowing.

The most current equipment design was deployed tocollect data starting in 2013; see Figure 6. To image the fruit,a 24MM F/2.8D AF NIKKOR lens is used in conjunctionwith a Prosilica GE4000 camera, which are mounted facingsideways on the vehicle viewing the fruit. Additionally, twoEinstein 640 mono flashlights are mounted on both sides ofthe camera. The stereo camera used for visual odometry es-timation is a PointGrey BumbleBee2 mounted to the vehiclepointed down the row used within the visual odometry al-gorithm to estimate the position of the vehicle along the row.Both cameras are triggered by external pulses to maintainsynchronization. The images are captured at 5 Hz and thevehicle is driven at 5.4 km/h, which as a rough comparison



Figure 6. Photos of the equipment used to collect the 2013 datasets. Part (b) shows the Prosilica GE 4000 camera with a red-casemounted in the middle of a vibration-damped mounting plate, attached to a custom reconfigurable aluminum frame with anadjustable ball joint at the head. Beside the camera are the two Einstein 640 flash units with reflectors and a custom diffusion filterattached. There is a Hokoyu UTM30LX laser scanner also seen mounted above the camera on the plate, however it is not used inthe experiments presented in this work; its placement is designed to measure the size of the vine’s canopy [see Grocholsky, Nuske,Aasted, Achar, & Bates (2011)]. In (c) the Bumblebee2 stereo camera can be seen pointed along the direction of travel to be used ina visual odometry algorithm to measure the position of the vehicle along the row.

to other vineyard vehicles is approximately the same speedas a pesticide spraying tractor and faster than a machineharvester.

In experiments before 2013, different cameras, cameralenses, and illumination sources were used. In 2010, a CanonSX200IS camera was used to image the fruit, and halogenlamps were mounted facing sideways toward the fruit. In2011 and 2012, we used a Nikon D300s camera to capture im-ages of the grape fruit, and an AlienBees ARB800 ring flashmounted around the camera lens to provide even lightingto the scene. In 2011, a 24MM F/2.8D AF NIKKOR lens wasused in conjunction with the Nikon camera, whereas in 2012a Nikon 18–55 mm f/3.5-5.6G ED II AF-S DX zoom lens wasused.

5.2. Datasets

The datasets analyzed consist of wine-grape varieties—Traminette, Riesling, Chardonnay, and Petite Syrah—and atable-grape variety called Flame Seedless. We demonstrateour method at a variety of stages during the growing sea-son, from just after the fruit begins setting right up until justbefore harvest. Over this time span, the berries range fromone-tenth their final size to almost fully grown. See Table Ifor details of the different datasets. See Figure 7 for imageexamples from the different datasets.

In each of the datasets, we collect harvest weightsof the fruit to evaluate against our image measure-ments.



Table I. Dataset location, time, and details.

Variety Location Date Days to harvest Trellis type Lighting No. Vines

Traminette Fredonia, NY Sep 2010 10 Vert. Shoot Pos. Day w/ lamp 98Riesling Fredonia, NY Sep 2010 10 Vert. Shoot Pos. Day w/ lamp 128Chardonnay Modesto, CA Jun 2011 90 Vert. Sprawl Night w/ flash 636Chardonnay Modesto, CA Jun 2013 75 Vert. Sprawl Night w/ flash 24Petite Syrah Galt, CA Jun 2013 75 Quad. w/ Vert. Shoot Night w/ flash 30Pinot Noir Galt, CA Jun 2013 75 Quad. w/ Vert. Shoot Night w/ flash 32Flame Seedless Delano, CA Jun 2011 40 Split-V Night w/ flash 88Flame Seedless Delano, CA Jul 2012 1 Split-V Night w/ flash 88Flame Seedless Delano, CA Jun 2013 7 Split-V Night w/ flash 88

5.2.1. Wine-grape Vineyards

The Chardonnay dataset was collected in Modesto, Cali-fornia in a sprawling vineyard that has guide wires that arelifted to tuck pendant shoots up such that it could be consid-ered a semivertical shoot positioned vineyard. We collectedimages on 636 vines on six rows of this vineyard. In 2011, theimages were collected just after the berries began to set at12 weeks before harvest. At this stage, the berries are verysmall, between 3 and 5 mm in diameter and one-tenth oftheir final weight. In 2013, the images were collected sev-eral weeks later in the relative growing season when theberries were larger at 0.8 g.

The Petite Syrah and Pinot Noir datasets were collectedin Galt, California, both in a split quadrilateral trained cor-don trellis system. The pruning practice and the trellis sys-tem both promote vertical growth of the shoots up and overtop guide wires such that the canopy grows up and overthe wires in a curtain. The fruit is mainly located close tothe cordon below the wires. The camera position in thisvineyard must be underneath the canopy looking up at thefruit-zone.

The Riesling and Traminette datasets were collectedfrom an approximately one-acre plot in Fredonia, NewYork. We used four rows of Traminette vines and fourrows of Riesling varieties, consisting of 224 vines total. TheTraminette were positioned at 8 ft spacing and the Ries-ling were positioned at 6 ft spacing, which totaled 450 m ofvines. Similarly to the Chardonnay vines sampled, the Ries-ling and Traminette vines were vertical shoot positioned,allowing fruit to be seen underneath the lifted vine canopy.

Basal leaf removal was performed for all of theChardonnay, Petite Syrah, Riesling, and Traminette vinesas per each vineyard’s standard operation. For all but theRiesling dataset, the leaf removal was performed only on thenorth side, and this is the side that was imaged. The practiceis commonly performed by vineyard owners to expose thefruit to the sun to change the flavor characteristics of thegrapes (Bergqvist, Dokoozlian, & Ebisuda, 2001; Crippen Jr& Morrison, 1986). Basal leaf removal also makes yield es-timation feasible after fruit-set at the end of the growing

season because the occluding canopy is removed from thefruit-zone.

5.2.2. Table-grape Vineyards

The Flame Seedless datasets were collected over threeyears—2011, 2012, and 2013—where each occasion con-sisted of a dataset of 88 vines of a split-V gable vineyard,in Delano, California. The vines are trained in a split sys-tem in which each vine is trained into two cordons and theshoots grow over a V-gable trellis such that the fruit hangsin two distinct sections at an angle on each side of the row.The canopy grows out over the V-gable trellis such that toview the clusters, either the canopy must be trimmed back(usually done close to harvest time to enable harvest crewsto access fruit) or a cane-lifting device [see Figure 1(b)] is re-quired to reveal the fruit. As per convention in table-grapecultivation, fruit-thinning, shoot-thinning, and leaf-pullingare performed quite rigorously to promote separation ofthe grape clusters from each other and from leaves. The re-sult of this separation of the fruit is that once the exteriorcanopy is removed or moved from between the camera andthe fruit, the cluster-to-cluster and vine occlusions are no-ticeably less for this type of vineyard than the wine grapevarieties (which is reflected later in Figure 13).

5.3. Berry Detection Performance

We evaluate the performance of our berry detection algo-rithm by first analyzing the keypoint detection algorithmsand then studying the feature classification.

5.3.1. Evaluation of keypoint detection

We take both the radial-symmetry detection algorithm andthe invariant-maximal detection algorithm and evaluatebased on how many grape centers are detected by eachalgorithm, as shown in Table II. The table illustrates that thetwo types of keypoint detection behave differently underdifferent conditions.



Figure 7. Example images of the different varietals from our yield prediction experiments.

Notable discrepancies between detectors are seen inthe 2010 datasets collected in natural illumination. Theinvariant-maximal detector does not perform well, due tothe lack of shading on the surface of the fruit. Also, the sameperformance is registered in the Flame Seedless 2011 datasetwhen a cross-polarized flash is utilized, also removing theshading on the surface of the fruit.

In most of the 2013 datasets, the invariant-maximal de-tector outperforms the radial-symmetry algorithm wherethe flash produces a peak in the middle of the fruit and a

gradual decrease in intensity from the curved surface. Theexception is the Chardonnay dataset, where it is believedthat a pesticide spray applied just before imaging causeda residue to be left on the surface of the fruit, causing areduction in the shading produced by the flash.

In general, the results indicate that multiple key-point detection algorithms are necessary to achieve highperformance in multiple different conditions. Further, wealso note that the actual location of the keypoints differsbetween algorithms and the location affects the feature



Figure 8. Example images demonstrating berry detection in different varietals.



Table II. Keypoint detection.

Dataset

Radial-symmetry

(Recall)

Invariant-maximal(Recall)

Riesling 2010 0.66 0.12Traminette 2010 0.89 0.09Chardonnay 2011 0.68 0.91Chardonnay 2013 0.86 0.77Flame Seedless 2011 0.89 0.30Flame Seedless 2012 0.73 0.54Flame Seedless 2013 0.81 0.85Petite Syrah 2013 0.87 0.96Pinot Noir 2013 0.75 0.91

descriptions. This was discovered since performancedropped when training the randomized KD-forest with onedetector type and testing with the other keypoint detectortype.

5.3.2. Evaluation of feature classification

As mentioned earlier in Section 3.2, we presented aset of three different texture feature descriptors that weevaluate within our algorithm from three broad featuretypes:

1. Filter banks: Gabor filters2. Local texture descriptors: SIFT3. Binary relations: FREAK

We evaluate their relative performance against eachother on a set of datasets collected in a variety of illumi-nation conditions; natural illumination, flash illumination,and cross-polarized flash illumination. We use training setsof 20 random images from each of the three types of illu-mination conditions and manually mark the grape berriesin each of the images for ground-truth. Points identifiedby the algorithm as berries that are near our manually de-fined ground-truth are considered true positives, and theyare false positives if they are not near any manually definedberries. Figure 9 presents the true positive rates and falsepositive rates through varying values for τr (see Section 3.2)from 0 to 1.

We see in Figure 9(a) that a bank of Gabor filtersperforms substantially better than the SIFT and FREAKfeatures, with color-only having the second best perfor-mance. This naturally illuminated dataset was collectedafter verason (onset of coloring), and even though this wasa green grape variety (Traminette), there was noticeableyellowing of the fruit in comparison to the leaves, indicatingwhy the color features do well. The images are noticeablyless clear and noisy than the other types of illuminationconditions, and as expected the Gabor filter responses that

are placed to encompass each image patch are more robustto these lower quality images.

In Figure 9(b) we see that FREAK features perform bestat classifying the berries under conventional flash illumi-nation. We suspect the curved surface of the grape is high-lighted by shading from the flash and is captured well in thedescription of the binary intensity relations of FREAK. Thisdataset was captured before verason, and the fruit is muchmore similar in color to the foliage on the vine. Therefore,we see, as expected, that color alone does not perform well.

Figure 9(c) presents the classification performance withthe cross-polarized flash. This type of illumination will re-move the glare from the flash, which reduces the glare fromthe surface of leaves and also removes any glare and shadingfrom the grape surface. In this case, SIFT performs the best,since the shading is not prominent on the grape surface. Wehypothesize that the gradient orientation histogram of SIFTis suited to describing the curved contour of the grapes,which is the key visible cue in the absence of shading ordistinguishing color.

Finally, we evaluate which descriptor generalizes thebest over different datasets and illumination conditions. Wetest on a dataset using training data from the other datasets.We find that SIFT is best at generalizing across conditions,whereas the Gabor and FREAK descriptions are less able togeneralize. In this test, we did not use color in the features,as we see here that color-only performs poorly, since colorvaries a lot between datasets and illumination conditions.This shows a contrast, as color was seen to be a valuablecue when a classifier was trained and then utilized over thesame dataset. In conclusion, it is possible to operate undernew conditions without retraining the classifier, but colorshould not be included in the feature description, and theresults indicate that SIFT is the texture descriptor that is bestat generalizing.

5.3.3. Timings of Berry Detection Algorithm

Evaluating our algorithm on an Intel i5-2500K Quad core3.3 GHz CPU, with 16GiB of RAM, produces the timingbreakdown in Table III.

There has not been great effort into optimizing algo-rithms for speed. In particular, the Gabor filters use an inef-ficient implementation. However, the efficiency of the sys-tem is currently appropriate for our existing deployments,given that the time before harvest when the images werecaptured is far greater than the time to process all imagesin a dataset. Further effort into optimization of algorithmscould bring the system close to real-time performance.

5.4. Evaluating Visible Berries and Self-occlusion

First, we evaluate the occlusion of berries within a cluster bythe outer layer of clusters (kb) and study some approachesto potentially improve the estimate of the number of hidden



Figure 9. Classification performance curves computed from various feature descriptor types evaluated on a set of differentillumination conditions. Natural illumination (a) causes grapes to be imaged with noise from uncontrolled lighting. Gabor filtersperform best in this setting, as they are not as responsive to local image noise. Under conventional flash illumination (b) intenseshading is seen upon the imaged grapes. The FREAK performs best with this shading on the grapes, which we hypothesizeis because the relative-intensity comparisons of FREAK are successful at describing the steady decrease in intensity across theshaded grapes. Under cross-polarized illumination (c) grapes are free of noticeable shading. The exterior contour of the grape isthe most noticeable feature on these grapes, and the SIFT descriptor seems to be better than FREAK at describing the value of thisstrong contour on the grape’s exterior. When comparing across datasets, SIFT is the most generalizable descriptor across imagingconditions, as seen in (d).

berries. For this specific study, we use a controlled labora-tory environment where we collected images individuallyof 56 grape clusters. We use ripe clusters of the ThompsonSeedless variety. For each cluster, we collected several

images from different orientations, at a fixed distance, andwe collected a weight and a count of the number of berries.In the laboratory dataset, we do not use our automatic de-tection algorithm and instead hand-mark all berries visible



Table III. Berry-detection timings.

Algorithm step Timing (s)

Image load and preprocess 0.097Keypoint detection–radial symmetry 0.65Keypoint detection–invariant maximal 0.55Feature extract–Gabor 6.42Feature extract–FREAK 0.045Feature extract–SIFT 1.09Grape/nongrape classification 0.017Group berries into clusters 0.22Total–slowest (radial/Gabor) 7.4Total–fastest (maximal/FREAK) 0.92

Table IV. Cluster model correlation to fruit weight (laboratorydataset).

Measure-type R2 correlation

Mean-squared

error

Total berry count(upper bound)

0.95 9.3%

2D Visible berry count[Eq. (4)]

0.88 15.4%

Ellipsoid 3D model[Eq. (6)]

0.85 17%

Convex hull 3D model[Eq. (5)]

0.92 13.7%

within the images to replicate a perfect detection algorithmand remove any bias from errors in the detection algorithm(kd and kr ). Also, in the laboratory dataset there are nobiases from the vine (kv) or from other clusters (kc) andhence we can isolate and study the bias from self-occlusions(kb).

Initially, we compare the total berry count (gatheredmanually) of each cluster against its weight, Table IV. Thecorrelation score for total berry count to weight is R2 = 0.95with a mean-squared error from a least-squares fit of 9.3%.We consider this an upper bound for the yield predictions,as the best yield prediction we could achieve depends onaccurately knowing the berry count.

Next, we study different image measurements startingwith the visible berry count, and we present the results inTable IV. The visible berry count correlates with R2 = 0.88,which provides a mean-squared error of 15.4%; similar vis-ible berry correlations have been found in Grossetete et al.(2012). The error is just 6% greater than the total berry count,and it indicates that a similar fraction of visible berriesis present for small clusters as with large. The ellipsoidalmodel has a correlation score of R2 = 0.88 and the lowestmean-squared error of 17%. Even though the ellipsoidalmodel attempts to predict the occluded berries behind the

Table V. Cluster model correlation to yield (vineyarddatasets).

Measure-type R2 correlation

2D Visible berry count [Eq. (4)] 0.75Ellipsoid 3D model [Eq. (6)] 0.61Convex hull 3D model [Eq. (5)] 0.41

visible layer of berries, it correlates with a lower score thanthe visible berry measure. The ellipsoidal model could beless accurate because it violates one of our assumptions:the clusters do not have uniform density, or the clusters arenot ellipsoidal, or the model could suffer from errors in thedesignation of the cluster contour.

The final image measurement model we evaluate isthe convex hull in Table IV. The correlation measures atR2 = 0.92, which is the best of the three image measure-ments we study. One possible reason for the high correlationis because it encompasses the entire cluster contour. There-fore, it includes a measure of the partially visible berries aswell as the completely visible berries, thus being more ac-curate than visible berry count alone. Despite finding thatthe contour area in the image is a more accurate measureother than the visible berry count, we do not yet deploy thismeasure outside the laboratory environment.

We evaluate our different cluster models in a vineyard.Table V presents results from the Traminette dataset. In thevineyard setting, several clusters are visible in each imageand we have yet to develop a technique for successfully seg-menting one cluster from another—a requirement of the vol-umetric ellipsoid and convex hull models. In practice, clus-ters grow to touch one another and it is difficult—withoutphysically moving the clusters—to determine which berriesbelong to which cluster. Hence, at present we have only beenable to demonstrate precise detection of individual berries,regardless of to which cluster they belong, and therefore inthe following vineyard results we consider just the visibleberry count.

5.5. Spatial Accuracy—Berry Count Correlation toVine Yield

We compare our berry counts against actual harvest weightscollected from the Traminette, Riesling, Flames Seedless,and Chardonnay datasets. First, we register images to vinespaces, and then we assign berry count measurements toeach vine space. For the Traminette and Riesling datasets,we manually defined the vine spaces in images, but in themore recent Flame Seedless and Chardonnay datasets wedeployed the stereo camera to perform this process auto-matically.

Once registered to specific vines, we compare our au-tomated berry counts with the harvest crop weights (see



Figure 10. Correlation between our detected berry count and harvest crop weights. The black lines show the one-sigma standarddeviation within the measurements, the red line represents a linear fit, and each of the blue data points represents the rawmeasurement of a single vine. The caption below shows the R2 correlation score. These graphs illustrate our method generating anondestructive measurement at every vine, whereas in conventional practice very sparse destructive samples are taken.

Figure 10 for details). The figure shows the raw data pointsand the distribution of measurements. Our automaticallygenerated berry counts produced a linear relationship withactual harvest crop weights with correlation scores rang-ing from R2 = 0.6 to 0.73 depending on the dataset (seeFigure 10 for details). The R2 correlation score quantifieshow much of the actual variance in yield our method canestimate. Thus, we capture 60–75 % of the variance. Simi-lar correlation scores to vine yield were achieved by Diagoet al. (2012), although it is difficult to do a direct compar-ison because in that work the yield distribution was ar-tificially induced by stages of defruiting, which reducesthe true variance in vine occlusions and cluster-to-clusterocclusions. Furthermore, without considering individualberries, the correlation scores will not hold once the berrysize changes since simple pixel count is used as an imagemeasurement.

Our measurements achieve good spatial correlation,first through the high precision of our detection algorithm,which rarely counts false positives (Nuske et al., 2011), andalso because there is some consistency in occlusion levelacross the vineyard. Further, increasing the correlation scorecould come from possible improvements to the detection al-gorithm and including a method to estimate the berries thatare not visible to the camera. The variance unexplained,derived by simply subtracting 1 − R2, is around 25–40 %,which could either be due to variance in our detection per-formance (i.e., changes in the recall of the algorithm) or tovariance in the occlusions, caused by either cluster, self-occlusions, or the vine.

Finally, we demonstrate how to optimize our systemfor spatial yield accuracy. Section 4.2 identified the metricand means to optimize the classification algorithm. Herewe evaluate this approach by configuring with the set of



Figure 11. The randomized KD-forest voting threshold τr is optimized against a normalized deviation in true positive rate (σα)and normalized false positive (σβ ) [Eq. (16)] to maximize resultant R2 correlation from berry counts, which is the quantitativemeasure of spatial yield accuracy.

different feature descriptors, and we optimize the key classi-fication parameter, τr . The variance in both the true positiverate and the false positive level is dependent on τr , and weoptimize over a small set of hand-labeled images (a set of 10images for a given vineyard dataset) and then evaluate theresultant R2 value against the harvest data; see Figure 11.

For the three types of feature descriptors, the graphsidentify that minimizing the error in Eq. (15) when con-figuring the algorithm produces a similar peak in r2 whencorrelating to harvest yield. For Gabor filters and FREAKfeatures, the optimal τr value is between 40% and 50%, al-though it is noticeable that the error metric is slightly shiftedfrom the r2 peak, which equates to less than a 5% reductionin spatial yield accuracy. For SIFT features, the τr parame-ter is optimal at 80%, indicating the SIFT feature space hasmuch different properties than FREAK and Gabor, and thatin the SIFT feature space the positive berries are in a tightercluster, but with negative features distributed sparsely butevenly. Nevertheless, the graphs reveal that the optimiza-tion strategy is adept and produces substantial increases inspatial yield accuracy in excess of 20%.

5.6. Yield Estimate Accuracy—Berry CountCalibrated to Harvest Yield

We have seen a linear correlation between our image berrycounts and harvest yield illustrating that our estimates cap-ture most of the true spatial variance in yield. The next met-ric to analyze is the accuracy of our system in estimating theoverall yield. The calibration procedure to relate the imagemeasurements to yield can be conducted in two differentfashions. Either harvest and image data from prior seasonscan be used, or destructive calibration samples could be col-lected at the time of imaging. Using prior seasons harvestdata theoretically requires less human labor, since record-ing total yield at the time of harvest is a standard processin grape production, and further an entire sample of yield

from all vines can be collected. The negative aspects to us-ing prior harvest seasons are that you need one full seasonbefore you can establish a calibration for computing yieldpredictions, and the consistency of calibrations from sea-son to season needs to be established so that the integrityand accuracy of the calibration are known. Using manualsamples collected at the time of imaging has the advan-tage that a calibration can be established immediately andthat the calibration can be assured to be representative ofthe current vineyard variety and training system. However,the human labor involved restricts the size of the sampleset and, in turn, may adversely affect the accuracy of thecalibration.

We present yield prediction results using calibration es-tablished from prior harvest data. We use different featuredescriptors for different datasets, as per the earlier discov-ery that different features work better under different con-ditions. Gabor filters are used for the Riesling, Traminette,and Flame Seedless datasets, and FREAK feature descrip-tors are used for the Chardonnay and Petite Syrah datasets.The same classification threshold is used throughout, al-though we also demonstrate in Figure 12 that we correctfor the particular classification performance using Eq. (10).Hence, a varying classification threshold can be compen-sated for, except at the extreme high and low classificationthresholds which cause too much variance in classificationperformance. We also find that for some datasets, somekeypoint-types and feature-types are not able to be com-pensated with Eq. (10), meaning that a changing thresholdchanges the fruit estimate. However, we repeat training forall features and keypoint combinations and pick the combi-nation that has the most stable fruit estimate and consensuswith fruit estimates of other types of features/keypoints.

The images for all datasets are processed otherwise inthe same manner, with only one difference being an illumi-nation normalization applied to the 2013 images at prepro-cessing to correct for the light attenuation at the peripheries



Figure 12. Illustration of how to correct for variations in berry detection performance due to variations in visual appearance orimaging conditions. The ratio of raw detected fruit to total fruit (a) is corrected by Eq. (10) to compute visible fruit (b) and plottedat different classification thresholds. The visible fruit is computed with measurements of berry detection accuracy computed froma small set of labeled images. The visible fruit in comparison to detected fruit is stable, with most of the different classificationthresholds except the extremal thresholds. The extremal thresholds, which are shaded gray, correspond to deviation errors greaterthan 50% of the maximum from Figure 11(b). The stability in estimated visible fruit in comparison to the instability of the rawdetected fruit highlights the necessity to compensate for detection performance when deriving yield estimates. The graph in (c)shows the yield prediction error when using the detected or visible fruit ratio computed from prior years with image measurementsfrom the present year. The visible fruit estimate is stable even with substantial changes in the classification parameter [Eq. (10)].

Table VI. Yield estimation accuracy—Flame Seedless 2013.

Calibration source Yield prediction error

Flame Seedless 2011 6.48%Flame Seedless 2012 11.65%Flame Seedless 2011 and 2012 9.07%

of the image. We compute the visible fruit using Eq. (10) andthen compute the visibility calibration function [Eq. (17)]from datasets collected in prior years, using the mean berryweight to compute an estimated total number of berries.We then apply the calibration to the image berry countscollected in 2013 and evaluate yield accuracy.

The Flame Seedless vineyard also has a consistent re-lationship between subsequent years. However, the FlameSeedless vineyard is grown in a substantially different man-ner than the wine grape vineyards. The vines are trainedon a V-gable trellis and the shoots are thinned, the fruitis thinned, and leaves are pulled from in front of clusterssuch that there is far less occlusion from foliage and fromcluster-to-cluster occlusion than the wine grape vineyards.This difference can be seen in the two sets of calibration linesin Figure 13, one for a table-grape split-cordon V-trellis andone for a wine grape single cordon. The yield predictionerror for the table-grape data is between 6% and 11.5% ac-curacy from prior harvest data, and using all prior data tocalibrate 2013 image data to an error of 9.07%; see Table VI.

Table VII shows the application of visibility calibra-tion from the Chardonnay 2011 dataset and applied to theChardonnay 2013 dataset, which achieves an accuracy of

Table VII. Yield estimation accuracy—Chardonnay 2013.

Calibration source Chardonnay 2013

Chardonnay 2011 −2.47%

2.5%. This result demonstrates that destructive calibrationsamples at the time of imaging may not be required, andprior season calibration may be sufficient. Further, calibra-tion taken from the Traminette or Petite Syrah vineyards alsoachieves good accuracy on the Chardonnay 2013 dataset of4% and 5%, respectively. However, calibration from Rieslingor Pinot Noir vineyards produces 17% and 29% error, whichwe believe is due to lower levels of vegetation in the Rieslingvineyard, and our hypothesis that the Pinot Noir vineyardhad a combination of less vegetation in the fruiting zoneand the system also detected a small percentage of grapeson the other side of the split cordon. The conclusion is thatcalibration must be site-specific to achieve high accuracy.For the Chardonnay vineyard the accuracy is 2.5%, and inthe Flame Seedless vineyard the accuracy is 9% on averagewhen calibrated from prior years in the same vineyard.

6. LESSONS LEARNED

We draw a number of lessons from this work. One of themost important is that consistent performance is required toensure that image-measurements stay well-related to yield.To maintain consistent performance, both imaging and al-gorithmic robustness is important. The illumination andimaging configuration is vitally important, and operating



Figure 13. All yield and image berry count data. Calibration lines from prior seasons are drawn from Chardonnay 2011 and FlameSeedless 2011–2012 harvests. The site-specific calibration is consistent over years, within the Chardonnay and Flame Seedlessvineyards, highlighting that destructive calibration samples at the time of imaging are potentially not needed. Similar trellisingtypes of Petite Syrah and Traminette also show similar visibility to the Chardonnay. However, it must be noted that Riesling andPinot Noir vineyards have noticeably higher visibility despite being from similar wine trellising, which indicates, contrary to pasthypothesis, that site-specific calibration must be collected.

at night is the most reliable way to control the imaging.Through the development in this work, it became apparentthat the imaging and illumination design was a major chal-lenge for practical and robust sensing. Careful engineeringis required to achieve the desired characteristics in the il-lumination power, the depth-of-focus, the frame-rate, flashrecycle time, motion blur, and image quality. A naive ap-proach to deploying a camera in a vineyard will simplybe unsuccessful. Position estimation and image registrationare also essential to maintain consistency, by accounting forthe vehicle’s motion along the vineyard rows and also thevariable distance of the camera from the fruit-wall. If bi-ases exist in the position estimation and image registration,the result will be an inaccurate relationship between image-measurements and yield. Finally, we have seen that witha few manually labeled images (we use sets of 10 labeledimages), it is possible to characterize detection performanceand configure the algorithm to increase the accuracy of theyield estimates. While it does take time to manually labelimages, this requires far less manual labor in comparison tothe alternative of collecting destructive yield samples.

In terms of what has been learned from the multipleyears, we reflect on the results presented in Nuske et al.(2012), and we see that the calibration relationships for thewine grapes aligned more accurately from 2010 to 2011,when errors were 4–5 %. The 2013 predictions report an ac-curacy of 3–11 % of total yield. The slight decrease in theworst-case accuracy for 2013 could be due to a few reasons.

One may be the different intensity profile created by thenew flash setup in 2013. In 2010 and 2011, a more uniformlight source was used, whereas in 2013 the light was fo-cused using reflectors to project the light forward for powerefficiency reasons. This causes a gradual decline in the lighttoward the edge of the images, and a slight decrease inthe amount of berries detected toward the peripheries ofthe image, which we do correct as best we can with inten-sity normalization across the image. Changes in the cameraplacement on the vehicle may introduce some unexpectedvariations in the fruit occlusion and appearance variationsof both the fruit and foliage. Nevertheless, despite a changein camera and flash and vine appearance, the yield predic-tion accuracy is still within 6% on average. It is logical toconclude that keeping the imaging position and configura-tion consistent, there may be a further increase in the accu-racy of yield predictions. We are also looking at methods topreprocess images to automatically normalize the lightingdistributions between datasets and extend our approach toautomatically configure the fruit detection algorithms (Sec-tion 4.2) to be more robust to dataset variations. Anotherlearned lesson is that the vine-trellis type and manage-ment practices of the grower are also critical considerations.For instance, for the vineyards studied in this work, thetrellis, the leaf-pulling, shoot/cluster thinning, and shootpositioning varied between the wine-grape vines and table-grape vines. These differences combined to affect the rela-tionship between image-measurements and yield, although



it was shown that the relationship is stable in the samevineyard over several years. For example, the lower errorsrecorded in the wine-grape vineyards are expected to bedue to more consistent visibility of the fruit zone and/ormore consistent berry detection performance in these vine-yards. In the table-grape vineyards the fruit zone is muchlarger, and due to the split vines, some of the fruit fromone side would grow close to fruit from the other side, andthere were challenges involved in positioning the cameraand configuring the algorithm to not incorrectly count fruitfrom the other cordon.

7. CONCLUSION

The method presented in this paper demonstrates nonde-structive yield estimation that can enable growers to per-form management in a far more effective manner than waspreviously possible. Unlike previous work in this area, wedemonstrate a complete system over several growing sea-sons and tens of acres that can be deployed efficiently. Weaddress the key challenges relating to distinguishing theberry appearance, the challenges in imaging, estimating theposition of the vehicle, and reducing variance in measure-ment, all together providing an automated solution to large-scale, high-resolution yield estimation.

The algorithm presented here exploits all visual cues ofgrapes, namely shape, texture, and color, enabling a detec-tion system applicable across many different varieties andimaging conditions. While ideally there would be one par-ticular visual texture-cue for all conditions, we discoveredthat different texture-descriptors work well in different con-ditions. While this poses a challenging question of whichdescriptor to use within the algorithm, we have shown thatwith a small amount of labeled data, it is possible to quan-tify the classification performance of individual feature de-scriptors at the training stage and identify the most optimaldescriptor. We also show how to tune the algorithm anddescriptor to maximize the accuracy of the resultant yieldestimates. In particular, we distilled the model relating im-age measurements to yield down to the components thatdirectly affect yield accuracy, and we were able to showthat our theory results in substantial increases in accuracy.

We also attempted to improve the estimate of berrieswithin a cluster with a volumetric prediction of cluster size,and we demonstrated a method that increases the accu-racy of cluster-size estimates within a laboratory setting.When applied to real-world data, the volumetric methodwas found to decrease accuracy because grape clusters growalongside one another, which makes it difficult to discoverthe boundary of neighboring clusters, leading to gross errorsin extrapolating cluster size. Therefore, a first-order modelthat relates the visible berry count to yield is the most ac-curate in practice. An interesting observation can be drawnthat humans are better at counting clusters per vine andweighing individual clusters, whereas, conversely, it seems

robotic sensing struggles to accurately count mature grapeclusters. Instead, it is easier to use robotic sensing to countthe number of berries on a vine, a measure that would notbe easy for a human to directly produce.

Finally, we identify stable calibration relationships forvineyards that hold true from year to year within 10% pre-diction of total yield. The results indicate that the system canbe efficiently deployed without new labor-intensive man-ual calibration, given that a site-specific calibration has beenestablished in prior years.

ACKNOWLEDGMENTS

This work supported by National Grape and Wine Initiative([email protected]) and United States Department of Agricul-ture (Award number: 2012-67021-19958).

REFERENCES

Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). FREAK: Fastretina yeypoint. In IEEE Conference on Computer Visionand Pattern Recognition, New York.

Berenstein, R., Shahar, O. B., Shapiro, A., & Edan, Y. (2010).Grape clusters and foliage detection algorithms for au-tonomous selective vineyard sprayer. Intelligent ServiceRobotics, 3(4), 233–243.

Bergqvist, J., Dokoozlian, N., & Ebisuda, N. (2001). Sunlightexposure and temperature effects on berry growth andcomposition of cabernet sauvignon and grenache in thecentral San Joaquin Valley of California. American Journalof Enology and Viticulture, 52(1), 1–7.

Blom, P., & Tarara, J. (2009). Trellis tension monitoring improvesyield estimation in vineyards. HortScience, 44, 678–685.

Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). Brief:binary robust independent elementary features. In Pro-ceedings of the 11th European Conference on ComputerVision: Part IV, ECCV’10 (pp. 778–792). Berlin, Heidelberg:Springer-Verlag.

Clingeleffer, P., Dunn, G., Krstic, M., & Martin, S. (2001). Cropdevelopment, crop estimation and crop control to securequality and production of major wine grape varieties: Anational approach. Technical report, Grape and Wine Re-search and Development Corporation, Australia.

Crippen, D., Jr., & Morrison, J. (1986). The effects of sun expo-sure on the compositional development of cabernet sauvi-gnon berries. American Journal of Enology and Viticulture,37, 869–874.

Dey, D., Mummert, L., & Sukthankar, R. (2012). ClassifiÞcationof plant structures from uncalibrated image sequences. InIEEE Workshop on the Applications of Computer Vision(WACV).

Diago, M.-P., Correa, C., Millan, B., Barreiro, P., Valero, C., &Tardaguila, J. (2012). Grapevine yield and leaf area esti-mation using supervised classification methodology onrgb images taken under field conditions. Sensors, 12(12),16988–17006.



Dunn, G., & Martin, S. (2004). Yield prediction from digitalimage analysis: A technique with potential for vineyardassessments prior to harvest. Australian Journal of Grapeand Wine Research, 10, 196–198.

Federici, J., Wample, R., Rodriguez, D., & Mukherjee, S. (2009).Application of terahertz gouy phase shift from curvedsurfaces for estimation of crop yield. Applied Optics, 48,1382–1388.

Grocholsky, B., Nuske, S., Aasted, M., Achar, S., & Bates, T.(2011). A camera and laser system for automatic vine bal-ance assessment. In American Society of Agricultural andBiological Engineers (ASABE) Annual International Meet-ing.

Grossetete, M., Berthoumieu, Y., Costa, J. P. D., Germain, C.,Lavialle, O., & Grenier, G. (2012). Early estimation of vine-yard yield: Site specific counting of berries by using asmartphone. In International Conference of AgriculturalEngineering—CIGR-AgEng.

Hung, C., Nieto, J., Taylor, Z., Underwood, J., & Sukkarieh, S.(2013a). Orchard fruit segmentation using multi-spectralfeature learning. In Intelligent Robots and Systems (IROS),2013 IEEE/RSJ International Conference (pp. 5314–5320).

Hung, C., Underwood, J., Nieto, J., & Sukkarieh, S. (2013b). Afeature learning based approach for automated fruit yieldestimation. In 9th International Conference on Field andService Robotics (FSR).

Jimenez, A., Ceres, R., & Pons, J. (2000). A survey of computervision methods for locating fruit on trees. In Transactionof the ASAE, 43, 1911–1920.

Kitt, B., Geiger, A., & Lategahn, H. (2010). Visual odometrybased on stereo image sequences with ransac-based outlierrejection scheme. In IEEE Intelligent Vehicles Symposium,San Diego.

Laws, K. (1980). Textured image segmentation. Ph.D. thesis,University of Southern California.

Lepetit, V., Lagger, P., & Fua, P. (2005). Randomized trees forreal-time keypoint recognition. In Computer Vision andPattern Recognition, 2005. CVPR 2005. IEEE Computer So-ciety Conference (vol. 2, pp. 775–781).

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vi-sion, 60, 91–110.

Loy, G., & Zelinsky, A. (2003). Fast radial symmetry for detect-ing points of interest. IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, 25, 959–973.

Marden, S., Liu, S., & Whitty, M. (2013). Towards automatedyield estimation in viticulture. In Australasian Conferenceon Robotics and Automation.

Martinez-Casasnovas, J., & Bordes, X. (2005). Viticultura de pre-cision: Prediccion de cosecha a partir de variables del cul-tivo e ındices de vegetacion. Revista de Teledeteccion, 24,67–71.

Nuske, S., Achar, S., Bates, T., Narasimhan, S., & Singh, S. (2011).

Yield estimation in vineyards by visual grape detection. InProceedings of the 2011 IEEE/RSJ International Confer-ence on Intelligent Robots and Systems.

Nuske, S., Gupta, K., Narasihman, S., & Singh, S. (2012). Model-ing and calibration visual yield estimates in vineyards. InProceedings of the International Conference on Field andService Robotics.

Rabatel, G., & Guizard, C. (2007). Grape berry calibration bycomputer vision using elliptical model fitting. In ECPA2007, 6th European Conference on Precision Agriculture.

Serrano, E., Roussel, S., Gontier, L., Dufourcq, T., et al. (2005).Early estimation of vineyard yield: Correlation betweenthe volume of a vitis vinifera bunch during its growth andits weight at harvest. In FRUTIC 05, Information and tech-nology for sustainable fruit and vegetable production: 7thFruit Nut and Vegetable Production Engineering Sympo-sium (vol. 2, pp. 311–318).

Singh, S., Bergerman, M., Cannons, J., Grocholsky, B., Hamner,B., Holguin, G., Hull, L., Jones, V., Kantor, G., Koselka,H., Li, G., Owen, J., Park, J., Shi, W., & Teza, J. (2010).Comprehensive automation for specialty crops: Year 1results and lessons learned. Journal of Intelligent Ser-vice Robotics, Special Issue on Agricultural Robotics, 3(4),245–262.

Swanson, M., Dima, C., & Stentz, A. (2010). A multi-modal sys-tem for yield prediction in citrus trees. In ASABE AnnualInternational Meeting, Pittsburgh, PA.

Tabb, A., Peterson, D., & Park, J. (2006). Segmentation of applefruit from video via background modeling. In AmericanSociety of Agricultural and Biological Engineers AnnualInternational Meeting.

Taylor, J., Tisseyre, B., Bramley, R., Reid, A., Stafford, J., et al.(2005). A comparison of the spatial variability of vineyardyield in European and Australian production systems. InPrecision agriculture’05. Papers presented at the 5th Euro-pean Conference on Precision Agriculture, Uppsala, Swe-den. (pp. 907–914). Wageningen Academic Publishers.

Wang, Q., Nuske, S., Bergerman, M., & Singh, S. (2012). Au-tomated crop yield estimation for apple orchards. In Pro-ceedings of the International Symposium of ExperimentalRobotics.

Wolpert, J. A., & Vilas, E. P. (1992). Estimating vine-yard yields: Introduction to a simple, two-step method.American Journal of Enology and Viticulture, 43,384–388.

Yang, L., Dickinson, J., Wu, Q. M. J., & Lang, S. (2007). A fruitrecognition method for automatic harvesting. In Mecha-tronics and Machine Vision in Practice, 2007. M2VIP 2007.14th International Conference (pp. 152–157).

Zhou, R., Damerow, L., Sun, Y., & Blanke, M. (2012). Usingcolour features of cv. agalaaapple fruits in an orchard inimage processing to predict yield. Precision Agriculture,13(5), 568–580.


Documents

Automated Visual Yield Estimation in Vineyards · 2014-10-17 · Journal of Field Robotics 31(5), ... Nuske et al.: Automated Visual Yield Estimation ... cant differences are in terms