Drift Compensation in AFM-Based Nanomanipulation by Strategic Local Scan

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 9, NO. 4, OCTOBER 2012 755

Drift Compensation in AFM-BasedNanomanipulation by Strategic Local Scan

Guangyong Li, Member, IEEE, Yucai Wang, Student Member, IEEE, and Lianqing Liu, Member, IEEE

Abstract—The drift distorts the atomic force microscopy (AFM)images as the time taken to acquire a complete AFM image is rel-atively long (a few minutes). As the AFM image is used as a ref-erence for most manipulation mechanisms, the image distorted bydrift will cause problems for AFM-basedmanipulation because thedisplayed positions of the objects under nanomanipulation do notmatch their actual locations. The drift during manipulation, simi-larly, will further exacerbate the mismatch between the displayedpositions and the actual locations. Suchmismatch is amajor hurdleto achieve automation in AFM-based nanomanipulation. Withoutproper compensation, manipulation based on a wrong displayedlocation of the object often fails. In this paper, we present an algo-rithm to identify and eliminate the drift-induced distortion in theAFM image by applying a strategic local scanmethod. Briefly, afteran AFM image is captured, the entire image is divided into severalparts along vertical direction. A quick local scan is performed ineach part of the image to measure the drift value in that very part.In this manner, the drift value is calculated in a small local areainstead of the global image. Thus, the drift can be more preciselyestimated and the actual position of the objects can be more ac-curately identified. In this paper, we also present the strategy toconstantly compensate the drift during manipulation. By applyinglocal scan on a single fixed feature in the AFM image frequently,the most current positions of all objects can be displayed in theaugmented reality for real-time visual feedback.

Notes to Practitioners—Fabrication of nanoscale devices byAFM-based manipulation has attracted much attention recently.However, it is a sequential approach and therefore the throughputis low. To increase the efficiency of AFM-based nanomanip-ulation, automation is considered to be the ultimate solution.Unfortunately, automation in AFM-based nanomanipulation isfacing several technical hurdles. The thermal drift is one of them.Because of the presence of thermal expansion and contraction,the relative position between the tip and the sample undergoesdrift with respect to time. Therefore, the AFM images are oftendistorted due to the thermal drift because the time taken toacquire a complete AFM image is in the range of minutes. Thispaper tries to tackle this problem by compensating the thermaldrift through direct measurement of the thermal drift by localscan. The direct measurement not only can be used to correct

Manuscript received December 07, 2011; revised June 14, 2012; acceptedJuly 20, 2012. Date of publication August 28, 2012; date of current versionOctober 02, 2012. This paper was recommended for publication by AssociateEditor S. Fatikow and Editor K. Bohringer upon evaluation of the reviewers’comments. This work was supported in part by the U.S. National Science Foun-dation (NSF) under Grant CNS-1035563.G. Li is with the Department of Electrical and Computer Engineering, Uni-

versity of Pittsburgh, Pittsburgh, PA 15261 USA (e-mail: [email protected]).Y. Wang was with the Department of Electrical and Computer Engineering,

University of Pittsburgh, Pittsburgh, PA 15261 USA. He is nowwith the Depart-ment of Electrical and Computer Engineering, McGill University,Montreal, QCH3A 2A7, Canada (e-mail: [email protected]).L. Liu is with the Shenyang Institute of Automation, Chinese Academy of

Science, Shenyang, Liaoning 110016, China (e-mail: [email protected]).Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TASE.2012.2211077

the drift-induced distortion in the initial AFM image, it can alsobe used to compensate the real-time drift during manipulation.Starting from a drift-clean image at the beginning and withdrift being compensated in real-time during manipulation, theAFM-based nanomanipulation can be more accurate and efficient.The research results here is one of the important steps to achievefull automation in AFM-based nanomanipulation.

Index Terms—Atomic force microscope (AFM), drift compensa-tion, nanomanipulation.

I. INTRODUCTION

A LTHOUGH a nanorobot constructed of molecularcomponents still remains a hypothetical concept, the

nanorobotics, the technology that allows for precise interac-tions with nanoscale objects with nanoscale resolution, hasbeen enabled by today’s high-resolution microscopy instru-ments such as atomic force microscope (AFM) [1]. AFM wasinitially invented as a “nano-imaging” instrument to observethe sample surface down to nanometer scale. It has become anindispensable tool for today’s research in nanoscale science andengineering. In addition to the imaging mode, an operator cancommand the motion of the AFM tip in an arbitrary manner,thus to manipulate nanoscale objects with nanoscale precision.Since the first try using AFM to manipulate nano-objects [2],many research groups have been working on AFM-basednanomanipulation [3]–[7]. Tasks such as pushing, pulling, cut-ting, bending, kinking, rolling, and sliding have been reportedin the literature [3]–[6], [8]–[10].The AFM-based nanomanipulation is a promising nanofab-

rication technique that combines both “top-down” and“bottom-up” advantages [11]. The major challenge forAFM-based nanomanipulation is the difficulty to achievethe real-time visual feedback because of the slow speed ofAFM scanning. In our recent work [12], we have developed anaugmented reality system, which can update the manipulationresults in real-time by the prediction from position controland compensation, motion behavior models of nano-objects,interaction among tip, surface, and objects [11], as well as bycorrections from local scans [13], [14]. Using this real-timesystem, nanodevices have been successfully fabrication byAFM-based nanomanipulation [15].Even with this versatile system, the throughput is still very

low. Therefore, automation in AFM-based nanomanipulationis considered as the ultimate solution. Unfortunately, the ac-complishment of full automation in AFM-based nanomanipu-lation is still facing several hurdles. One of these hurdles is the

1545-5955/$31.00 © 2012 IEEE

756 IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 9, NO. 4, OCTOBER 2012

thermal drift during the course of both imaging and manipula-tion. When nanomanipulations are performed in ambient con-ditions without rigorous environmental controls, the positionof objects displayed in the augmented reality may not be theexact location of the object due to the thermal drift. In addi-tion, the AFM image itself that has been used to initially con-struct the augmented reality may also have been distorted bythe drift because it usually takes a few minutes to capture anew AFM image and significant drift may occur during theimage capturing. The distortion in the image caused by randomdrift will significantly slow down the AFM-based nanomanip-ulation because more local scan corrections are needed whenlarge drift exists in the initial image. Therefore, solving theproblem of thermal drift is the primary step towards accurateand high-throughput nanomanipulation.Thermal drift cannot be compensated by the closed-loop

feedback equipped in the modern AFM system for compen-sation of hysteresis and creep effects because the closed-loopfeedback controls scanner to the desired position relative toitself not to its counterpart (stage for scanning-head type ofAFM or tip for scanning-stage type of AFM) [16]. Many re-searchers have tried to address drift distortion issue in scanningprobe microscopy (SPM). To compensate the drift distortionwithin the AFM image, correction after image acquisition is theusual way. After measuring the drift velocities either throughimaging known structures [17]–[19] or by cross correlation ofsubsequent images [20], [21], the thermal drift can be compen-sated thus to remove the distortion in the image. The correctionafter image acquisition, however, is unable to register therelative position between the tip and the nano-objects, which isof critical importance for success of nanomanipulation.To precisely register the tip-object relative position, Mok-

aberi and Requicha proposed a Kalman filter estimation ap-proach to drift compensation [16], which measures the drift bylocally scanning a feature or scanning a local area at certain timeinterval. By setting a maximum covariance value, drift measure-ment is scheduled to reset the origin of the AFM coordinateswhen the covariance exceeds the setup value. Similarly, Yangand et al. proposed a neural network predictor for drift com-pensation by measuring the drift through phase correlation ofsubsequence of images [22]. The Kalman filter estimation andneural network predictor are effective to estimate the drift in asmall time interval. The drift measurement in these approach,however, requires a full scan of the image to reset the origin.Such a full scan of the image is a time-consuming process, andas a result significantly reduces the efficiency of AFM-basednanomanipulation. Using feature tracking, Tranvouez et al. de-veloped an active drift compensation technique that is able todrive the tip to the exact location of the nano-object under ma-nipulation, thus to actively compensate the thermal drift for thespecific nano-object under manipulation [23]. Using a Bayesianfiltering approach, Krohs et al. are able to tack the drift withouta need to scan a feature, thus to compensate the thermal drift inreal-time during manipulation [24]. All these approaches, how-ever, do not tackle the drift distortion in the original AFM image,which could significantly reduce the compensation accuracy.An online correction method has been developed to compensatethe thermal drift within the image and at the same time register

the tip-object relative position [25]. It is achieved by tracking asingle feature thus to compensate the thermal drift during imageacquiring through feed-forward estimation. Unfortunately, suchmethod assumes a constant drift velocity during image acquisi-tion, which is hardly true in reality.In this paper, we propose a strategic local scan approach to

measure and compensate the drift. With this strategic local scanapproach, not only the drift distortion in original AFM imagecan be removed but also the drift during nanomanipulation canbe compensated. A distortion-free image can provide moreaccurate position for each object. With compensated drift inreal-time, the throughput of AFM-based nanomanipulationcan be further improved. The proposed drift compensationapproach make the full automation in AFM-based nanomanip-ulation move a step forward by removing the hurdle caused bythermal drift.We first presented this idea in a conference paper [26] and

the further development in another conference paper [27]. Inthis paper, we will present the details and the complete resultsalong with verification from experimental results. The rest ofthis paper is organized as follows. The augmented reality andlocal scan mechanism are briefly introduced in Section II. Thethermal drift problem and the algorithm for cleaning drift con-tamination are presented in Section III. The drift compensa-tion during manipulation is described in Section IV. Finally,Section V concludes this paper.

II. AUGMENTED REALITY SYSTEM AND LOCAL SCAN

Several attributes of the AFM-based nanomanipulation re-strict its throughput. For example, there is only one tip that canbe controlled by most commercial AFM systems. Once the tip isused for manipulation, its imaging function has to stop. There-fore, visual feedback from AFM images during manipulation isnot possible. Actually, the commercial AFM scanning speed isso slow that it cannot provide the real-time feedback even duringimaging. Optical visual feedback is also not possible becausethe size of nano-objects is beyond the diffraction limit. The lackof real-time visual feedback makes the AFM-based nanomanip-ulation inefficient. In the most recently available AFM-basednanomanipulation systems, the manipulation path is planned ei-ther manually by a haptic device [3], [9] or through a softwareinterface such as in Requicha’s system [10] and the Nanomansystem, a product of Veeco Inc. (now Bruker). Because there isno visual feedback information to update the manipulation re-sults in real-time, a new image scanning is required after eachbasic operation to check the result in order to plan next op-eration. This imaging-planning-manipulating-reimaging cycleis very time-consuming. Therefore, most of nanomanipulationtasks have to be performed by skilled scientists and requirelong time to be accomplished. Thus, it is desirable to have anAFM-based nanomanipulation system that can offer a real-timefeedback. With the real-time visual feedback, nanoscale objectscan be manipulated continuously and the manipulation resultswill also be less dependent on the operators.To tackle the problem of lacking real-time visual feed-

back during manipulation in AFM-based nanorobotics, anaugmented reality system has been developed by the au-thors and their co-workers. The augmented reality system for

LI et al.: DRIFT COMPENSATION IN AFM-BASED NANOMANIPULATION BY STRATEGIC LOCAL SCAN 757

Fig. 1. The configuration of the AFM-based nano-robotic system that can per-form nanomanipulation with real-time visual feedback.

AFM-based nanomanipulation can update the manipulationresults in real-time by the prediction from position controland compensation [12], motion behavior models of nano-ob-jects, interaction among tip, surface, and objects [11], and byreal-time corrections from local scan [13], [14]. The augmentedreality system was first implemented on a Veeco BioscopeAFM system (now Bruker) and now is further implementedon an Agilent ILM 5500 AFM system, as shown in Fig. 1.The augmented reality system consists of three subsystems:AFM system, real-time control module, and the augmentedreality interface. The AFM system includes the AFM head,AFM PC, and AFM controller. The AFM PC is responsible forrunning main control program and providing an interface forthe imaging. Peripheral devices of the AFM system also includean Olympus inverted optical microscope (Olympus IX71) frombottom and a camera from top, which help the operator to locatethe tip, point the laser beam to the cantilever, and search for theinteresting areas on substrates. The real-time control modulehas a data acquisition card (PCI-DAS4020/12, MeasurementComputing Company). It is responsible for running the localscan program to locate the actual position of the nanoscale ob-ject and update it to the Main PC via Ethernet. The augmentedreality interface consists of a haptic device (Phantom fromSensable Company) and a nanomanipulation program runningon the main PC. The augmented reality interface provides bothreal-time visual and force feedback for the operator to viewreal-time AFM image at video frame rate and feel the forcefeedback in real-time during nanomanipulation. The hapticdevice serves as both a force feedback device and a joystick,through which the operator can input the position command.The real-time visual display is a dynamic AFM image of theoperating environment which is locally updated based on thepreacquired environment information (initial AFM images),tip-object interaction model, nano-objects’ behavior models,real-time force information, and local scanning information.As the shape of each nanoscale object can be easily identi-

fied from an initial AFM image, the image of the object that ismanipulated can be updated as long as its center and orientationcan be identified. The local scan function in the augmented re-ality system can determine the new position of a nano-object bysimply performing two or three line-scans instead of scanningan entire area line by line to recover every details of that loca-tion. For example, a particle’s center can be determined by two

Fig. 2. Local scan of nanoparticles. First, the nanoparticle is scanned usingLine , which passes through point , the displayed center of the particlein the original AFM image. If the particle is not found, then the scanning linemoves up and down alternatively by a distance of until the particle isfound. Once the particle is found, the scanning line forms two intersection pointswith the boundary of the particle, and . Additional vertical line, , whichgoes through the midpoint between and and perpendicular to the previousscanning line, is scanned to find the actual center of the particle. The verticalscanning line has two intersection points with the boundary of the particle,and . The actual center of the nanoparticle, a, is located at the midpointbetween and . The local scanning range (the length of the scanning line,) is determined by the maximum random drift such that , where

is the maximum random drift distance [26].

Fig. 3. (a) AFM image of nanoparticles beforemanipulation (the particle insidethe circle will be moved. (b) The real-time visual display during manipulation(the particle inside the circle is supposed to be moved into the triangle box,however, it is found inside the square by local-scan-after-operation). (c) TheAFM image after manipulation. The scanning range of the image is

[27].

scanning lines across the particle. The location of a nanopar-ticle can be approximately represented by its center and radius.The radius of each particle, , can be identified from the cap-tured AFM image. The actual location of the nanoparticle canbe recovered with minimum of two scanning lines (Line and), as shown in Fig. 2 considering the orientation of the par-

ticle is not important in term of nanomanipulation. Similarly, ananorod’s position can be determined by three scanning lines,two transversal lines and one longitudinal line.With the local scan strategy, the real-time changes of manipu-

lation results can be locally updated in the visual feedback inter-face through an interleaving local scan mechanism. The inter-leaving local scan is implemented by switching local scan modeand manipulation mode back-and-forth at a frequency of 16 Hz.With this fast switching frequency, a truly real-time visual feed-back can be achieved [28]. The local scan also becomes trans-parent to the operator at such high switching frequency (the op-erator cannot feel the local scan is running in the background).Based on the local information, the actual location of a

nano-object under manipulation can be updated in real-time inthe augmented reality interface during manipulation. Therefore,nanomanipulation performed based on the actual position ismuch more reliable and has fewer failures. The experimentalresults of local scan in manipulation of nanoparticles withlocal-scan-after-operation are shown in Figs. 3 and 4. As


Fig. 4. (a) The local-scan-after-operation pattern for a particle generated ac-cording to the local scan strategy. (b) The local-scan-after-operation results ofa nanoparticle.

shown in Fig. 3, a particle is pushed from one location toanother location. A scanning pattern is generated, as shown inFig. 4(a). The local scan results are shown in Fig. 4(b). From thelocal scan information, the actual position of the nanoparticlecan be easily recovered by the particle center defined by thesecond scan line. Similarly, a local scan mechanism can also beapplied to nanowires with minimum of three lines to identifythe center and orientation [14].We have shown that the local scan during manipulation can

provide real-time visual feedback. However, the local scan mayaffect the throughput of the manipulation because it takes longertime to find the actual location of the object when anAFM imagehas large drift contamination, because more scanning lines areneeded in order to search for the actual location of the nano-object. Therefore, it would be favorable to plan themanipulationbased on a drift-free image to minimize the load of local scanduring AFM image manipulation, as a result, to improve theefficiency of manipulation but maintain the same reliability.In this paper, we will mainly focus on the drift compensa-

tion by local scan before manipulation and during manipula-tion. The purposes of the local scan in this study are to ob-tain a distortion-free AFM image before starting the manipula-tion and compensate the drift during manipulation in real-time.Starting from a distortion-free image and working on drift com-pensated images during nanomanipulation, the operation time ofthe local scan during manipulation can be significantly reduced,and therefore, the overall manipulation time can be reduced.

Fig. 5. Image sequences of carbon nanorod samples taken at 256 s intervals.The scanning area is 500 nm 500 nm [26].

III. THERMAL DRIFT

In ambient conditions where the environmental temperatureis not strictly controlled, the position of AFM tip relative tosample stage drifts with time, even though the voltage forcontrolling the tip position is held constant. In other words,successive AFM scans of a sample will get translated versionsof the sample surface even none of the scanning parametersis changed, as shown in Fig. 5. The thermal drift inducedtranslation is indeed nonlinear because of the time-variantnature of thermal drift. Therefore, the shape of the objects inthe image is often distorted especially when the object haslarge or long shape such as nanowires and nanorods. Thedrift velocities on the – plane are reported to vary from0.01–0.1 nm/s [16]. In our experiments, the drift in the anddirections around 0.08 nm/s are frequently observed. After

about ten minutes scanning, the drift along direction can beas much as 100 nm, which is comparable to or larger than thediameter of the nanoparticles that are normally manipulated.Thus, AFM-based nanomanipulation operations are fallibleand inefficient because objects can be easily missed due tothe position mismatch between the displayed position and theactual location.There is also drift in the direction, however, it is very dif-

ficult to precisely measure the drift in the direction becausethe topographic data acquired by an AFM are relative height in-formation of a series of discrete points on the sample surface.Fortunately, the drift along direction is usually relatively small(as small as 0.005 nm/s [22] and has little impact on nanomanip-ulation; moreover, it can be well compensated by the feedbacksystem. Thus, it is normally sufficient to deal with drift in theand directions under the existence of drift in direction, so

the nanomanipulation could be performed as if the drift does notexist in direction.


Previous experimental results have shown that the drift alongand directions is translation and little rotation is involved.

The translation over a larger area in the image is nonlinear butcould be considered as linear within a small area that is ac-quired in a short period. Moreover, the coupling between thedrifts along the two directions can be negligible [16]. When ig-noring the drift along direction, the height data between twosuccessive scans in a same sample area can be expressed as [16]

(1)

where and denote drift value in the and direc-tion, respectively, between time instants and . Onceand are known, a drift-corrected image can be simply cal-culated by making coordinates translation for all pixels of theAFM image by following equation:

(2)

Traditionally, and are calculated by comparing twosequential AFM images from full scan [16], [22]. Apparently,considerable time is required in order to acquire a full-scanAFM image. If our local scan can be used to measure the drift,the time required for the measurement will be significantly re-duced.

IV. CORRECTING DISTORTION OF AFM IMAGE BYLOCAL SCAN

All drift compensation algorithms for manipulation so far[16], [23], [24] are assuming that the drift value is unchangedduring imaging and hence the drift is computed by assumingthe same drift value for the entire image. In reality, however,drift happens during the course of acquiring the images. There-fore, different part of the image undergoes different distortionconsidering the time to acquire a high-resolution AFM image isabout a fewminutes andmay be longer. Starting from a distortedimage, more scanning lines are often required during local scanusing our augmented reality system, thus significantly reducethe efficiency of AFM-based nanomanipulation. Therefore, inorder to obtain high-throughput of AFM-based nanomanipula-tion, it is very crucial to find effective ways to minimize or elim-inate the distortion in the image before performing the manipu-lation. This image distortion caused by drift can be effectivelycorrected by cross correlation of subsequent images [20]–[22].The correction after image acquisition, however, is unable toregister the relative position between the tip and the nano-ob-jects in real-time, which is required in manipulation. In this sec-tion, we present an innovative way to correct the distortion inthe initial image by employing a strategic local scan mechanismtomeasure the local distortion that is caused by the random drift.Such approach is able to compensate the thermal drift within theimage and at the same time register the tip-object relative posi-tion in real-time.To find out how the drift distorts the initial image, we need to

examine how the image is acquired. Considering the raster scannature of AFM when acquiring an AFM image, the whole AFMimage can be divided into parts along direction, as shownin Fig. 6. The thermal drift can be considered as the same foreach pixel inside the same part in the image, which is reasonable

Fig. 6. Scheme of local scan for a whole image: a local scan is performed inevery region when there are particles located [26].

because the image inside that part is captured within a very shorttime. If we can measure the drift locally in each part, we are ableto compensate the drift in each part thus to correct the distortioncaused by drift in the initial image. The local scan introduced inSection II is a practicable way to measure the local drift in eachpart by performing a local scan on an object in every part.In our augmented reality system, all the objects in the initial

AFM image are identified before manipulation [12]. The objectsfor nanomanipulation are usually nanoparticles and nanorods ornanowires. They usually exhibit a symmetric shape, which has acenter. This information is saved as reference during manipula-tion. The center of these objects can be used as the landmark foraccurate measurement of drift. For nanoparticles and nanorods,our local scan algorithm can automatically identify their cen-ters as we discussed in Section II. If a local scan is performedon an object in every part, then the drift in every part can becalculated by comparing the center location of the object in theinitial image and the one obtained from local scan.Nano-objects are usually randomly distributed on the sub-

strate before manipulation. It is possible that there is no nano-objects in some parts while there is more than one nano-objectsin other parts. For the part without nano-objects, the thermaldrift has no effect on the manipulation, thus the drift value canbe set equal to that of the nearest neighbor part with nano-ob-jects. For those parts with more than one nano-object, the oneclosest to the center of the part will be chosen to perform thelocal scan to calculate the drift values and the drift values forother particles are set the same as the one chosen for local scan.With this strategy, the drift values along and direction inevery part, and , canbe calculated. According to (2), for any part , a co-ordinate translation for all pixels in this part can be performedusing the following equation:

(3)


The compensating algorithm is explained in details here. AnAFM image with pixels is represented by an datamatrix . Each element in the data matrix represents a corre-sponding pixel in the image. Let be the element in th rowand th column of matrix. The data matrix is divided intosubmatrixes with dimension of , each submatrix rep-resents an image stripe as in Fig. 6. Two data matrixes andare introduced, where is the data matrix after -drift com-

pensation and is the data matrix after -drift compensation.Similarly, and are introduced to represent the element inth row and th column of matrix and . The drift valuesalong and direction in any submatrix can becalculated as and by the local scan performed in thecorresponding image stripe. Here, and should be in-teger if we calculate the drift value in the unit of pixel. The driftcompensation for direction is performed first. For submatrix, the drift along direction is pixel. Every element in sub-matrix should be shifted by column to compensate thedrift. Namely, let

(4)

To make sure that , are within the range submatrix ,restriction , , and should be ap-plied. If , the restriction for becomes ;If , the restriction for becomes . For

, and , there is no corre-sponding elements in the original matrix for , here we de-fine a constant

(5)

and set . Similarly, for , andthere is no corresponding elements in the original matrix

for and is set to equal to .The drift compensation along direction is similar to the

above procedure. For submatrix , the drift along directionis pixels. Every element in submatrix should be shiftedby row to compensate the drift. Namely, let

(6)

To make sure that , are within the range of submatrix, restriction , , and should beapplied. If , the restriction for becomes ;If , the restriction for becomes For

, and , there is no corre-sponding elements for in the submatrix of . Consideringthat the last rows in sub-matrix are adjacent to the first rowsin submatrix , here we set

(7)

(8)...

(9)

The above equations are valid if . For , thereis no adjacent rows for last rows in submatrix , here we setevery element in row , equal toconstant .Similarly, for , and , if ,

we set

(10)

(11)...

(12)

if , there is no adjacent rows for first rows in submatrix 1,we set every element in row , equal to constant .After performing the coordinate translation, the drift effect

can be compensated and a distortion free image can be obtained.Increasing , the number of parts, may lead to improved accu-racy in estimation of the drift value. However, it is not alwaystrue since the AFM undergoes drift as well when performing thelocal scan. The operation time of local scan is around 2 secondsfor equal to 16 in our study. It is negligible while comparedwith the time of acquiring an AFM image, which is around 300seconds in our experiments. Thus, the drift during local scan canbe ignored for relatively small . As the increase, the timefor local scan will increases drastically and is no longer negli-gible, leading to inaccurate estimation of drift value. There is atradeoff between and the accuracy, so N should be properlychosen to achieve optimal accuracy.A simulation of drift compensation process is performed

to verify the feasibility of the compensation algorithm. Adrift-contaminated AFM image with 256 256 pixels is shownin Fig. 7(a) that is obtained from bottom-to-up scanning (thebottom portion of the image is obtained earlier). To compensatethe drift, this image is divided into 16 stripes along direction(here, the coordinate system is the same as in Fig. 6). Assumingthe drift value in stripes ispixels along direction. The drift value along direction is rel-atively small, thus we assume for stripes 1 to 4, there is no driftin direction, and for stripes 5 to 8, the drift along directionis , for stripes 9 to 12, the drift along direction is

, and for stripes 13 to 16, the drift along direction is. The drift compensation process is simulated using

the algorithm stated above. The simulation compensation resultis shown in Fig. 7(b) that represents the actual current positionof the nanoparticles. Comparing these two images beforecompensation [Fig. 7(a)] and after compensation [Fig. 7(b)],we can see that the assumed drift of each nanoparticle has beencompensated. For example, the most recent acquired locationof Particle A has no correction because no drift is assumed;the position of Particles B has small correction because smalldrift has been assumed; the position of Particle C has largecorrection because larger drift is assumed in this region.We use simulation here because the real drift contamination

(a few nanometers) is not easy to be shown in an AFM image. Inthe simulation, we can exaggerate the drift value to easily showthe compensating effect. In real operation, the compensation can


Fig. 7. (a) An AFM image of Au nanoparticles before drift compensation withA, B, and C indicating the positions of three groups of particles before driftcompensation. (b) The image after drift compensation with A’, B’, and C’ in-dicating the positions of the three groups of particles after drift compensation.The image area is 2.5 um 2.5 um, the diameter of the Au particle is around40 nm.

be down exactly the sameway as in the simulation. After the dis-tortion of image has been corrected by the drift compensation,the nanomanipulation based on distortion-free AFM image willbe more accurate and efficient.

V. DRIFT COMPENSATION DURING NANOMANIPULATION

The drift during manipulation also reduces the throughputof AFM-based manipulation because more scanning linesare required for local scan during manipulation. Currently,several methods are available to compensate for the thermaldrift, for example, the Kalman filter estimation proposed byMokaberi and Requicha [16], the neural network predictorproposed by Yang et al. [22], the feature tracking proposedby Tranvouez et al. [23], and the Bayesian filtering approachproposed by Krohs et al. [24]. These methods have been provenvery effective. In this section, we show that the local can is ableto measure the thermal drift in real-time thus to compensate thedrift during manipulation.Using the local scan strategy in the previous section, the drift

value in the and directions and can be quicklymeasured during manipulation (within ten milliseconds) by per-forming local scan on a fixed feature or an object that does notmove in previous manipulation cycle. The augmented realityimage can be quickly updated based on the drift measurementaccording to (2) because the original image is distortion-freeand the real position of the object under manipulation has beenupdated by the local scan during manipulation. For objects orfeatures with irregular shape, a center may not be well defined.If there is no object with regular shape (bead or rod) available,smaller objects (or any small fixed features in the image) withirregular shape will be chosen. As long as these irregular ob-jects (or features) are small enough, the accuracy of drift mea-surement is not a problem. Fig. 8 shows the local scan for driftmeasurement with respect to time, where Fig. 8(a) is the ini-tial real AFM images; Fig. 8(b) is the drift-compensated imagedisplayed in the augmented reality interface after 15 min. Fromthese two images, we can clearly see a significant drift occurredand being compensated through local scan in the augmented re-ality display interface. For example, the particle in the upperright corner has been drifted 180 nm up [880 nm from the topedge in Fig. 8(a) and 620 nm from the top edge in Fig. 8(b)]and also drifted 100 nm right [400 nm from the left edge in

Fig. 8. (a) An initial AFM image of latex nanoparticles. (b) The augmentedreality interface display after 15 min with drift compensation. (c) A new AFMimage acquired after 15 min (this is only for comparison purpose, it is not nec-essary to acquire a new AFM image during manipulation). The image area is 5um 5 um, the diameter of the latex particle is around 150 nm.

Fig. 8(a) and 300 nm from the left edge in Fig. 8(b)]. Comparedwith the drift compensation within the image (Fig. 7), at thistime, the drift compensation for the whole image is the same.To prove the effectiveness of the drift compensation throughlocal scan, we have performed a quick scan (assuming the driftduring scanning is small) to obtain the actual position of theseparticles at that moment just after drift compensation. Com-pared with the real AFM images acquired after the quick scan[Fig. 8(c)], the locations of objects in the images updated in theaugmented reality by drift compensation from local scan mea-surement [Fig. 8(b)] are exactly the same as those in AFM imageat that time.Combing our drift measurement by local scan with the drift

predictor either by Kalman filter [16] or by neural network [22]the drift can be compensated in real-time even when the manip-ulation is being carried out.

VI. CONCLUSION

In this study, a local scan strategy has been proposed to cleanthe drift contamination in the initial AFM images and measurethe real-time drift during manipulation. By dividing the AFMimage into many parts along direction, the drift value at thatpart can be measured by the local scan in each part. After com-pensating the drift in each part using our proposed algorithm, adrift-contamination-free image can be recovered simply from acoordinate transformation equation. The AFM-based nanoma-nipulation will become more reliable and efficient based on aninitial drift contamination free image. The drift value duringmanipulation can also be measured by the local scan strategyin real-time. Combining with a drift predictor, a drift-freeimage display can be achieved in real time during manipu-lation without requesting a complete new AFM image. Suchdrift-resistant augmented reality system builds a foundation forour future automated nanomanipulation with extremely highthroughput. Such high throughput nanomanipulation systemwill be the future practicable tool for assembling nanodevicesby manipulating nanoscale objects such as nanoparticles andnanowires.

REFERENCES[1] G. Binnig, C. F. Quate, and C. Gerber, “Atomic force microscope,”

Phys. Rev. Lett., vol. 56, no. 9, pp. 930–933, Mar. 1986.[2] R. Luthi, E. Meyer, H. Haefke, L. Howald, W. Gutmannsbauer, and H.

J. Guntherodt, “Sled-type motion on the nanometer scale: Determina-tion of dissipation and cohesive energies of C60,” Science, vol. 266,no. 5193, pp. 1979–1981, Dec. 1994.


[3] M. Guthold, M. R. Falvo, W. G. Matthews, S. Paulson, S. Washburn,D. A. Erie, R. Superfine, F. P. Brooks, and R. M. Taylor, “Controlledmanipulation of molecular samples with the nanomanipulator,” IEEE/ASME Trans. Mechatronics, vol. 5, no. 2, pp. 189–198, Jun. 2000.

[4] T. Junno, K. Deppert, L. Montelius, and L. Samuelson, “Controlledmanipulation of nanoparticles with an atomic force microscope,” Appl.Phy. Lett., vol. 66, no. 26, pp. 3627–3629, 1995.

[5] D. M. Schaefer, R. Reifenberger, A. Patil, and R. P. Andres, “Fab-rication of two-dimensional arrays of nanometer-size clusters withthe atomic force microscope,” Appl. Phy. Lett., vol. 66, no. 8, pp.1012–1014, 1995.

[6] M. Sitti and H. Hashimoto, “Controlled pushing of nanoparticles: Mod-eling and experiments,” IEEE/ASME Trans. Mechatronics, vol. 5, no.2, pp. 199–211, Jun. 2000.

[7] R. Resch, C. Baur, A. Bugacov, B. E. Koel, A. Madhukar, A. A. G. Re-quicha, and P. Will, “Building and manipulating 3-D and linked 2-Dstructures of nanoparticles using scanning force microscopy,” Lang-muir, vol. 14, no. 23, pp. 6613–6616, 1998.

[8] L. T. Hansen, K. A. Hle, H. S. A. Rensen, J. Bohr, and P. E. Lindelof,“A technique for positioning nanoparticles using an atomic force mi-croscope,” Nanotechnology, no. 4, pp. 337–337, 1998.

[9] M. Sitti and H. Hashimoto, “Tele-nanorobotics using atomic force mi-croscope,” in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 1998, vol.3, pp. 1739–1746.

[10] A. A. G. Requicha, C. Baur, A. Bugacov, B. C. Gazen, B. Koel, A.Madhukar, T. R. Ramachandran, R. Resch, and P. Will, in NanoroboticAssembly of Two-Dimensional Structures, vol. 4, pp. 3368–3374.

[11] G. Li, N. Xi, H. C. C. Pomeroy, and M. Prokos, ““Videolized”atomic force microscopy for interactive nanomanipulation andnanoassembly,” IEEE Trans. Nanotechnol., vol. 4, no. 5, pp. 605–614,Sep. 2005.

[12] G. Li, N. Xi, M. Yu, and W.-K. Fung, “Development of augmentedreality system for AFM-based nanomanipulation,” IEEE/ASME Trans.Mechatronics, vol. 9, no. 2, pp. 358–365, Jun. 2004.

[13] H. Chen, G. Li, and N. Xi, “CAD guided automated nanoassemblyusing automic force microscopy-based nanorobotics,” IEEE Trans.Autom. Sci. Eng., vol. 3, no. 3, pp. 208–217, Jul. 2006.

[14] L. Liu, Y. Luo, N. Xi, Y.Wang, J. Zhang, and G. Li, “Sensor referencedreal-time videolization of atomic force microscopy for nanomanipula-tions,” IEEE/ASME Trans. Mechatronics, vol. 13, no. 1, pp. 76–85,Feb. 2008.

[15] J. Zhang, N. Xi, H. Chen, K. Lai, L. Guangyong, and U. C. Wejinya,“Design, manufacturing, and testing of single-carbon-nanotube-basedinfrared sensors,” IEEE Trans. Nanotechnol., vol. 8, no. 2, pp.245–251, Mar. 2009.

[16] B. Mokaberi and A. A. G. Requicha, “Drift compensation for auto-matic nanomanipulation with scanning probe microscopes,” IEEETrans. Autom. Sci. Eng., vol. 3, no. 3, pp. 199–207, Jul. 2006.

[17] R. Staub, D. Alliata, and C. Nicolini, “Drift elimination in the calibra-tion of scanning probe microscopes,” Rev. Sci. Instrum., vol. 66, no. 3,pp. 2513–2516, 1995.

[18] J. T. Woodward and D. K. Schwartz, “Removing drift from scanningprobe microscope images of periodic samples,” J. Vac. Sci. Technol. B,vol. 16, no. 1, pp. 51–53, 1998.

[19] V. Y. Yurov and A. N. Klimov, “Scanning tunneling microscope cali-bration and reconstruction of real image: Drift and slope elimination,”Rev. Sci. Instrum., vol. 65, no. 5, pp. 1551–1557, 1994.

[20] B. A. Mantooth, Z. J. Donhauser, K. F. Kelly, and P. S. Weiss, “Cross-correlation image tracking for drift correction and adsorbate analysis,”Rev. Sci. Instrum., vol. 73, no. 2, pp. 313–317, 2002.

[21] R. V. Lapshin, “Automatic drift elimination in probe microscope im-ages based on techniques of counter-scanning and topography featurerecognition,” Meas. Sci. Technol., vol. 18, no. 3, pp. 907–907, 2007.

[22] Q. Yang, S. Jagannathan, and E. W. Bohannan, “Automatic driftcompensation using phase correlation method for nanomanipulation,”IEEE Trans. Nanotechnol., vol. 7, no. 2, pp. 209–216, Mar. 2008.

[23] E. Tranvouez, E. Boer-Duchemin, G. Comtet, and G. Dujardin,“Active drift compensation applied to nanorod manipulation withan atomic force microscope,” Rev. Sci. Instrum., vol. 78, no. 11, pp.115103–115105, 2007.

[24] F. Krohs, C. Onal, M. Sitti, and S. Fatikow, “Towards automatednanoassembly with the atomic force microscope: A versatile driftcompensation procedure,” J. Dyn. Syst., Meas. Control, vol. 131, no.6, pp. 061106–061108, 2009.

[25] P. Rahe, J. Schutte, W. Schniederberend, M. Reichling, M. Abe, Y.Sugimoto, and A. Kuhnle, “Flexible drift-compensation system for pre-cise 3D force mapping in severe drift environments,” Rev. Sci. In-strum., vol. 82, no. 6, pp. 063704–063704, 2011.

[26] Y. Wang, G. Li, N. Xi, and L. Liu, “Compensation of drift contami-nation in AFM image by local scan,” in Proc. IEEE Int. Conf. Robot.Biomimetics, ROBIO’08, 2009, pp. 487–492.

[27] Y. Wang, G. Li, and L. Liu, “Local scan for compensation of drift con-tamination in AFM based nanomanipulation,” in Proc. IEEE/RJS Int.Conf. Intell. Robot. Syst., IROS’09, 2009, pp. 1345–1350.

[28] G. Li, L. Liu, and N. Xi, “Augmented reality enhanced nanomanipula-tion by atomic force microscopy with local scan,” in Proc. ASME Int.Conf. Manuf. Sci. Eng., Atlanta, GA, Oct. 2007, pp. 643–652.

Guangyong Li (M’07–SM’01) received the B.S.degree in mechanical engineering from the NanjingUniversity of Aeronautics and Astronautics, Nan-jing, China, in 1992, the M.S. degree in aerospaceengineering from the Beijing Institute of ControlEngineering, China Academy of Space Technology,Beijing, China, in 1999, and the Ph.D. degree inelectrical engineering from Michigan State Univer-sity, East Lansing, in 2006.He is currently an Assistant Professor with the

Department of Electrical and Computer Engineering,University of Pittsburgh, Pittsburgh, PA. His current research interests in-clude micro/nanorobotic systems; fabrication of MEMS/NEMS, nanodevices,biosensors; control theory and applications, real-time system design, imple-mentation, and integration; and neural-network control.Dr. Li and his coauthors received the 2006 IEEE TRANSACTIONS ON

AUTOMATION SCIENCE AND ENGINEERING Best Paper Award.

Yucai Wang (SM’08) received the B.S. degreein electrical engineering from Fudan University,Shanghai, China, in 2004 and the M.S. degreein electrical engineering from the University ofPittsburgh, Pittsburgh, PA, in 2010. He is currentlyworking towards the Ph.D. degree in electricalengineering at McGill University, Montreal, QC,Canada.His current research interests include micro/nano

systems, lab-on-chip, and CMOS implantable sen-sors.

Lianqing Liu (M’09–SM’08) received the B.S. de-gree in industry automation from Zhengzhou Univer-sity, Zhengzhou, China, in 2002 and the Ph.D. degreefrom the Shenyang Institute of Automation, ChineseAcademy of Sciences, Shenyang, China, in 2009.He is currently a Professor at the Shenyang Insti-

tute of Automation, Chinese Academy of Sciences.His current research interests includemicro/nano sys-tems, haptic and visual interfaces, nanodevice fabri-cation, and biosensors.

Documents

Drift Compensation in AFM-Based Nanomanipulation by Strategic Local Scan