Visual Servoing Experiments of Underwater Vehicle

under Air Bubble Disturbances

Myo Myint, Mamoru MINAMI, Kenta YONEMORI, Yukihiro SAKA and Akira YANOU

1. Introduction

The role of decontamination work in radioac-tive contamination area has been highlighted inFukushima prefecture which has been afflicted bydisasters such as earthquake, tsunami and nuclearpower accident. Ministry of Agriculture, Forestry andFisheries researched concentration of radioactive sub-stances in mud and soil deposited at the bottom ofdam and reservoir in Fukushima [1]. Underwater ve-hicle has become essential for decontamination underwater, seabed resource exploration and so on . Basedon not only above motivations but also applications incountless domain in underwater tasks for society, thefootprint of research on underwater vehicle in our lab-oratory, therefore, has been explored especially withthe cooperation of KOWA Corporation.

A number of researches on underwater vehicle us-ing visual servoing have being conducted in worldwide[2]. Most of them using different image pro-cessing techniques[3][4], are very limited when envi-ronment under water poses difficulties against im-age recognition such as disturbances by marine snowto image. To overcome these issues, visual servoingsystem utilizing image recognition using Model-basedMatching method and Genetic Algorithm (GA) hasbeen proposed[7]. The proposed system (underwatervehicle) can be regulated at desired position and posethrough visual servoing. In order to assess the effec-tiveness of the proposed system against noises in cam-era images, this report presents how the dual-eye re-active image recognition system be robust against airbubble disturbances and how the visual servoing sys-tem maintains the servoing performance even throughthe bubbles disturb the image feedback.

2. Visual Servoing System for ROV

The proposed system does not involve IMU (In-ertial Measurement Unit) using certain sensors toestimate the position and orientation information.Instead, Based on known information about tar-get(size,shape), the target’s position and orientationinformation are calculated through recognizing pro-cess using model-based matching method and GeneticAlgorithm (GA). Model-based recognition approachis applied because of its performance in terms of lesssensitive on camera calibration, comparing to othermethods like feature based recognition in which the

Fig.1 Block Diagram of Visual Servoing System

pose of the target object should be determined bya set of image points, complex searching the corre-sponding points and time consuming[5].

Fig. 1 shows the block diagram of the proposedcontrol system. In this control system, the positionand posture (xd[mm], yd[mm],zd[mm], ε2d[deg]) be-tween the target and ROV are predefined so that therobot will regulate through visual servoing. Rotationsaround x and y-axes( ε1d[deg], ε3d[deg] ) are neglectedbecause of their less effectiveness to ROV’s motion inthis experiment. 3D position and posture of the tar-get object are recognized by model-based matchingmethod and GA. Finally based on the error betweentarget value and recognized value, the control signalis generated using Proportional controller to keep thetarget position and posture. Overall processing isdone in PC which is contacted to ROV by flexiblecable. The underwater experiment is conducted in apool filled with water.

2.1 Remotely Operated Vehicle (ROV)

Remotely Operated Vehicle (ROV) shown in Fig.2,manufactured by KOWA Corporation, is used for theproposed underwater experiment. Two fixed cameras(binocular camera) and four thrusters (traverse, hor-izontal and vertical direction) are installed in ROV.Maximum operating depth in water is 50[m].

Fig.2 Underwater Vehicle

RSJ2015AC2G2-03

第33回日本ロボット学会学術講演会（2015年9月3日～5日）

2.2 Feedback System using 3D Model-based

Matching and Genetic Algorithm (GA)Position and orientation of target object is esti-

mated using model-based matching and GA basedon known 6DoF (the position and orientation)of 3Dmodel of the target projected to 2D image. Targetobject is consisted of three spheres (40[mm] in diame-ter) whose colors are red, green and blue. The targetobject is fixed to the constant position and postureduring the experiment. It is assumed that the targetobject exists in the searching space.The recognition ofvehicle ’s pose through three dimensional marker isexecuted by GA. GA provide faster recognition per-formance to the vision system to which every inputimage is evaluated by model-based fitness function,and the convergence of GA is realized in the sequencesof dynamic images, which is named“ 1- Step GA”.This method has been confirmed in our our previ-ous researches[6][7][8]. The number of evolving gen-erations in this experiment is 9 per 33 [ms] and thenumber of genes is 60.

ROV

800[mm]��

�

��

��

�

��

��

��

400[mm]

GA search space

Target

800[mm]

Fig.3 Searching Space in GA

2.3 Visual Servoing ControllerProportional controller is considered as the main

control to compensate the error between target valueand recognized value. The control voltages of fourthrusters are calculated by the following proportionalcontrol laws.

v1 = kp1(zd − z) + 2.5 (1)

v2 = kp2(ε2d − ε2) + 2.5 (2)

v3 = kp3(yd − y) + 2.5 (3)

v4 =

5 : xd − x < −52.5 : −5 < xd − x < 50 : 5 < xd − x

(4)

where,v1,v3 and v4 are the voltages for thrust of z-axis, yaxis and x-axis direction respectively. v2 meansthe voltage for torque around y-axis. The controlvoltage(v4) for x-axis direction is just on-off control.Based on not only motion equation and thrusters’characteristics but also experimental results, gain co-efficients are tuned to have better performance in reg-ulator process.

3. Experiments and ResultsEven though the error from the relative target posi-

tion and orientation appears constantly and the sys-tem operates the four thrusters simultaneously andconstantly, recognition error changes with the waterpressure due to reaction force during robot movementand reflected wave from the experimental pool sides.Besides, the appearance of bubbles in front of thethree-dimensional marker is to perform as the maindisturbance to recognition image as shown in Fig. 4.

Fig.4 Experiment Layout with Coordinate Frames

According to the experimental result without dis-turbance as shown in Fig.6(left column),the fitnessvalue rises up to more than 1 for the first few secondsof recognition process and maintains with the mini-mum value of 0.8. It is confirmed, therefore, that therecognition accuracy of GA is to be more than 0.5 offitness value to perform visual servoing well.

(a)

(b)

Fig.5 Experiment of visual servoing(a) without airbubble disturbance and (b) with air bubble dis-turbance

Fig.6(right column) shows the regulator perfor-mance with air bubble disturbance to image recogni-tion. Variations in fitness value becomes larger com-paring to the results without air bubble disturbance.Although the fitness value is reduced to about 0.4,it is confirmed that the underwater robot is regulatedin relative target position and posture. Therefore, theproposed system has been verified to have the robustcharacteristics to return to the target position andposture.

RSJ2015AC2G2-03

第33回日本ロボット学会学術講演会（2015年9月3日～5日）

�

���

���

���

���

�

���

���

� �� �� �� �� ��� ���

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

���

��

���

���

�

��

��

�

��

� �� �� �� �� ��� ���

erro

r in

y-a

xis

dire

ctio

n [m

m]

fitn

ess

erro

r in

x-a

xis

dire

ctio

n [m

m]

erro

r in

z-a

xis

dire

ctio

n [m

m]

erro

r ar

ound

y-ax

is [

deg]

time [s]

time [s]

time [s]

time [s]

time [s]

(b)

�

���

���

���

���

�

���

���

� �� �� �� �� ��� ���

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

erro

r in

y-a

xis

dire

ctio

n [m

m]

fitn

ess

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

erro

r in

x-a

xis

dire

ctio

n [m

m]

����

���

���

���

���

�

��

��

��

��

���

� �� �� �� �� ��� ���

erro

r in

z-a

xis

dire

ctio

n [m

m]

���

��

���

���

�

��

��

�

��

� �� �� �� �� ��� ���

erro

r ar

ound

y-ax

is [

deg]

time [s]

time [s]

time [s]

time [s]

time [s]

(a)

(c)

(d)

(e)

Fig.6 Regulator performance without additional disturbance(left column) and, with additional disturbance(rightcolumn) on image: (a)fitness value, (b)error in x-axis direction, (c)error in y-axis direction, (d)error in z-axisdirection and (e)error around y-axis.

RSJ2015AC2G2-03

第33回日本ロボット学会学術講演会（2015年9月3日～5日）

4. ConclusionIn this paper, visual sevoing system is proposed

that focuses on the regulator performance with dis-turbance especially to recognized image. The systemis evaluated for both with and without air bubblesdisturbance. Results show that recognition processand regulator performance can be maintained usingthe proposed system even through there is air bub-ble disturbance to image. The system can controlROV for the proposed regulator function with goodaccuracy. It can therefore be said that the researchequation related to the robustness of visual servoingagainst disturbance has been answered to some degreeof solution with reasonable experimental results.

Reference

[1] Ministry of Agriculture, Forestry and Fisheriesof Japan: “Recent status and provision of ra-diation contamination in FUKUSHIMA ponds”,http://www.maff.go.jp/j/kanbo/jphp/saigai/pdf/

tyukan.pdf(2013).[2] Feihu Sun, Junzhi Yu and De Xu: “Visual measure-

ment and control for underwater robots: A survey”,Control and Decision Conference (CCDC), pp. 333 -338, 2013.

[3] Foresti G.L., Gentili S. and Zampato M.: “A vision-based system for autonomous underwater vehicle nav-igation”, OCEANS ’98 Conference Proceedings, Vol.1,pp. 195 - 199, 1998.

[4] Balasuriya, B.A.A.P, Takai, M., Lam, W.C., Ura,T. and Kuroda, Y.,: “Vision based autonomous un-derwater vehicle navigation: underwater cable track-ing”, OCEANS ’97. MTS/IEEE Conference Proceed-ings, Vol.2, pp. 1418 - 1424, 1997.

[5] Maeda. Y, Xu. G,: “Smooth Matching of Feature andRecovery of Epipolar Equation by Tabu Search”, IE-ICE, Vol.J83- D-2, No.3, pp.440-448, 1999.

[6] Suzuki. H, Minami. M,: “Visual Servoing to CatchFish Using Global/Local GA Search”, IEEE/ASMETransactions on Mechatronics, Vol.10, Issue 3, pp 352-357, 2005.

[7] Yu F., Minami M., Song W., Zhu J. and Yanou A.:“On-line head pose estimation with binocular hand-eye robot based on evolutionary model-based match-ing”, Journal of Computer and Information Technol-ogy, Vol.2, No.1, pp.43-54, 2012.

[8] Song W. and Minami M.: “ 3-D Visual Servoing UsingFeedforward Evolutionary Recognition”, Journal of theRobot Society of Japan, Vol.28, No.5, pp.591-598 (inJapanese), 2010.

RSJ2015AC2G2-03

第33回日本ロボット学会学術講演会（2015年9月3日～5日）