art%3A10.1007%2Fs10846-010-9514-8

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

1/19

J Intell Robot Syst (2011) 64:7795DOI 10.1007/s10846-010-9514-8

Robot-Assisted Bridge Inspection

Robin R. Murphy Eric Steimle Michael Hall

Michael Lindemuth David Trejo Stefan Hurlebaus

Zenon Medina-Cetina Daryl Slocum

Received: 18 May 2010 / Accepted: 29 November 2010 / Published online: 18 January 2011 Springer Science+Business Media B.V. 2011

Abstract The Center for Robot-Assisted Search and Rescue (CRASAR) de-ployed a customized AEOS man-portable unmanned surface vehicle and twocommercially available underwater vehicles (the autonomous YSI EcoMapper and

This work was supported in part by grants from the Texas Transportation Institute, ONR(N000140610775 Coordinated Operation of Humans, Agents, and Unmanned Vehicles forLittoral Warfare), and NSF (EIA-022440 R4: Robots for Rescue, Research and Response).

R. R. Murphy (B) S. Hurlebaus Z. Medina-CetinaTexas A&M, College Station, TX 77843-3112, USAe-mail: [email protected]

S. Hurlebause-mail: [email protected]

Z. Medina-Cetinae-mail: [email protected]

E. Steimle M. HallAEOS, LLC, St. Petersburg, FL 33711, USA

E. Steimlee-mail: [email protected]

M. Halle-mail: [email protected]

M. LindemuthUniversity of South Florida, St. Petersburg, FL 33701, USAe-mail: [email protected]

D. TrejoOregon State University, Corvallis, OR 97331, USAe-mail: [email protected]

D. SlocumYSI Inc., San Diego, CA 92121, USAe-mail: [email protected]

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

2/19

78 J Intell Robot Syst (2011) 64:7795

the tethered VideoRay) for inspection of the Rollover Pass bridge in the Bolivarpeninsula of Texas in the aftermath of Hurricane Ike. A preliminary domain analysiswith the vehicles identified key tasks in subsurface bridge inspection (mapping ofthe debris field and inspecting the bridge footings for scour), control challenges

(navigation under loss of GPS, underwater obstacle avoidance, and stable positioningin high currents without GPS), possible improvements to human-robot interaction(having additional display units so that mission specialists can view and operate onimagery independently of the operator control unit, incorporating 2-way audio toallow operator and field personnel to communicate while launching or recovering thevehicle, and increased state sensing for reliability), and discussed the cooperative useof surface, underwater, and aerial vehicles. The article posits seven milestones in thedevelopment of a fully functional UMV for bridge inspection: standardize missionpayloads, add health monitoring, improve teleoperation through better human-robotinteraction, add 3D obstacle avoidance, improve station-keeping, handle large datasets, and support cooperative sensing.

Keywords Security and rescue robots Underwater unmanned vehicles

1 Introduction

Unmanned marine vehicles (UMV) are being used and considered for post-disaster

bridge inspection and other activities in littoral regions. While hurricanes are as-sociated with large scale search and rescue activities on land, inspection of coastallittoral structures are also important. Bridges must be inspected, as they are neededfor responders and recovery workers to have access to affected areas, and becausethey influence the general recovery of the area. Seawalls, levees, or dikes maybe compromised and create a secondary disaster such as seen in New Orleans atHurricane Katrina. Channels must be restored and docks repaired as part of theeconomic recovery and restoration of shipping. It the disaster is man-made, eitherterrorism, an accident, or structural failure, forensic structural data is needed tohelp agencies such as the Federal Bureau of Investigation (FBI) and the National

Transportation Safety Board (NTSB). Likewise, bodies may need to be recovered.The term UMV encompasses three types of vehicles. Unmanned surface ve-

hicles (USV), often called autonomous surface vehicles (ASV), are boat or jet-ski variants that work on the water surface or are semi-submersible. Unmannedunderwater vehicles (UUV) generally refer to tether-less platforms which eitherexecute automated paths or communicate with the surface through low baud-rate acoustic modems. The term remotely-operated vehicle (ROV) designates aUUV that is controlled through a tether, enabling real-time control and perceptionnormally prevented by acoustic modems.

This article, an extension of [1], describes the use of all three types of UMVsfor post-disaster bridge inspection in the aftermath of Hurricane Ike. Hurricane Ikewas a Category four storm that struck Galveston, Texas on September 13, 2008. TheRollover Pass bridge on the adjacent Bolivar Peninsula, a major artery to the area,was severely damaged. In December, 2008, the Texas Transportation Institute (TTI)and the Texas Department of Transportation permitted CRASAR to test UMVsfor bridge inspection. In particular, the agencies desired an evaluation of how well

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

3/19

J Intell Robot Syst (2011) 64:7795 79

the Sea-RAI unmanned surface vehicle (shown in Fig. 1) could inspect the bridgefootings for scour (erosion and separation from the bottom) and map the debris fieldaround the bridge (i.e., is there debris likely to be pushed into the substructure anddamage it?).

As described in [2], the inspection of the underwater portion of a bridge, called thesubstructure, is currently done manually with divers who must work in high currents,low visibility, and debris to physically see and touch damage or scour (erosion underthe foot of the bridge). Manual inspection puts the divers at risk and is not efficient,for example due to the tidal currents at the Rollover Pass bridge, divers couldonly work for 15 min at the change of each tide. At the 2007 I-35 bridge collapsein Minneapolis, high currents and cramped, confined spaces were cited as thejustification for bringing in a team of 17 Navy divers [3]. Those cluttered conditionswere so difficult that the FBI was unable to successfully use a unspecified miniatureUMV[4].

Post-disaster inspection is an excellent domain for UMVs, as it scores high onLeibholz criteria for deploying a UMV[5], paraphrased and re-ordered below:

Of fers economic advantages. Post-disaster inspection has two major economicsimpacts. First, it reduces the need for expensive manual inspection and staffing,and second, it speeds economic recovery of an affected region.

Reduces human risk. As seen at Rollover Pass[2] and Minneapolis[3], a UMVwould reduce risk to dive teams.

Requires humanlike abilities. In this case, the maneuverability and perceptual

abilities of a diver are needed to adapt to conditions and reach all areas of thestructure; bridge inspection is not something easily done with a large vessel at astandoff distance.

Is a repetitive or highly boring task requiring continued alertness. Inspection isan intrinsically boring task, making it easy for a manual diver to miss importantdetails, even if not distracted by personal safety concerns. A UMV can provideprovable coverage of the entire area and structure.

Fig. 1 The sea-RAI operatingaround the Rollover Passbridge

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

4/19


While post-disaster bridge inspection by UMVs is clearly desirable, it is moredemanding than routine bridge inspection. Ageev et al. [6] offer perhaps the earliestdescription of the routine, unfavorable conditions around structures, most notablythe difficulties in controlling changes velocity and direction to support object in-

vestigation. However, experiences after Hurricane Wilma [7], as well as the effortreported in this article, add three additional challenges:

Extreme environmental conditions, where tides, surge, flooding, currents, wind,and water/wind disturbances near structures exacerbate control challenges;

Presence of debris and changes in topology, where a priori maps of channelsand structures may be rendered obsolete by damage, debris from upstreamdevastation distributed through out the area of interest, and new channels andshallows created by flooding and wave action, thus necessitating shallow waterUSVs, UUVs with obstacle avoidance, and ROVs with tether management; and

Unfavorable deployment conditions, where boat ramps for launching may bedamaged or unavailable and it may not be safe to stand on the bridge to lower orcontrol devices, suggesting the need for lightweight, easily portable vehicles.

This paper is an extension of [1] and[8], providing a deeper review of the relatedwork on UMVs for littoral inspection, more details of the CRASAR deployment,and contrasts challenges in vehicle control, multi-robot coordination, human-robotinteraction, and sensing with the state of the art. The article is expected to contributeadomain theory for post-disaster bridge inspectionfor use by the larger robotics andresponse communities and findings for the research community on control challenges,human-robot interaction, multi-robot cooperation, and uncertain sensor data. Thedomain theory and findings are expected to be applicable to UMV surveys of damagefrom natural events (e.g., hurricanes and flooding) but also to man-made incidentssuch as the 2007 I-35W bridge collapse in Minneapolis and general operations incluttered littoral regions.

The remainder of the article is organized as follows. Section 2summarizes therelated work in UMVs for disaster response, summarizing the deployments afterHurricane Wilma (2005) and the I-35 bridge collapse in Minneapolis (2007) and therelationship to other littoral applications. Section3describes the deployments, team,

and conditions, the three unmanned marine vehicles used, and the results of theinspection showing no damage, scour, or debris. Section 4 compares the suitabilityand performance of the three UMVs, while Section 5 reports findings in controlchallenges, human-robot interaction, multi-robot cooperation, and uncertain data.The article concludes with a roadmap for improving UMVs for bridge inspection.

2 Related Work

Unmanned marine vehicles have not been widely used for disaster response [ 9],though search and rescue has been cited as a motivation for UMV research in [10].CRASAR is a center at Texas A&M that promotes robots for emergency responseand sends out volunteer teams of scientists and robot manufacturers to disastersto assist and collect data, similar to tornado storm chasers and Doctors WithoutBorders. It has experimented twice with post-disaster uses at the Marco Island bridgeafter Hurricane Wilma in 2005 [7] and the Rollover Pass bridge after Hurricane Ike

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

5/19


in 2008 (this article). The FBI deployed an unspecified UUV and ROV to the 2007I-35 bridge collapse in Minneapolis, Minnesota.

2.1 Post Hurricane Wilma

CRASAR conducted a survey of the Marco Island Yacht Club docks and seawallsand near-by SR951 Marco Island bridge in the wake of Hurricane Wilma with anAEOS man-portable surface vehicle; the reader is directed to [7] for a comprehensivediscussion. The survey was conducted on Oct. 27, 2005, after the Wilmas landfall14 km away in Cape Romano, Florida. Despite being a Category five hurricane,Wilma generated very little damage in the area. The USV used a DIDSON acousticcamera, an imaging sonar that does not require the vessel to be moving, and foundscour under the pilings of the docks and railings from the damaged fishing pier

adjacent to the bridge.The Wilma deployment identified USV localization and control and human-robotinteraction as major research issues. Findings from the Wilma deployment led to theSea-RAI project which refined the AEOS platform, added navigational autonomy,and improved human-robot interfaces [11]. The Wilma effort also explored USV-UAV cooperation [7,12].

The deployment to Rollover Pass differed in that it was conducted on behalf ofthe Texas Department of Transportation and the team included two civil engineers;it was specifically tasked to inspect the substructure for scour, map the upstreamdebris field, and to compare USV, UUV, and ROV suitability; it did not involve

UAVs; and an improved USV and interface was used.

2.2 I-35 Bridge Collapse

The FBI Underwater Search and Evidence Recovery Team (USERT) reportedlysent two UUVs to assist with the recovery and forensic documentation of the Aug. 1,2007, collapse of the I-35 bridge in Minneapolis, Minnesota. The news report [4] didnot specify the models, describing the larger one as having a manipulator (possibly aSeaBotix or similar platform) and a smaller one being the size of shoe box (possibly a

VideoRay). No indication was given as to whether the UUVs were tethered, thoughit is likely. The larger UUV could not be successfully used because it was too bigto maneuver amid the unstable, twisted bridge wreckage and vehicles in the murkywater. No follow up reports on whether the smaller UUV was successful could befound, though the report implied some doubt over the utility of the smaller UUVdue to its smaller thrusters and high currents.

The I-35 bridge collapse deployment confirms the impact of extreme environ-mental conditions the presence of debris and changes in topology on UMVs. Italso suggests that development of micro-UMVs such as [13] may not be sufficientlypowerful to be of use.

2.3 Other Littoral Uses

UMVs have been used in littoral regions for commercial, security, and researchapplications. They have been used by the oil and gas industry for decades [14].Castelin and Bernstein give an overview of military operations in littoral regions

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

6/19


in[15]. Discussion of specific security applications include general littoral mappingand demining [16,17], homeland security for port protection[18], and amphibioustroop deployment [19]. They have also be proposed for chemical plume tracking[20]and inspection of underwater wrecks[21]. UMVs have begun assisting with research

efforts, including mapping the Great Barrier Reef [22], environmental mapping [23],sampling sea-air interfaces[24] and sediments [25]. With the notable exception ofmarina mapping[26,27], these littoral operations do not take place in close proximityto structures or constricted areas and thus provide little insight for post-disasterinspection. Unlike port and littoral security [28], the submerged threats in post-disaster inspection are debris, not other vehicles.

3 Mission

With permission from the Texas Department of Transportation (TxDOT) andsupport from the Texas Transportation Institute (TTI), the CRASAR team was inthe field from Dec. 1719, 2008, approximately three months after Hurricane Ike toinspect the Rollover Pass bridge (Fig.2). The Rollover Pass Bridge is a two-lane withmedian concrete span on Texas Highway 87 connecting a 0.06 km man-made channelbetween the Gulf of Mexico, Galveston Bay, and the intercoastal waterway. The passis subject to intense tides and turbidity. The bridge, which had been heavily damaged,had been inspected with divers and one of the three lanes had been repaired andmajor debris removed to prevent further damage. However, the bridge provided

a good test case for evaluating UMVs for post-disaster inspection because of theextreme environmental conditions (tides, currents), difficult deployment conditions(UMVs had to be hand carried to the water), and there was a possibility thatmanual methods had missed scour or debris. In addition, Rollover Pass was far morecluttered than the Marco Island bridge; the Marco Island bridge has a span of 1.4 kmwhile Rollover Pass spans only 0.06 km.

The deployment had a primary and secondary mission for the agencies as well asa general science mission for the robotics community. The primary mission was toevaluate the utility and performance of the Sea-RAI USV for two tasks: inspectionof the bridge substructure and mapping of the debris field. The metric for success

Fig. 2 Rollover pass bridgeafter Hurricane Ike (fromWikimedia Commons, publicdomain under PD-USGOV)

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

7/19


was the acquisition of comprehensible underwater imagery deemed of use to thestructural community by the civil engineers on the team. he Sea-RAI was able tomeet both objectives as described in Section 3.3. The secondary mission for theagencies to compare the types of UMVs for post-disaster bridge inspection and

those results are reported in Section 4. The general science mission was to makeethnographic observations to determine how a USV might be used in the future andexperimentation with the two UUVs; these findings are reported in the next section.

3.1 Deployments, Team, and Conditions

The team was comprised of roboticists, civil engineers, and responders and three un-manned marine vehicles. The six person team consisted of two roboticists (Murphy,Steimle), both of whom had participated in the Hurricane Wilma deployment, two

civil engineering professors (Hurlebaus, Medina-Cetina), a responder from the TexasEngineering Extension Service (May), which is the state agency for urban search andrescue, and a graduate student (Lindemuth). In addition, YSI sent two experts (Hall,now with AEOS, and Slocum) to use the Ecomapper UUV on Dec. 18. The primaryvehicle was the Sea-RAI unmanned surface vehicle, with a VideoRay tethered ROVand YSI UUV Ecomapper as secondary vehicles for comparision. The team focusedon the USV because as noted in [7], surface vehicles have important advantages overunderwater vehicles: they can be more accurately controlled and localized throughGPS, they can carry a larger payload, and they can continuously broadcast imagery

data to observers in real-time.The team arrived on Dec. 17, 2008, and surveyed the site, which was still indisarray, for access, possible problems, etc. The site was foggy and cloudy for mostof the duration, with bright sunshine only as the team departed on the afternoon ofDec. 19. Temperatures were 1013C. A launch point was selected on a sandy spiton the interior side of the bridge about 0.5 km away and all operations we conductedwithin line of sight. The spit had better access than lowering the vehicles over themetal seawall. The cars parked about 10 m from the water and the transport van gotstuck in the sand and had to be pulled out, reinforcing the unfavorable conditions forlarger UMVs requiring trailers. A boat ramp was available about 1.5 km away and a

chase boat was launched from there but could not navigate the shallows in the foggyconditions. No structures or electricity was available. There were some fishermenpresent but generally this did not pose a problem. The team worked for two days,spending two nights in a nearby beach house that had been rebuilt.

Deployments had to be scheduled for the change of tides to minimize the impact ofthe currents. The USV was deployed twice on Dec. 18, initially with a safety line untilit was established that the motors were sufficient for the current. The YSI Ecomapperwas deployed five times concurrently with the USV and the VideoRay was deployedonce. The USV was deployed twice on Dec. 19th to ensure repeatability and to get

practice deploying the vehicle.

3.2 Unmanned Marine Vehicles

The three vehicles used at Rollover Pass are shown in Fig.3and represent each ofthe three types of UMV. Each are commercially available.

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

8/19


a b

c

Fig. 3 Three UMVs used: a Sea-RAI USV, b VideoRay tethered ROV, and c YSI Ecomapperautonomous UUV

The Sea-RAI unmanned surface vehicle shown in more in Fig. 4 is a customplatform built by AEOS based on two 6ft catamaran hulls. It is similar to Charlie [29]and smaller than ACES [30] but more stable than the kayak-based SCOUT[31]. Itis capable of autonomous waypoint navigation and supports teleoperation througha wireless F4W 802.11 mobile ad hoc network. It is an adaptation of the AEOSplatform built for environmental mapping [23]. The Sea-RAI carries a DIDSONacoustic camera for subsurface inspection and a three video cameras (forward, rear,hemispherical) for viewing above the waterline. The robot can carry additionalsensors. The Sea-RAI is battery powered (lead-acid) and can operate for 46 hdepending on the currents. The Sea-RAI uses a team of three operators, one to pilot,one to collect imagery, and one to serve as a safety officer. In addition, one or more

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

9/19


Fig. 4 Sea-RAI with keycomponents labeled

other engineers were engaged in observing and directing data collection. A notablefeature of the Sea-RAI is that it stores sensor data and the internal state of the robot,creating a database that can be displayed in a unique Google Earth interface. Dataat any point in time or location can be retrieved with a click.

The VideoRay ROV and a YSI Ecomapper were also deployed. The VideoRay isa tethered ROV already in use by some departments of transportation for visualinspection of bridges and debris, either lowered off a bridge or from a boat. It

generally carries a video camera and a larger version can be modified to carry ascanning acoustic camera such as a Blueview imaging sonar. The VideoRay ROVrequired three operators, one to pilot, one to manage the tether and a safety officer.

The YSI Ecomapper is used for environmental research. It does not use a tetherand navigates with waypoints based off of GPS coordinates while above water andinertial navigation when submerged. It carries a side-scan sonar suitable for mappingdebris fields. The Ecomapper was autonomous but required a two-person team toset up and use.

3.3 Inspection Results

The Sea-RAI was successful in meeting its primary mission objectives. The robotwas deployed tghree times, twice on Dec. 18 and once on Dec. 19. It used waypointnavigation to either conduct mapping of the debris field or to travel to the bridge. TheSea-RAI was manually controlled near the bridge. Missions lasted approximately2 h, with about 1 h of active investigation of the bridge and debris field in betweenchanges in tides. The VideoRay and Ecomapper were also deployed.

The USV found no sign of scour or washout of the bridge pilings, as can be seenin Fig.5. The upper image shows the healthy pilings at the Rollover Pass Bridge.Pilings give off a bright, sharp line where the foot meets the ground. This is contrastto dark holes in the image below showing the scour at pilings for a dock at MarcoIsland inspected after Hurricane Wilma.

Figure 6 shows the path of a typical debris mapping run where the robot wasprogrammed to autonomously complete a raster scan of the area of interest usingGPS waypoints (green dots). The Sea-RAI did not find debris that would obstruct

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

10/19


Fig. 5 DIDSON imagery:ashows no scour at RolloverPass Bridge after HurricaneIke,bshows scour (darkholesin front of pilings) atMarco Island dock afterHurricane Wilma

a

b

Fig. 6 Screen capture fromUSV showing waypoints andpath for debris mappingoverlaid on Google mappre-Ike photo

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

11/19


Fig. 7 DIDSON imagery:pallet and debris in thechannel

navigation through the Rollover Pass channel or present a hazard to the bridge.Figure7shows the typical debris scattered throughout the channel.

4 Comparison of UMVs

To be successful for post-disaster bridge inspection, or any other shallow, clutteredlittoral application, the UMV must be able to meet the mission requirements underextreme environmental conditions and in the presence of debris and unpredictable

sea floor topologies and with unfavorable deployment conditions. As shown inFig.8,the criteria for comparing types of UMVs can be divided into five categories,

Fig. 8 Comparison of USV,ROV, and UUV for thepost-disaster bridge inspectiondomain.Greenmeans highlysuited,orangesome concerns,andredill-suited

USV ROV UUV

Mission requirements

perceive damage

perceive scour

map debris field

Operate in extreme conditions

function in currents

agile station-keeping

function with turbidity

Operate in shallow, cluttered littoral regions

work in close proximity to structures

avoid underwater obstacles

avoid above water obstacles

Deployability

portability

tethered

Desirables

Supply real-time imagery

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

12/19


described below. In general, the USV is more suitable for the entire scope of tasks,with the ROV and UUV highly capable for subsets.

The mission requirements are the essentials; in the case of post-disaster bridgeinspection a UAV must be able to perceive damage and scour in the substructure

as well as map the debris field. This requires both been able to get close enough tothe structures in order to map in order to perceive damages and scour, To do this,the UMV must be must be able to get close enough to the structure and carry thecorrect type of sensors that can penetrate turbidity if present. On the other on theother hand in order to map the debris field, the UMV must be able to transit longdistances while also penetrating some turbidity.

The UMV must be able to operate in extreme environmental conditions. It mustbe able to function in high currents and maintain accurate stationkeeping. As notedbefore it must also function in turbidity. This last condition is redundant with theperceptual tasks in the mission requirements category but is broken out for clarity.

Accurate stationkeeping means that the object of interest remains stationary whilethe operators study the structure. It makes a difference whether the operator islooking straight at or perpendicular to a structure, and it can be very frustrating ifthe system is constantly moving off course and repositioning at a different viewingangle.

The UMV must be able to operate in shallow, cluttered littoral regions. This meansworking in close proximity to structures with only a few meters standoff distance.The UMV should be able to avoid underwater obstacles as debris and the sea floortopology may have changed. Operations in shallow water requires a sensitivity to the

tides; for example, a surface vehicle and UUV may be able to operate in a certainregion during high tide but become grounded at low tide. Methods and sensors fordetecting obstacles, such as boats, structures, buoys, and debris, above the water lineexist.

Deployabilityis a practical concern. The UMV must be portable and not requirethe use of boat ramps, lifts, or any other aids that may have been destroyed. Tethersinterfere with deployability because of tether management; as noted in [32] andby our experience with the ROV, it is extremely difficult to keep the tether fromtangling around debris or structures in these complex, high current environments.

In addition to the mission requirements and operating and deployment conditions,

it is desirable for a UMV to provide the operators with real-time imagery. This allowsthe operators to inspect in real time in order to redirect the vehicle for further moredetailed observations if something is of interest. It also means that all data can becaptured immediately and if the vehicle malfunctions or sinks, data will not be lost.

Figure 8 captures the qualitative rating of the team for each of the robots atRollover Pass. Green means the vehicle generally meets that attribute, orange thatit meets it but with some notably drawback, and red means that it does not meetit. A simplistic scoring scheme, green=3, orange=2, and red=1, for the entire setof criteria suggests a preference ordering of USV, UUV, and ROV. However, thisinterpretation of the table is misleading. Note that the USV is the only vehicle thatcan perform both the close proximity portions of the mission (perceive damage tothe substructure) and the wider area component (map the debris field). The ROVcan perform the close proximity mission if the turbidity is low allowing the videocamera to see over several meters, but the UUV cannot because its navigation systemhas significant errors and it might miss or collide with the structure and would bedifficult to be sure it had surveyed the entire structure. The UUV can map a debris

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

13/19


field with its side-scan sonar, assuming no navigational hazards are present, but theROV nether has sufficient tether length nor a positioning system that would permitaccurate mapping.

In terms of operating in extreme conditions, none of the vehicles function well in

high currents. The USB and UUV being larger have more power, while the smallerROP has the tether to prevent it from being swept away. None of the vehiclesperform acceptable stationkeeping. The USV has to contend with currents and withthe wind as documented by[33], while the ROV has deal with currents. The UUVcannot operate in close proximity to structures. The sonars on the USV and UUVcan image through turbidity, while the ROV uses only a video camera which cannotpenetrate more than a meter through cloudy water.

The USV can operate in shallow, cluttered littoral regions, especially in closeproximity to structures, and can avoid underwater obstacles by virtue of its shallowdraft. The UUV can avoid above water obstacles by virtue of swimming under them.The USV and ROV must rely on the operator to manage avoidance of above waterobstacles. None of these systems have autonomous obstacle avoidance though thatmay change in the future.

Deployability in terms of the portability and launching of all three types of vehicleis high. The ROV poses some concerns because of where an operator might have tostand to deploy the tether; standing on a bridge that is suspect is worrisome and theareas surrounding the collapse bridge may also be difficult to reach or unsteady. TheUSV and UUV were rated higher because they were not tethered, due to problemswith tangling.

The UUV did not score well in regards to real-time imagery. Because a UUVoperates under water and can only communicate through a low baud rate acousticmodem, it does not supply real-time imagery. Instead data must be stored for deliverywhen the vehicle surfaces or downloaded when it returns to home. This poses asignificant risk of loss of data. In contrast, a USV operates above the surface withpresumably strong wireless conditions and the ROVs tether is a communicationsline supplying real-time imagery.

5 General Findings

The evaluation produced findings in three areas: control challenges for UMVs,human-robot interaction, and uncertain data. The control confirmed earlier findingsfrom Wilma[7] plus observations from other marine vehicle research [6,17,3234].The human-robot interaction findings are similar to the Wilma findings [7] and areconsistent with small unmanned aerial vehicle issues[35]. The challenges in handlingand fusing the sensor data is new.

5.1 UMV Control Challenges

The fieldwork at Rollover Pass identified three major control challenges for UMVs.The first is navigation and station-keeping in swift currents, a well-known problem[33,36] but as yet unsolved. The swift currents limited the times and duration theSea-RAI, EcoMapper, and VideoRay could be used. The Sea-RAI was actually puton a safety line during the first run to make sure it could be recovered. The problems

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

14/19


with currents and station keeping is not unknown, see [33] for another example.While station keeping and localization is difficult for all three types of UMVs, theUSV is especially challenging because of the higher degrees of freedom involved(rotation/translation of the vehicle, pan/tilt of the sensor) and more environmental

impacts (wind plus water).The second challenge is navigating in the presence of GPS loss or errors. As noted

in [7], operations near bridges interfere with GPS signals impacting the USV andthe surface operations of the UUV. The Sea-RAI had to be teleoperated for theactual inspection task. The GPS loss was approximately 1% away from the bridgeand 22% under or in the shadow of the bridge. Therefore GPS will not be sufficientfor close inspection. And GPS errors in cluttered littoral environments can lead tocollisions, as seen when the Ecomapper while using GPS for a surface-based scan ofthe channel bumped into a barricade. Figure9shows GPS outages based on locationand that while GPS errors are more frequent near the shadow of the bridge, theyalso occur in relatively open spaces.

The third challenge is obstacle avoidance for underwater vehicles [17], as inci-dents occurred in two of the five runs with the UUVs (40% incident rate). TheEcoMapper could not safely operate submerged through the channel until the Sea-RAI mapped the underwater debris and gave some indication that the channel wasclear; unlike bays or open water, littoral regions after a disaster may be cluttered withunmodeled obstacles. Without the Sea-RAI, it would have been risky to operatesubmerged. Even operating on the surface using GPS, the UUV collided with achannel barricade. The tethered VideoRay ROV became tangled around a pipeline.

This highlights the need for tether management and awareness of where the tetheris related relative to obstacles and the robot. Recent commercial developments havecreated a smart tether which keeps up with its own position, which might solve theproblem in the future.

Obstacle avoidance for surface vehicles is less of a problem. Above the waterlinenavigation should be possible with the addition of ranging sensors, while USV with

Fig. 9 Map of GPS outage fora typical USV run

Red is GPS loss

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

15/19


a shallow draft are unlikely to collide with submerged vehicles. The only concern isprotecting the sonar payload.

5.2 Human-Robot Interaction

While imaging and perception difficulties in using UMVs are documented [34], thefieldwork at Rollover Pass addressed the broader role of human-robot interaction.It confirmed the role of humans in shared autonomy, reinforced the need formultiple displays, and illustrated the lack of resilience in design and displays. TheUSV and ROV were teleoperated, though the USV performed the debris mappingand navigation in the open channel autonomously, resulting in a shared autonomyscheme referred to as remote presence [37], where the operators work with the vehiclein real-time as extensions of themselves.

The role of humans were piloting, payload specialists (operating the sensors),subject matter experts (interpreting the data), andsafety oversight. A minimum of fourpeople were involved. These roles relied on a close interaction between all groups.As noted above, the robot was teleoperated for a significant portion of the timeand the robot was in line of sight for the entire mission. Alternative teleoperationmechanisms such as creating a virtual reality system proposed in[38] may be usefuland reduce the logistics footprint. Work reported in[39] in human-guided autonomywork with ROV suggest operators prefer a exocentric, geolocative display but offerno studies to confirm this.

A surprising human-robot interaction was the need for two-way audio, as therewas always a person involved in launching or recovering the robot. At RolloverPass, the base station was approximately 0.4 km from the launching area. Cleggand Peterson [40] discusses trials for a REMUS UUV operating in shallow waterswhere the operators were several kilometers away but does not describe the lessonslearned or posit any guidelines. Two-way audio would have permitted the operatorto coordinate with the handler in the field rather than be reduced to gesturing at acamera and having the operator signal through panning or tilting and having a spareteam member run back and forth.

The need for multiple displays to accommodate multiple observers was noted in

general in[41] and the Hurricane Ike fieldwork reinforced this. Additional displaysare needed for the specialists and subject matter experts and the display shouldbe customizable. For example a civil engineer may just want to see the DIDSONoutput while the operator sees the complete interface which includes vehicle health,path, etc.

The deployment also showed problems with resilience and how hard it is forhumans to understand what is going on with vehicles. The DIDSON was knockedout of alignment by the force of the water, causing the operators to get confusedabout which way it was pointing relative to the vehicle. This led to coordinationchallenges as the specialist had to give counter intuitive commands to the pilotto maintain views of pilings. The design of the payloads should prevent normalslippage and the display should provide diagnostics for confirmation of settingsand positions. This has been addressed in the newest version of the DIDSONvisualization software, by providing an indication of the sonar orientation relativeto the vessel it is deployed on. However, more comprehensive health monitoringsystems such as those proposed in [32] are needed.

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

16/19


5.3 Multi-Robot Cooperation

The experiences also suggested that multi-robot cooperation may be beneficial.Homogeneous USV or UUV teams have been investigated. Prior work by [42]

has considered teams of surface vehicles acting as a water-based distributed sensornetwork near coasts, while[43] explored UUVs for lane-finding through minefieldswhich might be useful for mapping ship channels. Heterogeneous teams havealso been examined. USV-UAV teaming was explored at post-Hurricane Wilmadeployment [7, 12, 44] discusses ground-sea-air coordination, while [45] describescoordinating surface with untethered underwater vehicles. The USV-UUV teamingis attractive given that a USV could map the region sufficiently for UUV navigation.As noted in [7], the USV could also serve as a mother for a tethered ROV, seeingwhere the USV could not, reducing the amount of tether and risk of tangling of aROV, and help keep overwatch on the tether. UMV researchers have proposed theNEPTUS framework for multi-UMV control [46], while[47] has conceptualized amulti-level global and local planning system for team coordination.

5.4 Uncertain Sensor Data

The purpose of the UMVs is inspection, to observe and record; this creates chal-lenges in sensing processing, particularly in handling large datasets and managinguncertainty. The Sea-RAI collected a continuous stream of imagery from aboveand below the waterline. The imagery was color video and sonar, leading to large

datasets of information. Furthermore, the data had uncertainties in localization andin content due to shadows and differing viewpoints from the vehicle and DIDSONangles, presenting challenges for accurate 3D reconstruction and in understanding.

6 Conclusions

UMV Post-disaster inspection offers economic advantages by speeding up and reduc-ing the cost of rescue and recovery while reducing human risk to divers who currently

manually inspect bridges. In order to effectively conduct post-disaster inspection ofstructures in littoral regions, a UMV must be able to control changes in velocity anddirection to subject object investigation under extreme environmental conditions(e.g., tides, currents, wind) in the presence of debris and changes in topology, andunder unfavorable deployment conditions (e.g., no boat ramps or nearby safe areasof operation).

The experiences at the Rollover Pass Bridge showed that UMVs have sufficientutility for immediate use in littoral inspection. While tethered ROVs have begunto be explored by transportation departments, USVs appear to be more promisingthan UUVs because of navigability in high currents, no tether to tangle, reducedvulnerabilities to obstacles, ability to carry payloads such as acoustic cameras whichcan penetrate turbidity, and real-time transmission of data. Regardless of surfaceor underwater deployments, UMVs pose many open research questions in control(especially with GPS dropout rates on the order of 20% or higher), human-robotinteraction,cooperation between surface and underwater vehicles, and handling oflarge data sets of uncertain sensor readings.

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

17/19


The fieldwork combined with the review of literature suggest a possible roadmapfor improving post-disaster inspection of bridges and littoral regions. The ultimategoals are to improve inspection and to reduce the number of humans on the human-robot team. This may be accomplished through seven milestones; the milestones are

presented as sequential but in reality the research and development for several couldbe done concurrently.

Standardize mission payloads. UMVs should have an acoustic camera and avideo camera to cover both high and low levels of turbidity. Given that inspectionmay involve slow movements close to structures, side scan sonars which rely onmotion do not appear appropriate.

Health monitoring.A UMV should have sufficient sensors or encoders to cor-rectly determine the position of its components and the overall health.

Improved teleoperation through better human-robot interaction. HRI is a sig-

nificant weakness in UMVs. Studies are needed to resolve competing displays(see [11] versus [39]), while payloads such as two-way audio and extra displayscan facilitate team coordination.

3D obstacle avoidance. UMVs need to avoid obstacles above and below thewaterline but also consider the tides and depths. Above the waterline rangesensors must be hardened to work when splashed with water. Obstacle avoidanceis expected to speed navigation and improve the overall safety of operations.

Station-keeping.Significant advances in vehicle control must be made to enablestation-keeping. This is expected to reduce the load on the mission specialist andengineers because they have a steady image to look at and eventually reduce thenumber of people involved in the robot team.

Handle large data sets.As UMVs become more successful at inspection, a hugeamount of imagery and data will be generated and must be handled.

Cooperative sensing.Current systems put the responsibility for surveying the en-tire structure and identifying damage or scour on the human. Advances in controland in image processing should be able to ensure coverage of the structure andassist the image interpretation by cueing areas of interest. These advances incontrol theory and artificial intelligence would likely increase performance andthe reliability of the inspections.

Our current work has focused on creating a refined version of the Sea-RAIplatform and software to serve as a platform for further research in control, human-robot interaction, and sensing.

Acknowledgements The authors thank Dr. Dennis Christiansen (TTI) for his help, the TexasEngineering Extension Service (TEEX) for the generous loan of response equipment and personnel(especially Daniel May for his expert help and suggestions), and Kimberly Mallett for logistics andsupport.

References

1. Murphy, R.R., Steimle, E., Lindemuth, M., Trejo, D., Hall, M., Slocum, D., Hurlebas, S., Medina-Cetina, Z.: Robot-assisted bridge inspection after Hurricane Ike. IEEE Workshop on SafetySecurity Rescue Robotics, pp. 15. IEEE, Denver (2009)

2. Brown, J.: Robot inspects damaged bridge. Civ. Eng. 78(3), 40 (2009)3. Merriam, E.: MDSU-2 Arrives in Minneapolis Prepared to Help. 1 May 2010 (2007)

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

18/19


4. Goldwert, L.: Minneapolis Honors Bridge Collapse Victims. 1 May 2010 (2007)5. Leibholz, S.: Trends in robotics and implications for the marine environment. OCEANS 1984,

vol. 15, pp. 100105 (1983)6. Ageev, M.D., Kiselev, L.V., Shcherbatyuk, A.P.: Tasks for autonomous underwater robot. Fifth

International Conference on Advanced Robotics, 1991. Robots in Unstructured Environments,

91 ICAR, pp. 13601364, vol. 2 (1991)7. Murphy, R., Steimle, E., Griffin, E., Cullins, C., Hall, M., Pratt, K.: Cooperative use of unmanned

sea surface and micro aerial vehicle at Hurricane Wilma. Journal of Field Robotics 25(3), 164180 (2008)

8. Steimle, E., Murphy, R., Hall, M., Lindemuth, M.: Unmmaned marine vehicle use at HurricanesWilma and Ike. MTS/IEEE OCEANS 2009, pp. 16. (2009)

9. Murphy, R., Tadokoro, S., Nardi, D., Jacoff, A., Fiorini, P., Erkmen, A.: Rescue Robotics. In:Sciliano, B., Khatib, O. (eds.) Handbook of Robotics, pp. 11511174. Springer-Verlag (2008)

10. Anderson, B., Crowell, J.: Workhorse AUV: a cost-sensible new autonomous underwater vehiclefor surveys/soundings, search & rescue, and research. In: Proceedings of MTS/IEEE OCEANS,2005, pp. 16 (2005)

11. Lindemuth, M., Murphy, R., Steimle, E., Armitage, W., Dreger, K., Elliot, T., Hall, M., Kalyadin,

D., Kramer, J., Palankar, M., Pratt, K., Griffin, C.: Sea-RAI: a Marsupial USV-UAV team forlittoral inspection. IEEE Robot. Autom. Mag. (to appear, 2011)

12. Murphy, R., Stover, S., Pratt, K., Griffin, C.: Cooperative damage inspection with unmannedsurface vehicle and micro unmanned aerial vehicle at Hurricane Wilma. IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2006, pp. 99 (2006)

13. Watanabe, K., Nakamura, A.: A design of tiny basin test-bed for AUV multi agent. In: Proceed-ings of MTS/IEEE, OCEANS, 2005. pp. 10021008, vol. 2 (2005)

14. Meyrowitz, A.L., Blidberg, D.R., Michelson, R.C.: Autonomous vehicles. Proc. IEEE 84(8),11471164 (1996)

15. Castelin, S., Bernstein, P.: A notional scenario for the use of unmanned system groups in littoralwarfare. 2004 IEEE/OES autonomous underwater vehicles, pp. 14 19 (2004)

16. Alleman, P., Kleiner, A., Steed, C., Hook, D.: Development of a new unmanned semi-

submersible (USS) vehicle. OCEANS 2009, MTS/IEEE BiloxiMarine Technology for OurFuture: Global and Local Challenges, pp. 16 (2009)

17. C. v. Alt, Allen, B., Austin, T., Forrester, N., Goldsborough, R., Purcell, M., Stokey, R.: Huntingfor mines with REMUS: a high performance, affordable, free swimming underwater robot.OCEANS, 2001. MTS/IEEE Conference and Exhibition, pp. 117122, vol. 1 (2001)

18. Bick, E.T., Barock, R.T.: CENTURION harbor surveillance test bed. In: Proceedings ofMTS/IEEE OCEANS, 2005. pp. 13581363, vol. 2 (2005)

19. Healy, P.D., Bishop, B.E.: Sea-dragon: an amphibious robot for operation in the littorals. 41stSoutheastern Symposium on System Theory, 2009. SSST 2009. pp. 266270 (2009)

20. Farrell, J.A., Li, W., Pang, S., Arrieta, R.: Chemical plume tracing experimental results with aREMUS A.U.V. Ocean 2003 Marine Technology and Ocean Science Conference (2003)

21. Ohata, S., Ishii, K., Sakai, H., Tanaka, T., Ura, T.: Development of an autonomous underwatervehicle for observation of underwater structures. In: Proceedings of MTS/IEEE OCEANS, 2005,vol. 3, pp. 19281933 (2005)

22. Williams, S., Mahon, I.: Simultaneous localisation and mapping on the Great Barrier Reef. IEEEInternational Conference on Robotics and Automation, 2004. Proceedings. ICRA 04. 2004,vol. 2, pp. 17711776 (2004)

23. Steimle, E.T., Hall, M.L.: Unmanned surface vehicles as environmental monitoring and assess-ment tools. OCEANS 2006, pp. 15 (2006)

24. Caccia, M., Bono, R., Bruzzone, G., Spirandelli, E., Veruggio, G., Stortini, A.M., Capodaglio, G.:Sampling sea surfaces with SESAMO: an autonomous craft for the study of sea-air interactions.IEEE Robot. Autom. Mag.12(3), 95105. 1070-9932 (2005)

25. Schiavon, R., Finotello, R., Terribile, T.: The supervisory control of ARAMIS, a system forrobotic inspection of sediments. OCEANS 2000 MTS/IEEE conference and exhibition, vol. 2,pp. 823828 (2000)

26. Ribas, D., Ridao, P., Tardos, J.D., Neira, J.: Underwater SLAM in a marina environment.IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. IROS 2007.pp. 14551460 (2007)

27. Ribas, D., Ridao, P., Neira, J., Tardos, J.D.: SLAM using an Imaging Sonar for Partially Struc-tured Underwater Environments. IEEE/RSJ International Conference on Intelligent Robots andSystems, 2006, pp. 50405045 (2006)

8/12/2019 art%3A10.1007%2Fs10846-010-9514-8

19/19


28. Whalen, J.E., Wallace, G.: Multi-platform sonar-based systems for submerged threats operatingin littorals. Proceedings of MTS/IEEE, OCEANS, 2005. pp. 684689, vol. 1 (2005)

29. Bibuli, M., Bruzzone, G., Caccia, M., Indiveri, G., Zizzari, A.A.: Line following guidance control:application to the Charlie unmanned surface vehicle. IEEE/RSJ International Conference onIntelligent Robots and Systems, 2008. IROS 2008, pp. 36413646 (2008)

30. Manley, J.E.: Development of the autonomous surface craft. IEEE Oceans, vol. 2, pp. 827832(1997)

31. Curcio, J., Leonard, J., Patrikalakis, A.: SCOUTa low cost autonomous surface platform forresearch in cooperative autonomy. IEEE OCEANS, pp. 725729 (2005)

32. Rae, G.J.S., Dunn, S.E.: On-line damage detection for autonomous underwater vehicles. Pro-ceedings of the 1994 Symposium on Autonomous Underwater Vehicle Technology, 1994. AUV94., pp. 383392 (1994)

33. Pereira, A., Das, J., Sukhatme, G.S.: An experimental study of station keeping on an underactu-ated ASV. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS2008., pp. 31643171 (2008)

34. Duo-Min, H., Seet, G.G.L.: Underwater vision enhancement in turbid water by range-gatedimaging system. Summaries of papers presented at the Conference on Lasers and Electro-Optics,

2001. CLEO 01. Technical Digest, p. 378 (2001)35. Murphy, R., Pratt, K., Burke, J.: Crew roles and operational protocols for Rotary-Wing Micro-

UAVs in close urban environments. ACM SIGCHI/SIGART Human-Robot Interaction, pp. 7380 (2008)

36. Wood, S., Rees, M., Pfeiffer, Z.: An autonomous self-mooring vehicle for littoral & coastalobservations. OCEANS 2007Europe, pp. 16 (2007)

37. Murphy, R., Burke, J.: From remote tool to shared roles. IEEE ROBOT. AUTOM. MAG.(special issue on New Vistas and Challenges for Teleoperation) 15(4), 3949 (2008)

38. Qingping, L., Chengi, K.: Virtual tele-operation of underwater robots. IEEE International Con-ference on Robotics and Automation, 1997. Proceedings., 1997, vol. 2, pp. 10221027 (1997)

39. Murphy, C., Singh, H.: Human-guided autonomy for acoustically tethered underwater vehicles.OCEANS 2008, pp. 18 (2008)

40. Clegg, D., Peterson, M.: User operational evaluation system of unmanned underwater vehiclesfor very shallow water mine countermeasures. In: Proceedings OCEANS 2003, vol. 3, pp. 14171423 (2003)

41. Murphy, R., Stover, S.: Gaps analysis for rescue robots. ANS 2006: Sharing Solutions for Emer-gencies and Hazardous Environments (2006)

42. Eickstedt, D.P., Benjamin, M.R., Schmidt, H., Leonard, J.J.: Adaptive Control of heterogeneousmarine sensor platforms in an autonomous sensor network. IEEE/RSJ International Conferenceon Intelligent Robots and Systems, 2006, pp. 55145521 (2006)

43. Redfield, S.A.: Lane finding using homogeneous groups of cooperating autonomous vehicles(2004)

44. Yavnai, A.: Distributed decentralized architecture for autonomous cooperative. AutonomousUnderwater Vehicle, pp. 6167 (1994)

45. Curcio, J., Leonard, J., Vaganay, J., Patrikalakis, A., Bahr, A., Battle, D., Schmidt, H., Grund, M.:Experiments in moving baseline navigation using autonomous surface craft. IEEE OCEANS,vol. 1, pp. 730735 (2005)

46. Dias, P.S., Fraga, S.L., Gomes, R.M.F., Goncalves, G.M., Pereira, F.L., Pinto, J., Sousa, J.B.:Neptusa framework to support multiple vehicle operation. IEEE OCEANS- Europe, vol. 2,pp. 963968 (2005)

47. Khorrami, F., Krishnamurthy, P.: A hierarchical path planning and obstacle avoidance systemfor an autonomous underwater vehicle. American Control Conference, 2009. ACC 09, pp. 35793584 (2009)

Documents

art%3A10.1007%2Fs10846-010-9514-8