04/11/23 104/11/23 1HYPACK® Multibeam Training Seminar

PERFORMANCE PERFORMANCE TESTINGTESTING

HYPACK ® MULTIBEAM COURSE

Performance TestingPerformance Testing

04/11/23 HYPACK® Multibeam Training Seminar 2

Questions We Might Ask About Questions We Might Ask About Our Multibeam Data:Our Multibeam Data:

•How repeatable repeatable is the Multibeam?is the Multibeam?

•Out to ‘what’ Out to ‘what’ Beam Angle Beam Angle can I use?can I use?

•How does it compare with Single Beam?compare with Single Beam?

Two Performance Tests in Two Performance Tests in MBMAX Help to AnswerMBMAX Help to Answer

•Beam Angle Test Beam Angle Test – Shows multibeam repeatability vs. beam angle.

•Single Beam Test Single Beam Test – Compares multibeam vs. single beam depths.

•Compare results to standards.

Bottom TypeBottom Type HardHard SoftSoftMaximum OutlierMaximum Outlier 1 ft1 ft 1 ft1 ftMax. Mean BiasMax. Mean Bias <0.1 ft<0.1 ft <0.2 ft<0.2 ft95% confidence95% confidence

depth < 15 ftdepth < 15 ft ±0.5 ft±0.5 ft ±0.5 ft±0.5 ftdepth 15 – 40 ftdepth 15 – 40 ft ±1.0 ft ±1.0 ft ±1.0 ft ±1.0 ft depth > 40 ftdepth > 40 ft ±1.0 ft±1.0 ft ±2.0 ft ±2.0 ft

From Table 3-1: Minimum Performance Standards for Corps of Engineers From Table 3-1: Minimum Performance Standards for Corps of Engineers Hydrographic SurveysHydrographic Surveys

Reference SurveyReference Survey

04/11/23HYPACK® Multibeam Training Seminar 3

LocationLocation

•Small area with a very flat, uniform bottom.

•The flat surface is to prevent DGPS position errors from influencing the results.

•Using lots of overlap and a narrow swath we get the best possible estimation of depth.

Preparation and SurveyPreparation and Survey•Two sets of seven (7) parallel lines. Separation = Water Depth.

•Reference MTX with 1’ x 1’ cells.

•Survey at low or high tides to minimize errors. (SV cast immediately before survey)

•Survey the 14 Reference lines.

•Survey the Check lines (middle lines NS and EW).

Flat, uniform reference area.

Survey grid.

04/11/23 HYPACK® Multibeam Training Seminar 4

Reference Survey to Surface XYZReference Survey to Surface XYZ

•Load reference survey into MBMAX.

•Apply tide and SV corrections. Filter to +/- 45 degree angle limit. Edit out remaining flyers.

•Use the reference MTX file for Area Based Editing.

•Save cell average Save cell average (minimum 3 soundings per cell) to Reference XYZ.

ProcessingProcessing

Full Swath Check LinesFull Swath Check Lines•Load check survey into MBMAX.

•Edit without angle limits.

•Start the Beam Angle Test Beam Angle Test from the Tools menu.

Final reference surface is quite flat. Depth range = 51’ to 52’.

04/11/23 5

Creating your Reference Matrix Creating your Reference Matrix

Choose Cell Sizes that will Maximize your Full Data SetChoose Cell Sizes that will Maximize your Full Data Set

04/11/23 5HYPACK® Multibeam Training Seminar

04/11/23 6

Points Points PerPer__Cell _ Cell _

11

22

3+3+

Reference Matrix with 1’ x1’ CellsReference Matrix with 1’ x1’ Cells Reference Matrix with 1.5’ x1.5’ CellsReference Matrix with 1.5’ x1.5’ Cells

Points per Cell StatisticsPoints per Cell Statistics

04/11/23 6HYPACK® Multibeam Training Seminar

04/11/23 704/11/23 7HYPACK® Multibeam Training Seminar

04/11/23 8

Beam Angle TestBeam Angle TestDifference Statistics: Reference Minus Check.

Comparison Tab is Overall ResultComparison Tab is Overall Result•X Axis is beam angle.

•Y Axis is depth difference.

•X’s show bias.

•O’s show repeatability.

Maximum difference at 95% confidence.

Details Tab is Result at Specific AngleDetails Tab is Result at Specific Angle•Histogram is the distribution.

•Max Outlier = Max difference.

•Mean Difference shows bias between reference and the check lines.

•95% confidence = 2 * Standard Deviation.

04/11/23 8HYPACK® Multibeam Training Seminar

Case Study: S/V DobrinCase Study: S/V Dobrin

Requirement Dobrin Result

Max Outlier < 1.0 Ft 0.5 Ft

Mean Difference < 0.1 Ft 0.0 Ft

95% Confidence < 1.0 Ft 0.3 Ft

04/11/23HYPACK® Multibeam Training Seminar 9

USACE EM 1110-2-1003 manual

•Successful Beam Angle TestSuccessful Beam Angle Test: Results for NY District S/V Dobrin fall within USACE performance standards for survey data collected in 40’ on a hard bottom.

•Results give system repeatability. Other methods must be included to find system accuracy; bar check, lead line comparison, etc.

Performance Test in ActionPerformance Test in Action

04/11/23 HYPACK® Multibeam Training Seminar 10

04/11/23 HYPACK® Multibeam Training Seminar 11

Interferometric MultibeamInterferometric Multibeam

Example Performance Test ResultExample Performance Test Result

04/11/23 HYPACK® Multibeam Training Seminar 12

Example Performance Test ResultExample Performance Test Result

Beamforming MultibeamBeamforming Multibeam

04/11/23 1304/11/23 13HYPACK® Multibeam Training Seminar

SINGLE BEAM SINGLE BEAM VS. VS.

MULTIBEAMMULTIBEAM

HYPACK ® MULTIBEAM COURSE

Single Beam TestSingle Beam Test

04/11/23HYPACK® Multibeam Training Seminar 14

Single beam check lines over the reference surface will give us a multibeam vs. single beam comparison.

Do single beam and multibeam soundings match up?

They better!They better!

Steps to the Single Beam TestSteps to the Single Beam Test

04/11/23 HYPACK® Multibeam Training Seminar 15

Single Beam Editor

Export

MBMAX

Check Lines: Raw Data

Edited All Format

XYZ Format

Reference.xyz

Reference.mtx

Single Beam Test

Launch Single Beam Test from Tools menu (MBMAX Phase 3)

Single Beam TestSingle Beam Test

16

Running the TestRunning the Test

•Select Single Beam Test from the Tools menu in MBMAX. Open the Reference XYZ file.

ResultsResults•Once again using difference statistics = Reference minus Check.

•Histogram shows the distribution and bias.

•Max Outlier = Max Difference.

•Mean Difference = Bias. This is the main point of the test!

•Standard Deviation (Sigma): Spread of the difference distribution.

•95% Confidence = 2*Sigma. 95% of the differences fall within 2*Sigma.

Not bad. Bias less than 0.1 foot!

04/11/23 HYPACK® Multibeam Training Seminar 16

Sample: Single Beam TestSample: Single Beam Test

04/11/23 HYPACK® Multibeam Training Seminar 17