INTEGRATION OF A SPATIAL MAPPING SYSTEM USING INTEGRATION OF A SPATIAL MAPPING SYSTEM USING GPS AND STEREO MACHINE VISION GPS AND STEREO MACHINE VISION

Ta-Te Lin, Wei-Jung Chen, Fu-Ming LuTa-Te Lin, Wei-Jung Chen, Fu-Ming Lu

Department of Bio-Industrial Mechatronics Engineering,Department of Bio-Industrial Mechatronics Engineering,National Taiwan University,National Taiwan University,

Taipei, Taiwan, ROCTaipei, Taiwan, ROC

INTRODUCTIONINTRODUCTION

Precision Agriculture Precision Agriculture GPS and GISGPS and GIS Spatial Mapping MethodsSpatial Mapping Methods Stereo Machine VisionStereo Machine Vision

INTRODUCTIONINTRODUCTION

Direct MeasurementDirect Measurement Aerial PhotographyAerial Photography Satellite and Airborne ScanningSatellite and Airborne Scanning Vehicular-based Position DetectionVehicular-based Position Detection Field-walkingField-walking Etc.Etc.

SPATIAL MAPPING METHODSSPATIAL MAPPING METHODS

OBJECTIVESOBJECTIVES

To develop concepts and integrate a spatial To develop concepts and integrate a spatial mapping system that allows for sensor mapping system that allows for sensor fusion and establishment of basic GIS for fusion and establishment of basic GIS for farm management.farm management.

To implement the system with the capability To implement the system with the capability to visualize, retrieve and store field data.to visualize, retrieve and store field data.

To test the performance of the system and To test the performance of the system and its components. its components.

MATERIALS & METHODSMATERIALS & METHODS

System DescriptionSystem Description GPS and Electronic CompassGPS and Electronic Compass Coordinate TransformationCoordinate Transformation Stereo Machine VisionStereo Machine Vision System Integration System Integration

GPS

Electronic

Compass

Stereo Machine

Vision

Host Computer

Positioning Database

Image Database

Base Map

Map Layer 1

User Interface

Map Layer 2

Position

Orientation

Image

SCHEMATIC DIAGRAM OF THE SYSTEMSCHEMATIC DIAGRAM OF THE SYSTEM

SYSTEM DESCRIPTIONSYSTEM DESCRIPTION

SYSTEM COMPONENTSSYSTEM COMPONENTS

SYSTEM DESCRIPTIONSYSTEM DESCRIPTION

Trimble AgGPS 132Trimble AgGPS 132 TCM2-20 Electronic CompassTCM2-20 Electronic Compass Dual CamerasDual Cameras Matrox 4 Sight-II and Meteor IIMatrox 4 Sight-II and Meteor II Power SupplyPower Supply MIL Image Processing LibraryMIL Image Processing Library MapObjects (ESRI) LibraryMapObjects (ESRI) Library

TRIMBLE AgGPS 132TRIMBLE AgGPS 132

GPS AND ELECTRONIC COMPASS GPS AND ELECTRONIC COMPASS

NMEA-0183 string

RS-232 interface

DGPS capability

TCM2-20 ELECTRONIC COMPASSTCM2-20 ELECTRONIC COMPASS

GPS AND ELECTRONIC COMPASS GPS AND ELECTRONIC COMPASS

magneto-inductive magnetic sensor

RS-232 interface

Heading accuracy 0.5

Tilt accuracy 0.2

COORDINATE TRANSFORMATIONCOORDINATE TRANSFORMATION

Co

ord

ina

te

Co

nv

ers

ion

WGS84 Spheroid Coordinates

Transverse Mercator projections

WGS84 Cartesian Coordinates

TWD67Cartesian Coordinates

TWD67 Spheroid Coordinates

GPS NMEA-0183 string

Ma

p

Pro

jec

tion

STEREO MACHINE VISION BASICSSTEREO MACHINE VISION BASICS

STEREO MACHINE VISIONSTEREO MACHINE VISION

Z

P(X,Z)

X

B

Xl Xr

f

Z

Imaging plane

)x(xw

rfBZ

lr

STEREO MACHINE VISION SYSTEMSTEREO MACHINE VISION SYSTEM

STEREO MACHINE VISIONSTEREO MACHINE VISION

Two JAI MCL-1500 Camera

640x480 Resolution

NTSC Signal

Focal Length: 5.8~58mm

STEREO MACHINE VISION SYSTEMSTEREO MACHINE VISION SYSTEM

STEREO MACHINE VISIONSTEREO MACHINE VISION

Matrox 4 Sight-II

Celeron 566Mhz, 64MB RAM, 6GB HD, Windows NT 4.0

Matrox Meteor II Image Processing Board

SOFTWARE FUNCTIONAL MODULESSOFTWARE FUNCTIONAL MODULES

SYSTEM INTEGRATIONSYSTEM INTEGRATION

Data Acquisition Module

GPS

Dual Cameras

Electronic Compass

RS-232 I/O

NMEA-0183 String

Parsing

MIL Imaging Processing Library

Mapping Module

Image Processing Module

Database Management

Module

Coordinate Transformation MapObjects

Pattern Matching

Range Estimation

User Interface

Positioning database

Image database

HARDWARE INTEGRATIONHARDWARE INTEGRATION

SYSTEM INTEGRATIONSYSTEM INTEGRATION

RESULTSRESULTS

Performance of the GPS Performance of the GPS Distance Measurement Distance Measurement Overall System Performance Overall System Performance Spatial Mapping and RetrievalSpatial Mapping and Retrieval

PERFORMANCE OF THE GPSPERFORMANCE OF THE GPS

-25

-20

-15

-10

-5

0

5

10

15

20

25

-25 -20 -15 -10 -5 0 5 10 15 20 25

Easting Error (m)

Nor

thin

g E

rror

(m

)

-25

-20

-15

-10

-5

0

5

10

15

20

25

-25 -20 -15 -10 -5 0 5 10 15 20 25

Easting Error (m)

Nor

thin

g E

rror

(m

)

Error in sampling point location using AgGPS 132 without differential correction

PERFORMANCE OF THE GPSPERFORMANCE OF THE GPS

-25

-20

-15

-10

-5

0

5

10

15

20

25

-25 -20 -15 -10 -5 0 5 10 15 20 25

Easting Error (m)

Nor

thin

g E

rror

(m

)

-25

-20

-15

-10

-5

0

5

10

15

20

25

-25 -20 -15 -10 -5 0 5 10 15 20 25

Easting Error (m)

Nor

thin

g E

rror

(m

)

Error in sampling point location using AgGPS 132 with differential correction

DISTANCE MEASUREMENTDISTANCE MEASUREMENTError of distance estimation using the stereo machine vision system with f = 5.8 mm and B = 230 mm

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

DISTANCE MEASUREMENTDISTANCE MEASUREMENTError of distance estimation using the stereo machine vision system with f = 58.0 mm and B = 230 mm

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100Distance (m)

Abs

olut

e E

rror

(m

)

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100Distance (m)

Abs

olut

e E

rror

(m

)

DISTANCE MEASUREMENTDISTANCE MEASUREMENTError of distance estimation using the stereo machine vision system with f = 5.8 mm and B = 430 mm

0

2

46

8

10

12

1416

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

2

46

8

10

12

1416

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

DISTANCE MEASUREMENTDISTANCE MEASUREMENTError of distance estimation using the stereo machine vision system with f = 58.0 mm and B = 430 mm

0

2

4

6

810

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

2

4

6

810

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

DISTANCE MEASUREMENTDISTANCE MEASUREMENTComparisons of errors of distance estimation

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100Distance (m)

Abs

olut

e E

rror

(m

)

0

2

46

8

10

12

1416

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

2

4

6

810

12

14

16

18

20

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

f=5.8, B=230mm f=58.0, B=230mm

f=5.8, B=430mm f=58.0, B=430mm

OVERALL SYSTEM PERFORMANCE OVERALL SYSTEM PERFORMANCE Error of distance estimation of the integrated spatial mapping system (f = 5.8 mm, B=230mm).

0

5

10

15

20

25

30

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

5

10

15

20

25

30

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

OVERALL SYSTEM PERFORMANCE OVERALL SYSTEM PERFORMANCE Error of distance estimation of the integrated spatial mapping system (f = 5.8 mm, B=230mm).

0

5

10

15

20

25

30

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

0

5

10

15

20

25

30

0 20 40 60 80 100

Distance (m)

Abs

olut

e E

rror

(m

)

OVERALL SYSTEM PERFORMANCE OVERALL SYSTEM PERFORMANCE Comparison with the Aerial Mapping Method

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALThe user interface of the spatial mapping system

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALThe user interface of the spatial mapping system

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALRetrieved image and indication of its scope on the map

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALRelocating field objects and creating new map layer

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALRelocating field objects and creating new map layer

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALBase field map of TARI experimental paddy field

SPATIAL MAPPING AND RETRIEVALSPATIAL MAPPING AND RETRIEVALMap layer showing scopes of saved image pairs

CONCLUSIONSCONCLUSIONS

An integrated spatial mapping system combining An integrated spatial mapping system combining GPS, electronic compass, and stereo machine GPS, electronic compass, and stereo machine vision is proposed and tested.vision is proposed and tested.

In the range of 100 m, the relative error of the In the range of 100 m, the relative error of the whole spatial mapping system was 6.8whole spatial mapping system was 6.82.6%.2.6%.

The integrated spatial mapping system provides a The integrated spatial mapping system provides a method to create digital maps incorporating image method to create digital maps incorporating image database.database.

The capability of retrieval and addition of range The capability of retrieval and addition of range information from image database allows for information from image database allows for dynamic and site-specific management of dynamic and site-specific management of agricultural sectors. agricultural sectors.

FUTURE WORKSFUTURE WORKS

System AccuracySystem Accuracy

System MobilitySystem Mobility

Database with Panoramic ImagesDatabase with Panoramic Images

3-Dimensional Reconstruction3-Dimensional Reconstruction

THANK YOUTHANK YOU

謝 謝謝 謝