The Best Dishes Are Made With Fresh Ingredients: Creating CAD Files from an Enterprise SDE Server

Matthew FreidGIS Analyst – Bureau of Environmental ServicesCity of Portland Oregon

GIS to CAD Conversion:Project Goals

Provide most current data to all users Regardless of format & software used to access it

Provide data in native formats and in a structure that’s immediately useable No end user translation or management required

Make the same data and formatting available to internal and external users

City of Portland Bureau of Environmental Services

145,000+ accounts in 150 sq mi service area

2,200+ miles of pipe in combined, sanitary and storm system

14,000 sumps and sedimentation manholes that collect storm water runoff

CSO abatement program final years of 20 year, $ 1.4 billion program

Matthew Freid: GIS Analyst

Provide user support, database mgmt, app design/ integration, business intelligence

Focus on Collection System Asset Database

Our system combines an IMS/CMS with a GIS Data Maintenance Environment (DME)

GIS Data Delivery: High Availability

Data produced by DME is housed in SDE / SQL Server on Enterprise HUB for internal and external users, along with 100’s of other feature classes

Accessed with: Out-of-the box desktop GIS Custom applications for workgroups and

mobile users Web applications

GIS Data Delivery: High Availability

Used by maintenance workers, modelers and planners throughout the bureau and city

Integrated access to as-builts, easements and current data from IMS via www.portlandmaps.com

So … lots of the most current data available

Who’s Missing From This Picture???

Before DME was implemented Assets mapped in AutoCAD Batch exports converted for GIS users

Downsides Data consistency Lack of continuous coverage/integration with IMS Currency of other datasets from other sources

The Big Advantage CAD users had the most up to date files to work

with in their native format

From Best to Worst

Change in data input systems = no direct access to current data in familiar way!

Users could use GIS software to query and analyze the assets data, but… had to use out of date CAD files for designing

projects Result?

Create and use copies of final set of dwg’s

Initial Solutions: Nothing Is Ever Easy

Recreating file structure and extents? Not practical with GIS software

Direct access to SDE within AutoCAD? Cumbersome

Another need: Complete, self-contained set of CAD files for

distribution to outside consultants and contractors

Try and Try Again

Proprietary tool Ran initial translation from SDE to DWG using

AutoCAD - without complete formatting

Added formatting on the fly for a specific user defined area (BMAP)

Issues Preprocessing crashed often, needed human

intervention Can’t make files for users outside the city Sometimes didn’t handle datasets from outside

the bureau

Find A Better Way

Users liked the BMAP approach Get a file on demand within AutoCAD

Try FME? Already used for AutoCAD to SDE conversion so

…

Find A Better Way

New approach – use FME as a stable environment to:

Produce a complete set of fully formatted DWG files for external users

Deliver final files via BMAP tool for internal ACAD users

Allow any data on Enterprise GIS HUB to be incorporated into the DWG’s

Outside Help

Worked with Forte Consulting Our long suffering support and solution

provider for our initial conversion

Used FME Workbench and some Mapping files Goal: City staff could take over and manage

the process over time

Hooks to SDE

Inputs come directly from Enterprise SDE Servers Outputs to temp dir of DWG’s

Workbench File

Attributes and Transformers

Outputs

Beyond Workbench

2+ temporary DWG’s produced and loaded into a single sub-directory

Leave Workbench so we can produce a single DWG that contains features with different sets of OBJECT DATA attribution

Consultant helped us create a mapping file to accommodate this

Mapping File

DEFAULT_MACRO LogFileName mergeAcadLOG_FILENAME $(LogFileName)Count MAX_TO_LOG 50000LOG_APPEND NO

FME_DEBUG UNGROUPED UNCORRELATED MAPPING_FILE

READER_TYPE MULTI_READERMULTI_READER_TYPE{*} AUTOCAD_ODMULTI_READER_KEYWORD{*} AUTOCAD_OD

DEFAULT_MACRO SourceDataset C:\FMEdevelopment\temp_dwg_dirMULTI_READER_DATASET $(SourceDataset)

INCLUDE $(FME_MF_DIR)/../inc/FirstODautocadObject.fmi

FACTORY_DEF * SamplingFactory \ SAMPLE_RATE 1 \ INPUT FEATURE_TYPE * @Log()

MULTI_READER *

AUTOCAD_OD *

The FMI File

Controls Attribution and Field definitions Easy way to add new datasets Self documenting nature of a text file system

has it’s advantages

The FMI file

DEFAULT_MACRO DestDataset C:\FMEdevelopment\results\TEST.dwg

# Writer Information WRITER_TYPE AUTOCAD_OD

AUTOCAD_OD_DATASET $(DestDataset)

DEFAULT_MACRO AUTOCAD_Versions Release2007 AUTOCAD_OD_RELEASE $(AUTOCAD_Versions)

AUTOCAD_OD_AUTO_CREATE_LAYERS YES

AUTOCAD_OD_DEF TAXLOTS \ autocad_color 164 \ autocad_linetype CONTINUOUS \ autocad_layer_frozen no \ autocad_layer_hidden no \ autocad_layer_locked no \ autocad_od_entity_key_attr autocad_od_entity_key \ PROPERTYID char(7) \ RNO char(30) \ STATE_ID char(7) \ OWNER1 char(7) \ OWNER2 char(7) \ OWNER3 char(7) \ OWNERADDR char(7) \ OWNERCITY char(7) \ OWNERSTATE char(7) \ SITEADDR char(37) \ SITECITY char(37) \ SITE_STATE char(37) \ SITEZIP char(37)

Batch File Control

We use batch files to execute the Workbench and Mapping files

Allows for flexibility changing parameters such as server information

Allows for defining output map extent on the fly

Forte developed a simple interface to help us generate batch files

Batch File Control

fme C:\FMEDevelopment\workbench\NewSpodyOdy.fmw --GEODATABASE_SDE_IN_USERID_GEODATABASE_SDE XXXXXXX --GEODATABASE_SDE_IN_PASSWORD_GEODATABASE_SDE XXX --GEODATABASE_SDE_IN_INSTANCE_GEODATABASE_SDE

sde:sqlserver:gissqlprod1\gis92 --GEODATABASE_SDE_IN_SERVER_GEODATABASE_SDE gissqlprod1 --SourceDataset_GEODATABASE_SDE egh_public --LOG_FILE C:\FMEDevelopment\logs\1n2e36_P1.log --DestDataset_ACAD_3 C:\FMEDevelopment\temp_dwg_dir\1n2e36_p1.dwg --DestDataset_ACAD C:\FMEDevelopment\temp_dwg_dir\1n2e36_p2.dwg --XMIN 7685134.24619048 --XMAX 7690721.73190476 --YMIN 681689.792857143 --YMAX 687130.780952381 --REL_ACAD Release2004 --BlockTemplateFile C:\FMEDevelopment\template\portland_blocks.dwg --__WB_ODBC_SOURCE C:\FMEDevelopment\data\BatchDriver.xls

fme C:\FMEDevelopment\etc\mergeDWG.fme --LogFileName C:\FMEDevelopment\logs\merge_1n2e36.log --SourceDataset C:\FMEDevelopment\temp_dwg_dir --DestDataset C:\FMEDevelopment\results\1n2e36.dwg --BlockTemplateFile C:\FMEDevelopment\template\portland_blocks.dwg --ACAD_Versions Release2004

Output Example

Output Attribution

Results for CAD Users

More attribution

More datasets as input

Most current data available

Output standard for all internal and external users

Results For Me

A STABLE system to export from GIS to CAD that meets users expectation for file structure and contents

System is manageable – changes in GIS data or CAD standards can be easily accommodated going forward

Going Forward

Server-based solution for web-based ETL

Other formats such as Microstation

Citywide standard CAD files for all users with complete set of utilities

Utilize future versions of FME to streamline process and add new functionality

Thank You!

Questions?

For more information: Matthew Freid –

Matt.Freid@portlandoregon.gov Bureau of Environmental Services

Portland Oregon www.portlandmaps.com