Spatial Data, KML, and the University Web

17 September 2008

University of California, Santa Barbara -W

eb Standards Group

SPATIAL DATA, KML, and the UNIVERSITY WEB Alan Glennon

spatial.ucsb.edu

Image Source: NAIP, 2005

17 September 2008


eb Standards Group

SPATIAL DATA, KML, and the UNIVERSITY WEB

• Geographic Data Basics• Keyhole Markup Language (KML)• Authoring KML• Dynamic KML• Distributing Geographic Data• Querying Geographic Data• Policy and Best Practices

Topics


17 September 2008


eb Standards Group

Mission:

To facilitate the integration of spatial thinking into processes for learning and discovery in the natural, social, and behavioral sciences, to promote excellence in engineering and applied sciences, and to enhance creativity in the arts and humanities.

Engagement:

Hosting events (brownbags, workshops, lectures) Developing spatial analytic toolsOffering a help deskAssisting with research proposal development


17 September 2008


eb Standards Group

Geographic Data

Two practical perspectives:

Data that include or can be harvested for spatial references

Combinations of spatially-referenced points, polylines, polygons, and images with their associated attributes and relationships

17 September 2008


eb Standards Group

Geographic Data

17 September 2008


eb Standards Group

Geographic Data

Significance:

visually compelling

provide context and content; close things are usually more related; spatial order; pattern and trend recognition

popular; widespread use, particularly on the Internet

professional expectation: Google Maps as a baseline

17 September 2008


eb Standards Group

“…it would behove spatial scientists to articulate to the broader research community the potential of recording and making accessible spatial data in the appropriate formats — and the painlessness of the process.”

A place for everything: More researchers must record the latitude and longitude of their data.

Editorial: Nature 453, 2 (1 May 2008)

17 September 2008


eb Standards Group

Spatial Data at the UniversityResearch

data analysis and results dissemination

Classroomoffice locations, events, classrooms, miscellaneous directions, instructional projects

Infrastructuremaps, blueprints, streets, buildings, images, crisis management, physical resources, natural resources, office and classrooms, fleet management, planning, operational support, registrar and personnel databases

Studentdata on personal websites, often integrated with third party applications and data

NASA World Wind (client and server)OpenLayers (client)MapServer (server)Geoserver (server)GDAL/OGR (server-based geo-database toolkit)GRASS (full desktop GIS client)

Google Earth (client)Google Maps (client)Microsoft Virtual Earth (client)ArcGIS Explorer (client)MapQuest (client)

Google Earth Enterprise (server)ArcGIS (full desktop GIS client)ArcGIS Server (server)deCarta (server)AutoDesk AutoCad (client)AutoDesk ProductStream (server)Oracle Spatial (server)

17 September 2008


eb Standards Group

Sample Software

FreeO

pen SourceProprietary

• ESRI shapefile (.shp)• KeyHole Markup Language (.kml)• GeoRSS (.rss, .xml)• AutoCad DXF (.dxf)• Census TIGER• ESRI Coverage• ESRI Personal Geodatabase• GeoTIFF• Digital Raster Graphic (DRG)• Digital Elevation Model (DEM)• Spatial Data Transfer Standard (SDTS)• Image formats like jpg, tiff, gif, and png (often served via a WMS operation)

Considerations when choosing a file type for spatial data: What software support it? What does your consumer want? Is it fast? What type of data, complexity, and dynamics can it support? How easy is it to autogenerate and update (from your database to the new file format)? How well will it age?

17 September 2008


eb Standards Group

Example Geographic File Types

17 September 2008


eb Standards Group

Keyhole Markup Language (KML)

Keyhole Markup Language (KML) is an open source XML-based specification for expressing geographic data.

Developed as a Google Earth file format to represent georeferenced points, polylines, polygons, and images

KML has become widely supported by many other software applications and online mapping services.

Core data within a KML document include longitude, latitude, elevation, and name descriptions

A sizable number of advanced specifications also exist, including tags for cartographic customization, viewer position, time, and iterative data refresh calls.

17 September 2008


eb Standards Group

HelloWorld.kml

<?xml version="1.0" encoding="UTF-8"?><kml xmlns="http://earth.google.com/kml/2.2"><Document><name>HelloWorld1.kml</name><Placemark>

<name>Transformers</name><description>There are some transformers here.</description><Point>

<coordinates>-119.8512453552352,34.41944355498201,0</coordinates>

</Point></Placemark></Document></kml>

Something to noticeIf the first letter of a tag is upper case, it can hold child elements. If the first letter is lower case, it denotes a simple element—one that has no possible children.

17 September 2008


eb Standards Group

HelloWorld.kml

17 September 2008


eb Standards Group

HelloWorld.kml...<Placemark>

<name>Transformers</name><description>There are some transformers here.</description><Point>

<coordinates>-119.8512453552352,34.41944355498201,0</coordinates>

</Point></Placemark>

<Placemark><name>homebase</name><description>Line from baseball field home base to transformers.</description><LineString><tessellate>1</tessellate>

<coordinates>-119.8512439751936,34.41944165152853,0 -119.8524251973235,34.41862172899327,0</coordinates>

</LineString></Placemark>

</Document></kml>

17 September 2008


eb Standards Group

HelloWorld.kml

17 September 2008


eb Standards Group

HelloWorld.kml...

</LineString></Placemark>

<Placemark><name>Transformer Area</name><description>An area with electrical equipment in it.</description>

<Polygon><tessellate>1</tessellate><outerBoundaryIs>

<LinearRing><coordinates>-119.8511894139676,34.41968857861303,0 -119.8514849728913,34.41956527198274,0 -119.8512914112404,34.4192276886303,0 -119.851011240657,34.41934137112972,0 -119.8511894139676,34.41968857861303,0 </coordinates>

</LinearRing></outerBoundaryIs></Polygon>

</Placemark>

</Document></kml>

17 September 2008


eb Standards Group

HelloWorld.kml

17 September 2008


eb Standards Group

HelloWorld.kml...

</outerBoundaryIs></Polygon>

</Placemark>

<GroundOverlay><name>aerialphoto</name><description>A photograph of the transformers. Pictometry photograph taken from Microsoft Virtual Earth.</description><Icon>

<href>transformers.jpg</href><viewBoundScale>0.75</viewBoundScale>

</Icon><LatLonBox>

<north>34.41971426141618</north><south>34.41890159916832</south><east>-119.8508244826322</east><west>-119.851976467865</west><rotation>-84.34325349425195</rotation>

</LatLonBox></GroundOverlay>

</Document></kml>

17 September 2008


eb Standards Group

HelloWorld.kml

17 September 2008


eb Standards Group

Keyhole Markup Language (KM

L) Schema From: http://code.google.com/apis/kml/documentation/kmlreference.html

17 September 2008


eb Standards Group

Keyhole Markup Language (KML)

Specification Resources:http://code.google.com/apis/kmlhttp://www.opengeospatial.org/standards/kml/http://code.google.com/apis/kml/schema/kml22beta.xsd

KML Examples:http://code.google.com/apis/kml/documentation/http://bbs.keyhole.comhttp://www.google.com/search?q=filetype%3Akml

Schema Tools:XML editorValidating parser (like Xerces-C++)http://feedvalidator.org

17 September 2008


eb Standards Group

Authoring KML

Graphical User InterfaceGoogle EarthExport from ArcGISSpecialty website and applications (particularly plentiful

for GPS data importing)

Text EditorKomodo EditNotePad++

Parsers, Automation, and OnlineOGR from PostgreSQL and MySQLCoded parsersEditGridGoogle Docs

17 September 2008


eb Standards Group

Position the viewer to contain the spot you want to mark, and zoom to an appropriate viewing scale. To mark the spot, either:

Select Placemark from the Add Menu; or,

Click the Placemark icon on the toolbar menu at the top of the screen.

Authoring KM

Lin G

oogle Earth

17 September 2008


eb Standards Group

Authoring KML in a text editor

17 September 2008


eb Standards Group

Authoring KML with an online spreadsheet

Google Spreadsheets Tutorial: http://earth.google.com/outreach/tutorial_mapper.html

source: Google, Inc.

17 September 2008


eb Standards Group

Styling and Cartography

Using the CDATA Element If you want to write standard HTML inside a <description> tag, you can put it inside a CDATA tag. If you don't, the angle brackets need to be written as entity references to prevent Google Earth from parsing the HTML incorrectly (for example, the symbol > is written as > and the symbol < is written as <). This is a standard feature of XML and is not unique to Google Earth.

Source: http://code.google.com/apis/kml/documentation/kml_tut.html

<description><![CDATA[ <h1>CDATA Tags are useful!</h1> <p><font color="red">Text is <i>more readable</i>and <b>easier to write</b>when you can avoid using entity references.</font></p> ]]> </description>

<description><h1>Entity references are hard to type!</h1> <p><fontcolor="green">Text is <i>more readable</i> and <b>easier to write</b> when you can avoid using entity references.</font></p> </description>

17 September 2008


eb Standards Group


<description><![CDATA[<center><a href="http://www.uweb.ucsb.edu/~glennon/geysermap/images2/t25.jpg"><img src="http://www.uweb.ucsb.edu/~glennon/geysermap/images2/t25tb.jpg"></a></center><p>Category: geyser <br>Height: 5 m<br>Duration: 15 min <br>Interval: 2 hours + <br>Description: <a href="http://www.uweb.ucsb.edu/~glennon/geysermap/info/t25.htm">read more</a> / <a href= "http://www.uweb.ucsb.edu/~glennon/geysermap/tatiometadata.htm">metadata</a><p>Source: Glennon and Pfaff (2003)<br> <center><img src="http://www.uweb.ucsb.edu/~glennon/geysermap/ucsbgeog.jpg" alt="UCSB Geography">

]]></description>

Source: J. A. Glennon and R.M. Pfaff (2003), El Tatio Geysers, http://turnhole.com/chile.kmz

17 September 2008


eb Standards Group


Source: J. A. Glennon and R.M. Pfaff (2003), El Tatio Geysers, http://turnhole.com/chile.kmz

17 September 2008


eb Standards Group


<?xml version="1.0" encoding="UTF-8"?><kml xmlns="http://www.opengis.net/kml/2.2">

<Document><Style id="transBluePoly">

<LineStyle><width>1.5</width>

</LineStyle><PolyStyle>

<color>7dff0000</color></PolyStyle>

</Style><Placemark>

<name>Transformers</name><styleUrl>#transBluePoly</styleUrl><Polygon>

<extrude>1</extrude><altitudeMode>relativeToGround</altitudeMode><outerBoundaryIs>

<LinearRing><coordinates> -119.851205,34.419680,42-119.851008,34.419342,45-119.851317,34.419221,46-119.851486,34.419563,43

-119.851205,34.419680,42</coordinates>

</LinearRing></outerBoundaryIs>

</Polygon></Placemark>

</Document></kml>

17 September 2008


eb Standards Group



17 September 2008


eb Standards Group

Dynam

ic KML

Network Link

Using the <Link> element with the tag <href>, KMLcan reference local and remote files.

The <href> tag can refer to:

An image file used by icons in icon styles, ground overlays, and screen overlays

A COLLADA file used in the <Model> element

A KML or KMZ file loaded by a Network Link

17 September 2008


eb Standards Group

Dynam

ic KML

Network Link Example<?xml version="1.0" encoding="UTF-8"?><kml xmlns="http://www.opengis.net/kml/2.2"><Folder><name>Network Links</name><visibility>0</visibility><open>0</open><description>Network link example 1</description>

<NetworkLink><name>Random Placemark</name><visibility>0</visibility><open>0</open><description>A simple server-side script that generates a new random

placemark on each call</description>

<refreshVisibility>0</refreshVisibility><flyToView>0</flyToView><Link>

<href>http://nanocarta.com/tools/randomsphere.php?sample=100</href><refreshMode>onInterval</refreshMode><refreshInterval>5</refreshInterval>

</Link></NetworkLink>

</Folder></kml>

17 September 2008


eb Standards Group

Dynam

ic KML

Network Link Example

One hundred random locations refreshing every five seconds.

17 September 2008


eb Standards Group

Dynam

ic KML

Network Link

http://radar.weather.gov/ridge/kmzgenerator.phphttp://www.srh.noaa.gov/gis/kml/

Via a Network Link, a server-side script could call external databases or other online sources.

You could also create a webpage interface that assembles and returns a KML based on user preferences.

For example:Source: NOAA

17 September 2008


eb Standards Group

Dynam

ic KML

TimeIn KML, features can be associated with time with these tags:

<TimeStamp> associates a feature to an instant in time

<TimeSpan> associates a feature to a length of time. The tag requires a begin time and/or end time.

With respect to rendering, these tags typically serve as a visibility filter against a timeline.

17 September 2008


eb Standards Group

Time

When Google Earth reads a temporal tag within KML, a time browser appears.

Clicking on the clock icon brings up additional time navigation options.

Source: Google Earth and Declan Butler; http://www.nature.com/news/author/Declan+Butler

17 September 2008


eb Standards Group

Dynam

ic KML

Temporal Tags Example…

<Placemark><name>Transformers</name><description>There are some transformers here.</description><Point>

<coordinates>0,0,0<!simplified></coordinates></Point><TimeSpan>

<begin>1990</begin><end>2009</end>

</TimeSpan></Placemark><Placemark>

<name>homebase</name><description>Line from home base to transformers.</description><LineString>

<tessellate>1</tessellate><coordinates>0,0,0 1,1,0<! simplified></coordinates>

</LineString><TimeStamp>

<when>2008-09-24T10:30:15-08:00</when></TimeStamp>

</Placemark>…

17 September 2008


eb Standards Group Source: Google Earth

17 September 2008


eb Standards Group Source: Google Earth

17 September 2008


eb Standards Group

Dynam

ic KML

Temporal Tags Example

Source: Google Earth

17 September 2008


eb Standards Group

Distribution and SharingA) Delivery of raw or auto-assembled KML

KML file is shared via a website, email, or diskUser loads KML into the Google Earth application

B) Delivery of KML from Google MapsKML is hostedGoogle Maps renders KML

C) Delivery of KML with the Google Earth browser plug-inWebpage created using Google Earth APIHosted KML is rendered within Google Earth frame

D) Delivery with desktop applications like ArcGIS Explorer, Virtual Earth, NASA World Wind, etc.

Generally similar to standalone KML (option A)

E) Mediator web serviceWebsite service allows user collaboration, provides hosting, and file index or discovery mechanism. Examples: Google MyMaps, Flickr, Platial , Facebook, Yahoo! FireEagle

(Note: some of these overlap and/or could be combined)

17 September 2008


eb Standards Group

Distribution and Sharing

F) Server-based deliveryApplication and data are hosted. Interface may be public or, in some cases, secured. Examples: Google Earth Enterprise, OpenLayers, ArcGIS Server

G) Enterprise relational database managerTypically for intranet-type usage. Manages multiple user collaboration, conflicts, and versioningOracle Spatial, ArcSDE

H) Search engine discovery of spatial data (an emerging case); programmatic structured query using various sites’ APIs. Application launches as defined by user or OS (assisted by properly configured MIME type). Examples: Google Search, Yahoo! Pipes, Microsoft Popfly

(Note: some of these overlap and/or could be combined)

17 September 2008


eb Standards Group

Query

Select Geographic Database Analysis Tasks

Relational-spatial attribute query

Proximity analysis (buffer and distance calculation)

Spatial joins (intersection and union comparisons; inside/out)

Network analysis (routing and optimization; left/right; topology)

Raster comparison (Map Algebra)

17 September 2008


eb Standards Group

Query

Geocoding

The process of assigning geographic coordinates to a map feature, description, or address.

Input: Address or location descriptionExamples: Goleta, CA; 90210; New ZealandWorld Trade Center, downtown Los Angeles

Output: Geographic Coordinates45.2342W, 15.2346N, 1000m asl, WGS84

There are many complicating factors to geocoding. For example: the grammar of the input, positional accuracy / vagueness / scale, places with the same name, foreign languages, deprecated names, new names, misspellings, etc.

17 September 2008


eb Standards Group

Query

Geocoding

Despite the problems, geocoding is at the core of many web-based mapping applications.

KML offers the <address> element as an alternative to coordinates. Google Earth and Google Maps will attempt to geocode the address and render the position. Usability will largely depend on the input address and the intended application.

For example, the following KML uses the street address of a UCSB electrical transformer station (shown in the middle left of the screen; geocodedaddress is on the bottom right).

Note: In Google Earth and Maps, if a <Point> tag is provided in the KML also, it will take precedence over an <address>.

17 September 2008


eb Standards Group

<?xml version="1.0" encoding="UTF-8"?><kml xmlns="http://earth.google.com/kml/2.2"><Document>

<name>addressexample.kml</name><Placemark>

<name>Transformers</name><description>Better than nothing.</description><address>552 University Road, Santa Barbara, CA 93106</address>

</Placemark></Document></kml>

17 September 2008


eb Standards Group

Query

Geocoding Resources

Google Maps API

Yahoo! Maps and FireEagle API

MetaCarta Labs

Geocoder.us

Batchgeocode.com

NGA Geographic Names Search

Mapping Hacks by Erle, Walsh, and Gibson (available from O’Reilly). How to build your own geocoder.

Routing and Service Area

Routing implementations remain largely proprietary, though some open source options are beginning to emerge.

The Google Maps API offers access to driving directions. Proprietary web services are plentiful.

ArcGIS Server offers routing functionality.

Image source: ArcGIS Network Analyst, esri.com

17 September 2008


eb Standards Group

17 September 2008


eb Standards Group

Query: A

nalysis Resources

Google Earth DevelopersInteract with other developers and explore their work.(http://groups.google.com/group/kml-support)

ArcGIS ExplorerProprietary virtual globe with analysis functionality, particularly when coupled with other ESRI products. Note: UCSB has an ArcGIS site license. (esri.com)

Yahoo! PipesGraphical multistep web query that includes spatial data. (pipes.yahoo.com)

Microsoft PopflyGraphical multistep web query that includes spatial data. (popfly.com)

OpenLayersJavascript slippy map library with a gallery of numerous web applications. (openlayers.org)

GIS.comA “paleo” introduction to geographic information and analysis

Query: Exam

ple

Source: Yahoo! Pipes; Photos near Wineries, Author: Ido

17 September 2008


eb Standards Group

17 September 2008


eb Standards Group Source: ArcGIS Explorer, esri.com

http://www.esri.com/software/arcgis/explorer/graphics/showcase/longbeach-plume-lg.jpg

17 September 2008


eb Standards Group

Policy and Best Practices

The Policy Landscape

No homogenous body of geographic data web standards or regulations exist. So far, emerging precedents are largely arising from privacy and intellectual property law. Private enterprise is also gauging consumer reaction and trying to maximize utility, create monetization potential, and not alienate users.

Accessibility

Suggested Design Practices

Locational Privacy

17 September 2008


eb Standards Group


-Wired.com, September 11, 2008

17 September 2008


eb Standards Group

AccessibilityProvide a text description of the map’s or data’s purpose.

Provide descriptions of any included, integrated multimedia.

Use clear, descriptive names and labels.

Consider appropriate colors and contrast for people with color discernment difficulties or other visual impairment

Cite data sources; allowing users to investigate other mechanisms for its visualization

In technology selection, consider open 3D rendering formats. For instance, OpenGL calls to a graphics card can be captured and manipulated. Haptic feedback devices can more readily interpret the data.

Ensure links are simple, visible, and exposed (no hidden image links)

Be very cautious about rapid blinking and dynamic data refreshes. Provide warnings as necessary to assist epileptic population.


17 September 2008


eb Standards Group

Best PracticesValidate your KML

Use spaces and indentions even when the KML is automatically generated (when text editing, be careful for hidden characters)

Comment within the KML when something is not obvious (like with network links)

Generally, do not use KML as the database (usually better as a result of a database). Use KML to describe the geographic elements of a database. Also, consider that most software limits renderablegeometry

When hosting, use .htaccess to create MIME associations for KML and KMZ to Google Earth.


17 September 2008


eb Standards Group

Best PracticesUse GeoRSS to syndicate geographic data (refer to KML). HTML:RSS::KML:GeoRSS

Maximize link confidence (hide awkward script calls; links to external resources should be reliable; make sure the external resource acts like you think it should).

Provide usage information and instructions for complex data

Use object identifiers in KML. It will maximize users’ ability to search and access subunits of the data.

Provide data authorship credit (also affords author responsibility).


17 September 2008


eb Standards Group

Best PracticesConsider window viewing sizes (large images and long descriptions can take over an entire viewscreen).

Allow users control and navigation of layers.

Consider file size, number of points, complexity (warn as necessary). For instance, use Regionatorto manage large image overlays

Minimize the distraction of legends or screen overlays.

Do not use the awful (thankfully sparsely documented) blink style in KML.

Cache geocodes


17 September 2008


eb Standards Group


Right to privacy varies greatly by jurisdiction. The California Constitution, Article 1, Section 1, describes privacy as an inalienable right.

Spatial and temporal data resolution is a key component with respect to invasion of privacy. For example, spatially aggregated U.S. Census data are available soon after compilation. Individual Census responses are prohibited from release for 72 years.

Aggregating multiple individuals’ personal information is considered less invasive than individually identifiable information.

Geographic Privacy

17 September 2008


eb Standards Group


Geographic Privacy

Graphic source: David H. Williams/E911-LBS, October 25, 2006, The associated article argues for privacy considerations to be an fundamental part of Location-Base Service design

http://www.directionsmag.com/article.php?article_id=2323&trv=1

Internet users are comprised of all age groups, including those that may have no understanding of personal privacy or its consequences.

17 September 2008


eb Standards Group


Always tell users what you want to do with their location

Let users know when you are collecting their location information

Give users control of their own data

Make sure users' data are secure

Only allow users to manage their location, but not others

Don't be creepy

Yahoo! Fire EagleDeveloper Code of Conduct

Source: Fire Eagle Developer Code of Conduct, http://fireeagle.yahoo.net/developer/documentation/code_of_conduct

Note: These points are paraphrased.

17 September 2008


eb Standards Group


Notice - Individuals must be informed that their data is being collected and about how it will be used.

Choice - Individuals must have the ability to opt out of the collection and forward transfer of the data to third parties.

Onward Transfer - Transfers of data to third parties may only occur to other organizations that follow adequate data protection principles.

Security - Reasonable efforts must be made to prevent loss of collected information.

Data Integrity - Data must be relevant and reliable for the purpose it was collected for.

Access - Individuals must be able to access information held about them, and correct or delete it if it is inaccurate.

Enforcement - There must be effective means of enforcing these rules.

International Safe Harbor Privacy Principles

Source: United State International Trade Administration

17 September 2008


eb Standards Group

The near Future of Spatial Data

more. everywhere.


17 September 2008


eb Standards Group

The Future of Spatial Data

An evolving platform:extending the geographic model to Eames’ and Morrisons’ Powers of Ten

The Progression of Internet Maps Interaction:1. view (see my house)2. add (tag locations important to me)3. query (get directions, see patterns)4a. communicate (sharing and social interaction)4b. mirror world (more realism/physics)5. inhabit


17 September 2008


eb Standards Group

The Future of Spatial Data

Semantic Spatial Web(spatially literate, natural language Internet)

3D Internet(spatially visualized and inhabitable Internet. Virtual Reality)

Augmented and Mixed Reality(combination of real world and computer generated data)

Digital Earth(spatiotemporal database of everything)


• Geographic Data Basics• Keyhole Markup Language (KML)• Authoring KML• Dynamic KML• Distributing Geographic Data• Querying Geographic Data• Policy and Best Practices

Topics


17 September 2008


eb Standards Group


17 September 2008


eb Standards Group

Acknowledgments

Thanks to Guylene Gadal, the UCSB Web Standards Group, Rhonda Glennon, Mike Goodchild, Indy Hurt, Josh Bader, Karl Grossner, and the spatial@ucsb team!



Alan Glennonspatial.ucsb.edualan[at]spatial.ucsb.edu

CONTACT

Technology

Spatial Data, KML, and the University Web