Breathing lifeinto a travel survey

Roland Martin

Our client operates a large campus site in California. There is a limit to the number of vehicles that can enter their site at any given time.

Every year, they carry out a travel survey, with 20 prizes, including a grand prize of $2,000.

Around 90% of site users responded.

The survey

1. Please select the answer that applies to you.• I work on the main campus• I work at an off-campus site• None of the above

If “none of the above is selected” then skip to end of survey.

2. How do you usually commute to campus?• Drive alone in a passenger automobile• Bicycle• Carpool (two or more people)• Campus Shuttle (not as a transfer from another mode)• Motorcycle• Public transit (e.g. bus)• Walk

Raw data

Required output

• A database (PostgreSQL / PostGIS)• Access via desktop GIS (ArcGIS / QGIS)• Possibly some kind of dashboard…

• Insight

Cleaning up the non-geographic data

• A lot of questions are not fully expressed in column headers• Several column headers are repeated• Not all questions are answered• Some responses are “Other”, with the response in a second column• Some questions allow multiple responses, so responses are collected

into multiple columns• A lot of questions are not of interest for this study

Step 1: extract the schemaResponseID SchemaIDExternalDataReference q1Affiliation q2Staff class/Class level q3CAC q5Home lat q6Home lon q7FPC q8Please select the answer that applies to you. q9Although you are not eligible for this survey, we thank you for / your time and interest. q10Where on campus is your primary work site?  Please click on one location. - X q11Where on campus is your primary work site?  Please click on one location. - Y q12Which of the following is your off-campus work / site?(Note: Consider your commute to/from / this loc... q13Which of the following is your off-campus work / site?(Note: Consider your commute to/from / this loc...-TEXT q14From what city and zip code do you typically begin your / commute? / If you choose to, please also p...-City q16From what city and zip code do you typically begin your / commute? / If you choose to, please also p...-Zip q17From what city and zip code do you typically begin your / commute? / If you choose to, please also p...-Cross street 1 q18From what city and zip code do you typically begin your / commute? / If you choose to, please also p...-Cross street 2 q19How do you usually commute to campus/work? (i.e., / What is the primary mode you use during your ty... q20How do you usually commute to campus/work? (i.e., / What is the primary mode you use during your ty...-TEXT q21Which transit system do you use for the longest distance of your / commute? q22Which transit system do you use for the longest distance of your / commute?-TEXT q23How do you usually get to the bus stop or train station from / your home? q24How do you usually get to the bus stop or train station from / your home?-TEXT q25

Step 1: extract the schema

Step 2: make JSON

{     "4":"A1",   "5":"No",   "6":"No",   "7":31.2345,   "8":-120.1234,   "10":"None of the above",   "11":1,   "Zip":90210,   "City":"Santa Claus",   "State":"CA",   "Staff_Class":"A1"}

Step 3: populate database

Step 4: set up database with more columnsalter table survey_data alter column raw_survey_data type jsonb using raw_survey_data::jsonb, add column survey_date date, add column response_id text, add column affiliation text, add column staff_class_level text, add column work_location_description text, add column geom_work geometry, add column commute_start_city text, add column commute_start_zip text, add column commute_mode_primary text, add column commute_mode_transit text, add column commute_mode_transit_access_home text, add column commute_mode_transit_access_work text, add column typical_station_home text, add column typical_station_work text, add column commute_regularity text, add column commute_mode_primary_regularity text, add column typical_work_time_arrive text, add column typical_work_time_depart text

Step 5: populate data columnsupdate survey_dataset survey_date = '2016-01-05', response_id = raw_survey_data ->> '1', external_data_reference = (raw_survey_data ->> '2')::integer, affiliation = raw_survey_data ->> '3', staff_class_level = raw_survey_data ->> '4', work_location_description = raw_survey_data ->> '10', geom_work = null, live_on_campus = (raw_survey_data ->> '16')::boolean, commute_start_city = raw_survey_data ->> '17', commute_start_zip = raw_survey_data ->> '18', commute_mode_primary = case when

upper(raw_survey_data ->> '21') not like 'OTHER%'

then raw_survey_data ->> '21'

else concat(raw_survey_data ->> '21', ' ', raw_survey_data ->> '22')

end

Cleaning up the non-geographic data

A lot of questions are not fully expressed in column headers Schema loaded separately into database

Several column headers are repeated

Not all questions are answeredPostgreSQL’s JSON capabilities used to reduce data volumes while still retaining

raw data

Some responses are “Other”, with the response in a second column Questions automatically and manually

assessed to ensure data integritySome questions allow multiple responses, so responses are collected into multiple

columns

A lot of questions are not of interest for this study Some questions are ignored for this study

The story so far…

Transport Scotland: http://www.transport.gov.scot/report/j9425-10.htm

Two questions were very creatively posed: where do you work on campus, and where do you park?

Where do you work on campus?

Linford Nursing: http://www.linfield.edu/portland/about-portland/location/campus-map.html

580 194

502 270

41,616,668,701,171,800 3,360,666,809,082,030504,5 3,298,699,951,171,870563 250

4,951,040,344,238,280 3,744,800,109,863,280

486,5 3,238,699,951,171,870507 3,296,800,231,933,590

487,5 3,748,699,951,171,870463,5 2,688,699,951,171,870

414,5 2,808,699,951,171,870

584 19646,261,907,958,984,300 2,562,952,575,683,590

Step 1: create links table for main map

Linford Nursing: http://www.linfield.edu/portland/about-portland/location/campus-map.html, map data © 2016 Google

Step 2: create links table for inset map

Linford Nursing: http://www.linfield.edu/portland/about-portland/location/campus-map.html

225.22 -86.66 -14960212.79 4051860.95153.83 -205.26 -14960403.95 4051498.46

52.06 -91.52 -14960759.31 4051780.9948.24 -95.43 -14960771.54 4051771.3558.99 -97.74 -14960735.30 4051764.5653.99 -101.68 -14960749.27 4051750.6391.17 -61.12 -14960584.69 4051890.29

151.37 -213.02 -14960403.08 4051498.46153.84 -205.21 -14960400.46 4051508.46167.14 -152.50 -14960349.38 4051670.98166.24 -158.44 -14960368.15 4051650.62170.70 -146.17 -14960338.90 4051692.41181.56 -158.92 -14960314.89 4051669.55179.37 -161.54 -14960324.50 4051649.55211.50 -94.75 -14960229.33 4051838.50256.58 -53.58 -14960082.20 4051964.59250.41 -46.34 -14960093.99 4051972.45262.27 -48.50 -14960078.71 4051970.30254.56 -42.21 -14960088.32 4051977.80369.27 -29.80 -14959562.69 4052246.41367.87 -17.80 -14959562.69 4052256.77362.14 -24.05 -14959567.93 4052250.70373.59 -22.93 -14959557.45 4052252.48

36.34 -247.21 -14960790.33 4051387.2451.50 -209.42 -14960743.18 4051481.06

Step 3: create a mask

Linford Nursing: http://www.linfield.edu/portland/about-portland/location/campus-map.html

Main map

Inset map

Step 4: fun with FME

Step 5: fun with QGIS

Linford Nursing: http://www.linfield.edu/portland/about-portland/location/campus-map.html

InteractiveDashboard

Thank you!

roland.martin@arup.com