Cesar working document 7 urban strategy experiment 5

CESAR WORKING DOCUMENT SERIES Working document no.7

Different process strategies for PSS Influence of process strategies on added value of Urban Strategy (trial No5)

M. te Brömmelstroet

01 August 2013

This working document series is a joint initiative of the University of Amsterdam, Utrecht University, Wageningen University and

Research centre and TNO

The research that is presented in this series is financed by the NWO program on Sustainable Accessibility of the Randstad:

http://dbr.verdus.nl/pagina.asp?id=750

CESAR Working Document Series no. 7 Different process strategies for PSS

Page 2

TABLE OF CONTENT

1. INTRODUCTION ................................................................................................................. 3

2. SETUP OF THE EXPERIMENT ............................................................................................. 4

2.1 Intervention: Urban Strategy PSS ......................................................................................... 4 2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap? ................... 4 2.3 Setup of the experiment ....................................................................................................... 5 Student groups ............................................................................................................................. 6 Control and treatments ................................................................................................................ 7 2.4 Data gathering and analysis .................................................................................................. 7

3. EVALUATION RESULTS ...................................................................................................... 9

3.1 Perceived quality of the planning process ............................................................................ 9 3.2 Perceived quality of the planning outcome .......................................................................... 9 3.3 Effect of different backgrounds .......................................................................................... 10 3.4 Sensitivity of roles for treatments ...................................................................................... 11

4. CONCLUSIONS AND IMPLICATIONS ................................................................................ 14

4.1 Reflections .......................................................................................................................... 14 4.2 General conclusions ............................................................................................................ 15

REFERENCES ........................................................................................................................... 16

APPENDIX I: COMPETITION SHEET (DUTCH) .......................................................................... 17

APPENDIX II: EVALUATION FORM FOR QUALITY OF THE PROCES ......................................... 19

................................................................................................................................................ 19

APPENDIX III: EVALUATION FORM FOR QUALITY OF THE OUTCOME ................................... 21

APPENDIX IV: EVALUATION FORM FOR USABILITY OF URBAN STRATEGY ............................ 22


Page 3

1. INTRODUCTION

In this working document we report on a fifth randomized controlled trial with Urban Strategy to test the added value of this Planning Support System (PSS) in strategy making practices. Urban Strategy is a software package developed by TNO that aims to improve the planning process and planning outcomes of strategic planning. It does so by offering a range of quick models that show the effects of planning interventions in an easy to understand visual environment. To gain more insight into these potential improvements, we have conducted an experiment with a group of master students in Urban Planning of the University of Amsterdam. In this fifth experiment, we were especially interested in how different process structuring would influence the usability of PSS and the general performance of strategy making groups. We make use of the conceptual- and measuring frameworks as introduced and discussed in CESAR Working Document No. 1. First, we describe the setup of the experiment and the different treatments that we have compared (section 2). Then, the findings of the experiment are presented (section 3). In the fourth section we will briefly discuss the implications of these findings and further research.


Page 4

2. SETUP OF THE EXPERIMENT

2.1 Intervention: Urban Strategy PSS

TNO started around 2005 with the development of a PSS – Urban Strategy (Borst et al. 2007; 2009a; 2009b) – specifically aiming to bridge the existing flexibility- and communication bottlenecks. Urban Strategy aims to improve complex spatial planning processes on the urban- and regional level. To do this, different computer models are linked to a central database and interface to provide insights in a wide area of urban indicators and maps. The effects of interventions in infrastructure, land use, build objects and their functions can be calculated and visualized. Because the PSS is able to calculate fast and present the results in an attractive 1D, 2D and 3D visualisation this can be used in interactive sessions with planning actors. Starting point for Urban Strategy is the use of existing state-of-the-art and legally accepted models. To link these existing models a number of new elements were developed: - a database with an uniform datamodel; - interfaces that show a 3D image of the modeled situation, indicators and that offer functionality

to add interventions; - a framework that structures the communication between the models and the interfaces.

2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap?

The goal of Urban Strategy is to enable planning actors in workshop sessions to communicate their ideas and strategies to the PSS and to learn from the effects that are shown. This interactivity calls for fast calculations of all the model and fast communication between all elements. For this, the models were enabled to respond on events (urban interventions from the participants in the workshops. A new software architecture was developed to have all these elements communicate (figure 2). Through this increases speed and the wide variety of models that are linked together, the PSS aims to be highly flexible in offering answers to a large number of questions that a group of urban planning actors can have.

Figure 2 Schematic overview of communication architecture of Urban Strategy.

The 3D interface generates, based on objects in the database, a 3D digital maquette of the urban environment. To this, different information layers can be addes, such as air quality contours, noise contours and groundwater levels. Also, the objects can be colored according to their characteristics (function, energy use, CO2 emissions, number of inhabitants, etc). The 2D interface can be used by the end user (or operator) to add changes to the database. Objects can be added or removed, their


Page 5

location can be changed and the characteristics of the object can be changed. The 1D interface shows indicators that are calculated by all the models that are included. Examples are the percentage of noise hindrance, group risk in an area or the contribution of types of objects to CO2 emission.

Figure 3 The three interfaces of Urban Strategy

Urban Strategy as a PSS is not only a computer instrument, but also offers process support for groups of planning participants. It does so by using a Maptable (a surface table screen on which people can interact with the information from the instrument) and by having a mediator organize the exchange of knowledge between participants and between them and the computer instrument. In earlier experiments we have found that especially the process support is an essential element in understanding the added value of an instrument such as Urban Strategy for planning practice. First, adding a mediator already appears to improve the functioning of the group by offering external structure. Next to structuring the interaction between the participants, the process support also structures the interaction between the participants knowledge and the knowledge of Urban Strategy. Doing this well seemed to positively influence the added value of the instrument to the planning process.

2.3 Setup of the experiment

These observations of the importance of the process support led to the setup of this controlled experiment. The research question here is: Does the setup of the process support influence the added value of Urban Strategy for strategy making? We designed our study as a randomized controlled laboratory experiment. With this we aimed to optimize the internal and external validity of our findings. Testing the most general claim of the PSS literature (that it improves planning) calls for a strong focus on the ability to translate our findings to theory. Although we have had special attention to mirror characteristics of urban planning practice as good as possible (see below), this means a sacrifice of ecological validity.


Page 6

Student groups

The experiment was set up as an obligatory part of a first year course of Traffic Engineering of the Windesheim University of Applied Sciences, and as a voluntary part of a second year course of Environmental Engineering of the Saxion University of Applied Sciences and a second year course of Urban planning of that same University. A total of 65 students participated; 9 in the control group, 32 in Treatment A and 24 in Treatment B. These students were randomly divided into groups of twelve. Each group contained 3/4 traffic engineering students, 2/3 environmental engineering students and 6 urban planning students. These last students were randomly divided into 3 project economists and 3 urban designers. With this setup we attempted to create the complexity of real world planning processes, while avoiding the overrepresentation of one single group. Therefore, the urban planning students (that were a large majority) were split up into two characters for they are both educated. Several weeks before the experiment, each student was sent an e-mail in which they were invited to join a TNO-UvA design competition. They received the basic information about the planning area (figure 4), the existing strategy (figure 5) and the setup of the meeting (60 minutes to come up with an improved strategy). Each of the four roles received a role specific target for the competition. Also, they were informed that the competition was within their own role: the best traffic engineering idea would win etc. Next to that, they received role specific information about specific problems that the existing plan had (i.e. the traffic engineering students received road-capacity maps, while the environmental engineering students received noise and external safety boundary maps).

Fig. 4. Brownfield location in the old harbors of Rotterdam

Fig. 5. Original design for the area provided to each group


Page 7

Control and treatments

The resulting seven groups were randomly divided into one control- and six treatment groups. The control groups received no support; they were assigned to a table with empty plans, instructed to start, and informed that the time was up. We used this control group to see if and how the findings of this experiment can be compared with those of earlier studies. There can be some general effects on our indicators that relate to the fact that we work with different types of students, larger groups and/or a slightly changed planning challenge. The six control groups were divided into three groups that received treatment A and three groups that received treatment B. Both of them were using the same physical setup in the same room (figure 6): Treatment A: Full support by Urban Strategy including the use of the surface table. All plans and

ideas are put into Urban Strategy and their effects are translated back to the group. Process is structured by one mediator into a sequence of brainstorm, analysis, feedback and re-design.

Treatment B: Full support by Urban Strategy including the use of the surface table. All plans and ideas are put into Urban Strategy and their effects are translated back to the group. Process is structured by one mediator into a sequence of personal thinking, , analysis, feedback, role based re-design.

Fig. 6. Physical setup for groups: Table with maps and whiteboard to develop ideas (left) and interaction with the

calculated effects through a surface table with a chauffeur (right)

2.4 Data gathering and analysis To find out if there are any systematic differences in the performance of the control- and treatment groups we have made use of several data gathering techniques. For this we used the framework for quality of planning as presented on the 2013 CUPUM conference. See table 1 for the dimensions and subdimensions that were measured.


Page 8

Table 1. Dimensions of the quality of planning

First, two external planning experts (PhD candidates in Urban Planning of the University of Amsterdam) rated the general quality of the resulting strategies. For each strategy, they were asked to respond to statements on the dimensions A to F of table 1 (on a 7 point Likert scale). They were not informed of the hypothesis, nor were they aware of which strategies came from control- or treatment groups. Secondly, all participants filled in an evaluation form in which we solicited their personal perceptions of the quality of the planning process. They responded to statements relating to the dimensions J-W of table 1. And thirdly, we have used direct observation. These observations were mainly used to understand the outcomes of the first three analytical instruments. For the analysis, the responses on the statements were averaged and then compared and tested for systematic differences. To indicate the strength of the differences in effects, we used the p-value of a ANOVA F-test to compare two independent means (<0,05 is considered statistical significant). The statements were grouped for the subdimensions and overall dimensions by averaging them. The two statements on conflict were first inverted to make them compatible with this process. For the outcome dimensions, the scores of the raters were also averaged and then processed in the same way. Finally, we asked all participants in the treatment group to rate Urban Strategy on a number of usability indicators, using a 7-point Likert scale.


Page 9

3. EVALUATION RESULTS

3.1 Perceived quality of the planning process

All findings are represented in figure 7. On the x-axis, all process dimensions are represented (the grouped dimensions are capitalized). On the y-axis, the score on a seven point Likert Scale is shown. The four categories of which the average scores are shown are the control groups of our former experiment (see CESAR Working Document Series no.6), the control group of this experiments, treatment A and treatment B. The outcomes show several trends. First, the control group show a number of negative impacts compared to experiment 4. This indicates that we have simulated planning practice more realistic, since the scores in experiment 4 were very high. We see these drops most notably in Shared Language and Cohesion and to a lesser extent in Credibility, Efficiency and Consensus. Second, we see that Treatment B scores higher on almost all dimensions (except Efficiency). None of these differences are significant. For the majority of dimensions, Treatment A scores lower than the control group.

Fig. 7. Scores for control and treatment groups on all process dimensions on 7-point Likert scale (composite dimensions

are capitalized)

3.2 Perceived quality of the planning outcome

Figure 8 has a similar setup as figure 7. By adding different backgrounds to the groups, we expected that also the quality of the strategies would suffer. This is indicated by the drops in scores for the control groups, especially on Acceptability, Implicational Explicitness and Clarity. Implementability however increased. In these scores, we find that Treatment A scores notably better than Treatment B on all Specificity subdimensions. The other dimensions show very little difference and do not indicate a clear direction. The total score for Treatment B is higher than for Treatment A.


Page 10

Fig. 8. Scores for control and treatment groups on all content dimensions on 7-point Likert scale (composite dimensions

are capitalized)

3.3 Effect of different backgrounds

In the experiment, participants from different educational backgrounds participated. If we look at the responses for each of these groups separately (figure 9), we see that on most dimensions the environmental engineering students were most positive on the quality of the process. The transport engineering students are also mostly on above average, while both roles played by the urban planning students scored lower. The plan economists were in general quite dissatisfied with the impact of their role on the process and outcomes. They often used the open question to ventilate their frustration with this.

Fig.9 Scores on process dimensions for the different educational backgrounds on 7-point Likert scale (composite

dimensions are capitalized)

The participants of Treatment 1 and Treatment 2 groups were also asked for their opinion on the usability of Urban Strategy. Again, we have looked at their scores per role (figure 10). We find again that the environmental- and transport engineering students are mostly more satisfied with the usability characteristics of Urban Strategy.


Page 11

Fig.10 Scores on usability indicators for Urban Strategy for the different educational backgrounds (please note that

‘Creativiteit beperkend’ is an inverted dimension)

3.4 Sensitivity of roles for treatments

Next to the general responses on the usability characteristics of Urban Strategy, our setup allows us to see the sensitivity to the treatments controlled for the four different backgrounds. First, we have done this analysis for perceived process quality. Figure 11 scores the difference between Treatment A and Treatment B on the composite dimensions of process quality. A positive number means that on average the respondents score higher under Treatment B. An asterix (*) means that the difference is statistical significant (P<0,05).

Fig. 11 The different sensitivities to the treatments for the four educational backgrounds on the composite process

quality dimensions


Page 12

The data shows that the environmental engineering students and to a lesser extent the traffic engineering students are more positive when they received Treatment B. This is seen on almost all dimensions. The Urban Planning students were much less sensitive to the difference in treatments. The Urban Designers show small positive effects of Treatment A on most dimensions (especially on efficiency) while the Plan Economists show a more mixed image. The same has be done for the perceived usability characteristics of Urban Strategy. The differences between Treatment B and A is presented in figure 12.

Fig. 12 The different sensitivities to the treatments for the four educational backgrounds on the perceived usability

characteristics of Urban Strategy (Treatment B – Treatment A)


Page 13

Most of the effects are small and not statistically significant. The once that are significant are marked with an asterix in figure 12. These however do not present a clear trend. As with the trends on process quality dimensions it is notable that again both engineering student groups are positively influenced by Treatment B.

4. CONCLUSIONS AND IMPLICATIONS

4.1 Reflections

There are several important reflections to make on the methodology that have serious implications for the results and their generalizability. The laboratory setup with students allows for strong control over the treatment and control groups, but it also simplifies the complexity of daily planning practice on some important aspects (which is the target application of Urban Strategy). The most important ones are listed below. Students: In the context of planning practice, the student population of this experiment can be considered as being empty vessels. They do not have much tacit knowledge about the issues in the Waalhaven case; neither do they have vested interests or accountability for it. This makes the experiment relatively insensitive for some important effects that can be expected in real planning practice, such as developing a shared language or gaining consensus. Related to that, we would also expect to see more effects on the quality of the planning outcome in such a practical environment. Group size: The group size was planned to be considerably larger than in the former experiments (with up to 12 participants per group). However, many students (especially from the urban planning group) were not showing up. This didn’t only mean that the groups became much less populated, but also that the internal characteristics of each group differed. This makes it quite hard to distinguish between the effect this has and the effect that the treatments have. First timers: A large majority of the students stated in the open question that they enjoyed the strategy making exercise, since they never did this before. This is mirrored in the relatively high scores on the process dimensions (such as satisfaction) by the students in the control group. Again, this makes the experiment less sensitive for process effects that would be expected in planning practice. Here, the participants would have much more (negative and positive) experiences with similar processes and are better able to assess the added value of Urban Strategy. Also, they would be able to score the instrument characteristics (table 3) more in relation to other instruments. Related to that, we cannot assume that the control group worked as ‘business as usual’. Raters: Due to time constraints, we worked with two external raters from the University of Amsterdam. For them it was difficult to assess some of the quality dimensions of the planning outcome. This affected both their judgment of the control and treatment planning outcomes, which makes its implications less relevant. They both noted that the outcomes were poorly written and some lacked a legend. This sometimes made rating difficult. Reliability: To ensure that the statements are measuring the elements that we aim to measure, we have analyzed the reliability of the composite scores for all dimensions that were measured by more than one statement. When Cronbach’s Alpha is higher than 0,7, the variance on the responses on separate statements can be considered to be comparable. The results of this are shown in table 2.


Page 15

Table 2 Outcomes on reliability of composite scores for dimensions of process- and outcome quality

Cronbachs Alpha Cronbachs Alpha

REACTION 0,904 NOVELTY 0,956

enthusiasm 0,759 original 0,924

satisfaction 0,806 paradigm relatedness 0,815

credibility 0,668 WORKABILITY* 0,643

INSIGHT 0,823 implementability

insight in problem* 0,563 acceptability 0,794

insight in assumptions 0,818 RELEVANCE 0,879

COMMITMENT applicability

COMMUNICATION effectiveness 0,908

SHAREDLANGUAGE* 0,418 SPECIFICITY 0,909

CONSENSUS 0,733 completeness 0,786

consensus on problem* 0,528 implicational explicitness

consensus on goals clarity 0,812

consensus on strategies* 0,504 OUTCOM QUALITY 0,941

COHESION* 0,428

EFFICIENCY

PROCESS QUALITY 0,932

The composite scores that have a low reliability are marked with an asterix.

4.2 General conclusions

With these methodological limitations in mind, the experiment has offered us some interesting insights how different process strategies influence the added value of a PSS such as Urban Strategy. As in earlier experiments the classical usability characteristics of the instrument itself are highly rated. Overall, participants are enthusiastic about the instrument and often surprised about its speed and ease of use. They also noted some missing functionality, mainly the lack of other modes than car, more information retrieval on demand (i.e. on nr of inhabitants etc) and the lack of a way to overlap different maps on the surface table. Working with students that have real differences in their educational backgrounds allowed us to simulate planning practice much closer than in earlier experiments. This lead to insights in that especially the engineering students were sensitive to the way the process was organized. They perceived both the quality of the process as usability of the instrument higher when they received Treatment B. It seems that the individual thinking element that was offered allowed the participants to develop their own ideas better and both engineers were than more satisfied with how the instrument provided them with feedback on their ideas. For the urban planning students, this was much less visible. It even seemed that the once that were given the role of urban designers were more satisfied with the group brainstorm of Treatment A. It is however important to realize that we found almost no significant proof for these insights. They are mainly based on looking at the trends and observing the experiment itself.


Page 16

REFERENCES

Al, J., & van Tilburg, W. (2005). Basisboek instrumenten regionale bereikbaarheid. Rotterdam: Rijkswaterstaat AVV.

Couclelis, H. (2005). “Where has the future gone?” Rethinking the role of integrated land-use models in spatial planning. Environment and Planning A, 37, 1353-1371.

Dean, D. L., Hender, J. M., Rodgers, T. L., & Santanen, E. L. (2006). Identifying Quality, Novel, and Creative Ideas:Constructs and Scales for Idea Evaluation. Journal of the Association for Information Systems, 7, 646-699.

Friedmann, J. (1987). Planning in the public domain: From knowledge to action. Princeton: Princeton University Press.

Klosterman, R. (1999). The What If? collaborative planning support system. Environment and Planning A, 26, 393-408.

Lee, D. B. (1973). Requiem for large-scale models. Journal of the American Planning Association, 39, pp. 163-178.

Lee, D. B. (1994). Retrospective on large-scale urban models. Journal of the American Planning Association, 60, 35-40.

Rittel, H., & Webber, M. (1984). Dilemmas in a general theory of planning. In N. Cross (Ed.), Developments in design methodology pp. 135-144). Chicester: John Wiley and Sons.

Rouwette, E. A. J. A., Vennix, J. A. M., & Van Mullekom, T. (2002). Group model building effectiveness: a review of assessment studies. System Dynamics Review, 18, 5-45.

Te Brömmelstroet, M. (2010). Equip the warrior instead of manning the equipment: Land use and transport planning support in the Netherlands. Journal of Transport and Land Use, 3, 25-41.

Te Brömmelstroet, M. (2011). What do we support and how (well) do we do it? A multidimensional framework to measure the effectiveness of Planning Support Systems. Environment and Planning B: Planning and Design, submitted.

Te Brömmelstroet, M., & Schrijnen, P. M. (2010). From Planning Support Systems to Mediated Planning Support: A structured dialogue to overcome the implementation gap. Environment and Planning B: Planning and Design, 37, 3-20.

Vonk, G. (2006). Improving planning support; The use of planning support systems for spatial planning. Utrecht: Nederlandse Geografische Studies.

Vonk, G., & Ligtenberg, A. (2009). Socio-technical PSS development to improve functionality and usability—Sketch planning using a Maptable. Landscape and Urban Planning, doi:10.1016/j.landurbplan.2009.10.001

Waddell, P. (2002). UrbanSim: Modeling Urban Development for Land Use, Transportation and Environmental Planning. Journal of the American Planning Association, 68, 297-314.

Waddell, P. (2011). Integrated Land Use and Transportation Planning and Modelling: Addressing Challenges in Research and Practice. Transport Reviews, 31, 209-229.


Page 17

APPENDIX I: COMPETITION SHEET (DUTCH)

Introductie De Rotterdamse haven groeit nog steeds, maar deze groei speelt zich steeds meer ten Westen van de stad af. Omdat de havenindustrie steeds grootschaliger wordt, verschuiven havenfuncties naar nieuwe gebieden zoals de Tweede Maasvlakte. De hierdoor vrijkomende gebieden zijn aantrekkelijke locaties voor stedelijke transformatie: ze liggen relatief dicht bij de binnenstad en bieden in combinatie met het water interessante mogelijkheden voor wonen, werken en recreatie. Naast deze mogelijkheden zijn er uiteraard ook grote uitdagingen, zoals de aanwezigheid van industrie en afwezigheid van stedelijke infrastructuur. Eén van de gebieden die momenteel voor zo’n opgave staat is Waalhavens (zie kaartjes hieronder).

Figuur 1: Projectgebied Waalhavens in haar stedelijke context


Page 18

Figuur 2: Stedenbouwkundig plan (Oranje = wonen , blauw = werken, rood = horeca).

De gemeente Rotterdam heeft in samenwerking met het Havenbedrijf Rotterdam ambitieuze plannen geformuleerd voor dit gebied dat voornamelijk bestaat uit twee oude havenarmen. Het gebied moet een toplocatie worden voor toonaangevende kennisbedrijven en een aantrekkelijke woonkwaliteit bieden voor hoogopgeleide tweeverdieners. Daarnaast moet het bijdragen aan de duurzaamheid van de stad én moet het voldoen aan eisen aan de milieukwaliteit. Het huidige stedenbouwkundige plan (zie figuur 2) scoort nog mager op ruimtelijke kwaliteit. Om ervoor te zorgen dat het voor bedrijven en hoogopgeleide tweeverdieners een interessante plek wordt is het aan jullie interdisciplinaire planningsteam om dit sterk te verbeteren! In deze fase van het planproces staan de vorm en functie van de gebouwen al vast. Je kunt dus vooral spelen met de oriëntatie, locatie en de openbare ruimte. Het gaat uiteindelijk om de gebiedsschets en geschreven strategie. Opdracht

In 60 minuten met je team tot een verbeterd stedenbouwkundig plan komen (dit lever je in met 1 kaart plus tekst). Organiseer jezelf. De ruimtelijke kwaliteit van het plan (kaart plus tekst) wordt beoordeeld op de aantrekkelijkheid van het plangebied. Maar ook op de creativiteit van de oplossingen. De ruimtelijke planner die hier het best op scoort krijgt een TNO prijzenpakket.


Page 19

APPENDIX II: EVALUATION FORM FOR QUALITY OF THE PROCES

Uw groepsnummer:

Uw studentnummer:

Wat was uw rol/achtergrond?

NVT

1 2 3 4 5 6 7

Ik heb er vertrouwen in dat de uitkomst goed is

Ik ondersteun de meeste resultaten van de sessie

De sessie heeft geleid tot nieuwe inzichten

Ik begrijp nu de voorgestelde oplossingen van de andere

deelnemers beter

Het resultaat van de sessie is gebaseerd op correcte

aannames over het stedelijk systeem

De sessie was succesvol

Het proces heeft me inzicht gegeven in de meningen en

ideeen van anderen over het probleem

Ik heb een goed gevoel over de sessie

Ik heb nu meer inzicht in de processen die een rol spelen

in het probleem

Ik zal inzichten uit de sessie gaan gebruiken in mijn

dagelijkse praktijk

Ik begrijp nu hoe andere deelnemers het probleem zien

Het resultaat biedt een echte oplossing voor het

probleem

Het is duidelijk voor mij wat de oorzaken van het

probleem zijn

De andere deelnemers begrijpen beter hoe ik het

probleem zie

Ik ben tevreden met de sessie

De sessie heeft geresulteerd in bruikbare resultaten

Mijn inzicht in het probleem is vergroot

Mijn begrip van de meningen van andere deelnemers

over het probleem is toegenomen

We hebben tijdens de sessie een gemeenschappelijke

professionele taal ontwikkeld

De sessie heeft mijn inzicht in de relatie tussen de

verschillende elementen van het probleem vergroot

De andere deelnemers zijn tevreden met de sessie

Zeer mee

oneens

Zeer mee

eens

RO (stedenbouwkundig) RO (planeconomie)

Milieukundige Verkeerskundige


Page 20

NVT

1 2 3 4 5 6 7

Ik was goed in staat mijn mijn rol uit te voeren

We hebben een gedeelde visie over strategische doelen

bereikt

Er was sprake van conflict tijdens de sessie

We waren in staat consensus over het probleem te

ontwikkelen

We hebben een gedeelde visie over mogelijke

oplossingen bereikt

Er was conflict over de uit te voeren taak

We hebben een gemeenschappelijke visie over het

probleem bereikt

We hebben de tijd efficiënt benut

Tijdens de sessie ontstond een platform die het delen

van ideeen ondersteunde

De sessie bracht me dichter bij de andere deelnemers

Ik heb vanuit mijn rol interventies kunnen inbrengen in

het plan

De onderwerpen die vanuit mijn rol van belang zijn, zijn

tijdens de sessie goed naar voren gekomen

De resultaten zijn een integratie van diverse meningen en

ideeen van de deelnemers

Ik had een sterk groepsgevoel tijdens de sessie

Heeft u nog andere opmerkingen over de sessie?

Zeer mee

oneens

Zeer mee

eens


Page 21

APPENDIX III: EVALUATION FORM FOR QUALITY OF THE OUTCOME

Geef a.u.b. het nummer van de strategie aan:

NVT

1 2 3 4 5 6 7

De strategie is ingenieus

De strategie is fantasierijk

De strategie is verrassend

De strategie is vernieuwend

De strategie is radicaal

De strategie heeft een transformerende potentie

De strategie is eenvoudig te implementeren

De strategie is sociaal acceptabel

De strategie is juridisch acceptabel

De strategie is politiek acceptabel

De strategie heeft een duidelijk verband met het probleem

De strategie zal het probleem oplossen

Dit is een effectieve strategie

De strategie kan worden opgedeeld in verschillende componenten

De strategie gaat in op "wie"

De strategie gaat in op "wat"

De strategie gaat in op "waar"

De strategie gaat in op "wanneer"

De strategie gaat in op "waarom"

De strategie gaat in op "hoe"

Er is een duidelijke relatie tussen voorgestelde acties & verwachte uitkomsten

de strategie wordt duidelijk gecommuniceerd

De strategie is makkelijk te begrijpen

De strategie sluit aan op stedelijke dynamiek

Heeft u nog verdere opmerkingen over deze strategie?

Zeer mee

oneens

Zeer mee

eens


Page 22

APPENDIX IV: EVALUATION FORM FOR USABILITY OF URBAN STRATEGY

Jullie groepsnummer

Je Studentennummer

Wat was uw rol/achtergrond? RO (stedenbouw)

RO (planeconomie)

Milieukundige

Verkeerskundige

Welke themas hebben jullie vooral gebruikt Luchtkwaliteit

in Urban Strategy? (meerdere mogelijk) Externe veiligheid

Geluid

Stedenbouw

Verkeer

Planeconoom

Anders. nl.;

N.V.T

1 2 3 4 5 6 7

Urban Strategy is transparant

De communicatieve waarde van de output is hoog

De output wordt duidelijk weergegeven

Urban Strategy is gebruiksvriendelijk

De output is geloofwaardig

Urban Strategy is uitgebreid genoeg

De focus van Urban Strategy is voldoende

Het detailniveau van de kaarten is voldoende

Urban Strategy is makkelijk te begrijpen

Urban Strategy sluit goed aan bij de rol die ik vervulde

Urban Strategy faciliteerde het evalueren van

alternatieven goed

Urban Strategy faciliteerde het creëren van ideeën goed

Urban Strategyondersteunde het schetsen van ideeën

goed

Door het gebruik van Urban Strategy werden zin en onzin

van elkaar gescheiden

Door het gebruik van Urban Strategy werd onze

creativiteit beperkt

Ik begrijp wat er (niet) wordt meegenomen in de

indicatoren

Door Urban Strategy waren we in staat om meer te doen

in minder tijd

Zeer mee

oneens

Zeer mee

eens

Heeft U nog andere opmerkingen over Urban Strategy?

Technology

Cesar working document 7 urban strategy experiment 5