Embed Size (px)
Citation preview
testing the effectiveness of digital storytelling
Peter Spyns – Flemish dept. of Economy, Science and Innovation
Flemish government | Department of Economy, Science and Innovation2
Agenda
initial problem on introducing digital storytelling for funding agencies
organising an experiment on digital story telling in a call for project funding aim policy context set-up outcomes comments discussion
outlook
conclusions
Flemish government | Department of Economy, Science and Innovation3
initial situation
con’s of digital storytelling potential additional threshold
time technical set up adequate content good scenario, style
favours already “strong”/experienced submitters
pro’s one image is worth a 1000 words (Frederick R. Barnard in 1921) not all submitters are excellent writers for some themes or topics, words fall short
problem: how to convince external funding/facilitating agencies/organisations that digital storytelling has an added value for them and their stakeholders and proposal submitters ?
Flemish government | Department of Economy, Science and Innovation4
aim of the experiment
organise a “real life” context to apply digital storytelling
find out how popular digital storytelling already is
investigate in a serious way the effect of digital storytelling
collect evidence-based arguments to convince funding organisations
Flemish government | Department of Economy, Science and Innovation5
EWI call for proposals on social innovation what: definition of social innovation
SI aims in a structural way at meeting a societal need, such as tackling multidimensional societal problems of the most vulnerable groups in society;
SI is about a new or significantly improved product, service, process, marketing method or organisational model.
adequate topic how: call document excerpt (translated):OPTIONAL:Submitters are allowed to add, as an extra support to their proposal, a (digital) animation, video or self referencing presentation. By doing so, submitters can visualise better some aspects of their proposal (e.g., innovative features, way of putting to use the results aimed at, etc.). Submitters who want to make advantage of this possibility are invited to contact [email protected] to arrange to file transfert that has to be done before the deadline.
context of the experiment
Flemish government | Department of Economy, Science and Innovation6
1) collecting the proposals and – if any – the associated digital story
2) distributing the proposal amongst the reviewers without the digital story
3) regular reviewing process (without the digital story)
4) uploading digital stories to separate (and protected) webpage
5) after the regular review, each expert with a proposal supported by a digital story is invited to also have a look at the digital story and send an e-mail with comments about
1) whether or not the digital story adds new information
2) whether or not this new information makes him/her change his/her opinion on one or more of the evaluation criteria
3) whether on not this leads to a change in score for one or more of the evaluation criteria
4) if so, please give a short explanation why
6) a statistical test can determine whether or not a change in expert behaviour is significant or not
practical set-up of the experiment
Flemish government | Department of Economy, Science and Innovation7
1) on 105 eligible proposals, 11 added a digital story (10,5%)
2) leading theoretically to 23 reviews by 19 different reviewers
3) 3 proposals with a digital story were accepted (vs. 17 without)
4) in practice, 4 reviewers responded with 6 reviews
example: Blue Assist movie (a best practice of FP7 INNOSERV project)
outcomes of the experiment
Flemish government | Department of Economy, Science and Innovation8
1) presentation made more clear what I guessed but did not privde real answer and hence, did not lead to a change in score or opinion
2) movie confirmed my initial negative opinion, but did not lead to a change in score or opinion
3) the URL for the movie was in the proposal so I already had a look at it: it contributed positively to my scores (2x)
4) proposal was well written and contained photos so the movie did not really add but rather confirmed my opinion – hence, no change in score or opinion
5) movie merely confirmed my initial opinion, hence no change in score
comments by the participating reviewers
Flemish government | Department of Economy, Science and Innovation9
discussion on the experiment
1) too few responses for a statistical analysis mostly due to1) lack of time (regular review was delayed)
2) bad timing (too close to holidays)
3) digital story already taken into account during original review
2) only 10% of the submitters added a digital story1) could depend on the nature and topic of the proposal
2) for many submitters it was their first proposal
3) in most cases, the digital story merely confirmed the initial impression of the reviewer
4) no change of mind and scores
5) when looked at during regular reviewing, it was considered as a positive source of additional information
Flemish government | Department of Economy, Science and Innovation10
outlook and subsequent steps
participation of “the Flemish Social Innovation Factory” has to stimulate social entrepreneurship and social innovation in
Flanders by a.o. capturing ideas for project proposals supporting submitters in elaborating their ideas into project proposals enriching the project proposal to enhance its acceptance potential guiding social entrepreneurs to setting up a “social entreprise”
acts as a thematic front-end for the Flemish Innovation Agency (IWT) that formally approves and funds the submitted proposals
have discovered “themselves” digital storytelling are interested in becoming involved in InnoFun and trying out digital
story telling as part of their regular way of working in the long term, that could imply that the IWT accepts a digital story as
(additional) part of a proposal
we hope to integrate them in the RIP
Flemish government | Department of Economy, Science and Innovation11
conclusions
1) scientifically inconclusive experiment
2) more experiments are needed – maybe in a different setting
3) low participation (submitters and reviewers)
4) effectiveness of digital story telling is questionable
5) it is no source/driver of change in opinion or score
6) at best, it is an extra source to confirm an opinion in the making
7) the “social innovation factory” could be a promising route for adopting digital storytelling as a regular part of a project proposal preparation procedure.
Thank youDepartment of Economy, Science and Innovation (EWI)
Koning Albert II-laan 35 box 10, 1030 Brusselswww.ewi-vlaanderen.be | [email protected]
