Partners for Health Report

Kate Mackenzie Davey and Richard Giordano16 June, 2007

Summary

The Partners for Health programme was established with three main aims. These were1. Partnership: working together to learn, encouraging participatory learning and reflective

practice with the partners2. Grant management: managing funds to ensure the efficient delivery of a service in

three key areas of health3. Evaluation: creating and sharing knowledge for the King’s Fund and wider community in

order to influence policyThe programme is entering its final stages and while we cannot draw final conclusions at this stage there are some preliminary indications of the success and limitations of aspects of the programme that can be shared and considered in planning the future developments.

The programme has been effective in building close Partnerships with the funded organizations. Grant management has worked well through these Partnerships. However, while in many cases organizations have reflected on how they can make a difference to the service users and have appreciated the opportunity to consider the barriers to success in a supportive environment, the knowledge creation and evaluation has been more complex and difficult to achieve that anticipated.

The successes of the project appear to be due to the attempts by the Partner’s for Health team to make aims and decision processes explicit, the openness of the interactions in grant management and the close challenging, but supportive relationships between individual members of the team and the Partners. The difficulties have been due to contradictions between the different demands of the programme, resulting in a number of paradoxical demands and conflicts over use of limited resources, especially service delivery versus critical reflection.

Context

The aim of Partners for Health was to change the contract between funders and funded in two ways. First, the programme aimed to build a supportive relationship between funder and funded partner, encouraging individual learning and so increasing the funded organization’s capacity for development. This approach drew on previous work within the King’s Fund on management development and learning. Secondly, the programme aimed to move from standard measures of efficiency of funded partners, to working with them to evaluate the effectiveness of their approach. This approach aimed to create and capture knowledge of what did and did not work in practice and to share this knowledge with other funding bodies to build up an evidence base to support practice.-------------------------------------------------------------------------------------------------------------------------------Evidence based practice and practice based evidenceEvidence based practice has become increasingly important for medicine, social care and organizational change. The focus on evidence is driven by greater concern about budgets and a focus on effectiveness rather than efficiency.

There are two broad approaches to knowledge. The first is to focus on the interpretation and use of existing material, either in the form of a meta-analysis of quantitative studies or, more commonly, a review of existing research literature. The second is to focus on creating new knowledge from existing practice through systematic evaluation and action research. These have been contrasted as evidence based practice and practice based evidence (Staler, 2006)

There is a general consensus on the importance of evaluation, whether in social care, medicine or organizational change. However, evidence for sound practice in evaluation and the development of a reliable knowledge base is weak in all areas. -------------------------------------------------------------------------------------------------------------------------------Funders in areas of health and social care see their role primarily as supporting and monitoring the efficiency of the project. The focus tends to be on the implementation rather than the outcome of an intervention. The project is likely to be evaluated on criteria such as efficiency or number of clients seen rather than on more fundamental questions of impact of the intervention.

Partners for Health aims to build on the unique position of funders to nurture a knowledge base from their monitoring and development work with projects. Researchers from Management and Organizational Psychology with experience in knowledge management, organizational change and evaluation were brought in to facilitate this process.

The Partners for Health Programme

Partners for Health is developing an innovative approach to funding projects. The key aims are spelt out in the website.

“ We want to become a more active partner by developing new ways of working across the four health areas identified. Development support, with a special focus on evaluation is a key element of the Partners for Health in London programme.

Our expectation is that applicants will apply to the partners for Health programme because their desire to learn, influence and improve health is as strong as ours.”(Partners for Health website)

The emphasis on being an “active partner” signals fundamental changes to the traditional funder–project relationship. The final paragraph, in bold, specifies the expectations of the desires of the projects. This is not just to help but specifically ‘to learn, influence and improve health’ and reinforces the idea of partnership through the emphasising that these desires should be shared with the funder.

While this appears straightforward, research suggests power relations between a funding and a funded organization may be a real barrier to the creation and, more importantly, the sharing of knowledge. It raised a number of paradoxes for the relationship between funders and the projects and for the identity of the funders. Two key tensions that the team struggled to resolve were, first, the nature of the relationship between the King’s Fund and the projects; and secondly, the nature of, and relations between, evaluation, development, learning and knowledge.

The Partners for Health Team

This section of the report will examine the difficulties faced by the funding team in addressing these aims, the methods used to try and address them and reflections on the effectiveness of these methods.

The team were aware that this was a radical change in traditional funding relationships where the evaluation tended to be in terms of level of practical activity rather than contribution to knowledge. While there was considerable excitement about the project, the team were themselves struggling to make sense of the change in grant management style. It was a change in culture.

There were two key paradoxes confronted by the team. Many of the terms were understood differently by members of the team and the particular foci for the tension were:

on the role of the team in managing contradictions between partnership and evaluation in its judgmental sense

on the definition and ownership of knowledge

There are links between these two issues that centre around the notion of power. The question that was constantly examined was the extent to which the role of members of the Partners for health team was about nurturing, supporting and developing project workers and the extent to which it concerned challenging them to produce persuasive evaluations of their work.

The term evaluation has negative, absolute judgemental associations that contradict the more social and individual aspects of partnership. This conflict is fundamental to many arguments about the nature of learning as disciplined acquisition of information as opposed to individual exploration and development.

Evaluation

The ideas of evaluation were introduced through an evaluation workshop for grant applicants. The aim of the workshop was to make the process by which grants were awarded explicit rather than based on custom and practice. The model of evaluation adopted was Pawson and Tilley’s realistic evaluation. They appeared to avoid the impossibility of carrying out controlled experiments in practice and the relativity of stakeholder based approaches.

The approach was to guide partners to making their theories explicit. When applicants outlined the service they intended to offer they were encouraged to consider how it would work, what contextual variables could be important and what outcomes might be observed to demonstrate that it had been effective. While this initially appeared to be a simple and practical approach to evaluation it proved immensely complex in application. The discussion of context, mechanism and outcome was not helpful for designing evaluation. However, in some cases it led to high levels of reflection about the nature of the planned interventions. These were exciting and testing for the individuals involved but were often lost in the return to work and the delivery of the service.

The difficulties were both in the complexity of evaluation, the novelty of being asked to consider such questions and the threat of questioning a dearly held belief about how best to help those in need. The use of realistic evaluation was also confusing for those who already had some understanding of evaluation, whether from a medical (controlled trial) or a social (stakeholder analysis) approach. Asking why an intervention might be expected to make a difference and how they could tell when it was successful was undermining for those who dedicated themselves to helping others as best they could.

In order to help clarify the mechanisms by which the intervention could work applicants were introduced to a User Pathway. The aim here was to consider how the service would be experienced by a user, what aspects of context may help or impede them and what outcomes they may hope for.

After the first year it became apparent that the language of the realistic evaluation model, as rigorously applied, was unhelpful and it was dropped. The workshops changed and the evaluation plans were developed with individual guidance and support from a member of the team.

The Partners still struggled to make progress with the evaluation. In many cases it was clear that they were attempting to guess what was wanted in order to release the funding and get on with their customary practice. While reflection and challenge was stimulating, it interfered with the daily pressures to deliver a service.

The evaluation plan was reviewed and in an attempt to move away from the disciplinary tones of evaluation was re-christened a learning plan. One difficulty that the partners struggled with was that while the Partners for Health team were interested in testing theory and exploring what did not work, the Partners themselves were more concerned to justify and defend their practice and very much did not want to share any failure either with the funder or with other partners.

Throughout the programme the Partners for Health team met to reflect on their own practice and the development of the project. The evaluation of Partner’s for Health was formative and iterative and involved developing new tools and adapting existing methods.

The nature of the relationship.

Identity: Partner or judge?The role of an ‘active partner’ implies a change from a more disciplinary relationship to one of greater involvement and equality. The change in relationship is reinforced by the emphasis on ‘new ways of working’. However, there are tensions in the levels of equality likely between a funder offering substantial but limited support over three years and the longer term values of projects. The identity of team members was being re-constructed in relation to the partners. These difficulties and re-negotiations of role are associated with change in organization but may be especially salient in the not for profit area. Even more fundamentally, the funder wants to learn from failure while the funded want to exemplify the value of their intervention in order to continue to be seen as a good project.

While there was a powerful espoused ideal of learning and developing knowledge there was also an awareness that the partners most wanted to know what the funder wanted in order to be able to deliver. There was a concern about how far there was a learning focus and how far this was seen as paying lip service to please the funder. As one of the team asked, ” Do they buy into the learning process?” There was a rather rueful recognition of the feedback from one of the partners after the evaluation workshop which was that it was “A fabulous opportunity to learn how a funder thinks”. As one of the team observed, “They say they have a good time, but slip into old habits.” While the Partners for Health team were convinced of the radical change in the funder relationship and the benefits to all parties, the traditional funder relationship is not so easily undermined. Whatever the rhetoric, the team still had the responsibility to approve grant applications, and once approved, to clear payments. The shared interests implied by partnership may not be visible on a day to day basis and may be viewed more cynically by the funded partner (Cobb & Rubin, 2006; Craig & Manthorpe, 1999).

One of the questions the team raised was. “What is our aim? Change in evaluation or partnership with projects?” While clearly giving and with-holding funding are key aims for grant management, the image of Partners for Health made this especially salient. The difficulties of criticising the partners were felt strongly. One concern was that the King’s Fund culture was one of openness and courtesy that fitted the partnership approach and engendered happy and relaxed relationships. Evaluation changed the whole tone and was seen as having “dissipated the energy”. There was a sense of unease about challenging or criticising the partners’ evaluation proposals. The relationship shifted from Partner to “Big bad funder”. No matter how positive a change may appear, for most organisations routines are much less effort to maintain. Achieving a change from monitoring service delivery to evaluation would take a shift in the relationship. Using the frame of partnership to transmit clear goals and values presents people with something of a paradox (Sanyal, 2006)

Over the course of the project, the role of members of the team moved to one of closer collaboration and direction but allowing recognition that the team would not release funds to

projects that were not seen to have considered their evaluation seriously. Balancing relationships between different stakeholders was seen as challenging. While the aims and methods became clearer and the team members more confident, there could be more room for growth in understanding of the dialogue in developing evaluation (Milewa,Dowswell et al, 2002).

Identification with the partnersTeam members voiced their concerns of over-identifying with the partners. This was partly linked to the rhetoric of partnership and support but was also supported by team members reflecting on their own difficulties in adapting to new processes. They recognised the partners’ uncertainties and insecurities they recognised their own struggles with making sense of what the project might look like. So as one team member said “I beat myself up for not being clear”. So while on one side there was an enthusiasm for allowing the space for learning in these projects there was also the recognition that “We can underestimate how difficult it is.”

Change and habitsTeam members found Partners for Health a huge change for both the team and the partners. This raised issues about how to manage as team members were unclear about their roles. They had moved from an expert on grant management to a much more ambiguous and conflicted role as co-researcher. There was a lack of clarity or confidence about role and the processes of change also led to high levels of anxiety. As mentioned, this was often recognised as shared with the partners. “It’s true for us as well. When does it start? Are we good enough?”

There was a further implication for the relationship which was the difficulty in persuading partners of the interest in the problems encountered in the project whether or not they had been overcome. “Harder on the more honest. Don’t want to be too critical.” In the past projects were concerned with impression management to maintain a reputation as efficient and to do as much as possible. The current project we “want to push them to be more thoughtful. They tend to get on with it rather than think what we can learn. They are engaging but what can we learn. “ this means “Trying to persuade them we are paying them to do less and think more.” Partners do not naturally wish to share stories of failure either with funders or with competing projects. While projects that are less successful are usually regarded as contributing more to learning, the reputational risks are high.

Initially there was considerable lack of clarity about the role of team members, the criteria for the projects and the relationship in delivery. Part of the lack of clarity was about the nature of the exercise as one of evaluation for an external stakeholder or as a developmental exercise in reflective learning for the partners’ development as individuals. The main concern felt by the team was the need for consistency. There was a focus on “What are the criteria and what relationship will deliver that.” This demonstrates the core role of interpersonal boundary spanning functions in maintaining and, especially changing interagency relations (Marchington & Vincent, 2004).

Learning, development and knowledge

The aims of the Partners for Health project include both “Development support” with a focus on “evaluation”. While the intention is based on research and evaluation and what can be learned from the projects, the tone of evaluation is disciplinary. This contradiction between development support and the critical and judgmental nature of evaluation is another key tension in the project.

The aims of the project were seen as “Asking questions – conceptual challenges.” “Doing something exciting.” And “Putting learning into the public domain”. However the team also acknowledge the discomfort of change “Feel difference between safe secure knowledge and wisdom to what is bemusing amusing confusing”. While there was an acknowledgement within the team of the “importance of uncertainty” they also recognised that they were “under pressure to revert to what we do”. While themselves under this pressure they also needed to know “How to support people in the discomfort of the liminal space”

In the early enthusiasm for the project it was anticipated that the evaluation process would be a form of participatory action learning offering development for the partners. The aims of the King’s Fund and the partners were seen as identical. As the project developed it became clear that far from being a liberatory development experience, the evaluation was seen as a form of discipline and surveillance of a project by a funder. The less the form of the evaluation was specified the greater the anxiety of the partners. While the project was seeing the lack of specificity as giving the projects greater freedom, the partners themselves found themselves attempting to second guess the requirements of the funder. The complexity of power relations in participatory research are frequently under-estimated.

The initial approach to the project was to introduce partners to realist evaluation (Pawson & Tilley, 1997). “The priority to generate or buy high quality evaluation to inform practice.” Realist evaluation was chosen instrumentally because it avoided both the unrealistic demands of experimental rigour and the relativism of social constructionists, stakeholder evaluations. However, we are not sticking to realistic evaluation in the way it is outlined by Pawson & Tilley. The shared view was that the team were not so concerned with the precise language used as with the involvement of the projects. i) The language of realistic evaluation was confusing for many people. ii) the projects appeared more complex and ambiguous in their outcomes than many of those described by Pawson and Tilley. iii) Realistic evaluation had been a useful model to introduce when attempting to provide a template for all projects. Since the team were now working more closely with individual projects we felt the strict template was no longer appropriate.

One of the initial findings was that established projects were much more likely to know how to evaluate. The more innovative projects had much greater difficulty. A constant tension is in how far the team members prescribe either what is evaluated or methods of evaluation.

However, there was often a conflict between established methods of evaluation and those that the team were trying to introduce. While testing theory is regarded as the best method of developing knowledge, much work, even at a high academic level is more “dust bowl empiricism” of collecting data and then looking for relationships. The current approach to working very closely with teams allows considerable discussion but is an expensive process. In some cases there was a tension between the role of funding a service and encouraging learning about the service.

The aim of building knowledge showed a “Concern about abolishing bad practice, improving performance out there, flush out duff theory” But there were initial difficulties in the tension between both being, “emancipatory, and wanting to be more directive”. In order to achieve this the projects were seen as “Two distinct phases first critically review assumptions, second service delivery.” However, the first stage is much more complex than it appears. Many of those working in the sector have strong personal and ideological commitments to particular positions and are used to defending their service and the service users at all costs (Kornberger & Brown, 2007). Critical examination of the service offered did not come naturally to many of the partners. “What are we asking? Too much? They are providing a service.”

Members of the Partners for Health team found that challenging assumptions held by the partners could threaten trust in the relationship and needed to be developed over a long period of time. In discussing the difficulties one comment was of the “Conceit of thinking one conversation with me would make a difference”. This focus on challenging content meant that partners felt in the early stages they were, “not in a strong enough relationship to do development work.” Trust was extremely important to the development of a serious evaluation otherwise it was felt that all the attempts to support the evaluation were seen as “being spied on”. This reinforced the understanding of evaluation as a political process involving judgment by one party of another (Legge, 1984) for their own not necessarily benign purposes.Evaluation loaded term change to action plan – how can you make an impact. How do you feel safe talking about this. Huge issue on feeling safe about saying “I don’t really know” Modelling at workshops?

A further tension surrounded the aim of Partners for Health in achieving evaluation of a sufficient standard to be judged as useful evidence by the scientific community. Discussion raising this as a key issue resolved much of the ambiguity about the purpose of the evaluation. The techniques associated with personal development of individuals were not, in the main, consistent with those necessary for evaluation of the project. While this was a disappointment, it is anticipated that once the project has developed a certain momentum the two aims may be brought together again.

External evaluationAnother issue for the team was the role of the outside evaluator. In following ideals of individual and group development, learning communities and a democratic approach to knowledge the focus was on Partners conducting their own evaluation as an integral part of

the intervention design. However, a common view by partners was that outside evaluation was preferable. The arguments for outside evaluation were i) technical skill of a professional evaluator; ii) objectivity of an outsider rather than the subjective bias of reflective learning; iii) resourcing requiring partners to spend all available time on service delivery rather than reflection on practice. One disappointing finding was the poor standard of much of the evaluation advice that was bought in. This tended to recommend attitude questionnaires. Questionnaires were not often useful because the sample size was not sufficient to allow meaningful analysis. Furthermore the development of the questionnaire appeared to be somewhat haphazard. No information was given on how items were derived, reliabilities or validity of the measures. One role for the King’s Fund in future may be to review available measures for use with specific projects.

Conclusion

Partners for Health is a model of radical cultural change for funders and their partners. As with any culture change it can be both inspiring and elevating and difficult and time consuming for the individuals involved. The project shows the importance of recognising the political nature of evaluation in the funder relationship. It demonstrates the need for both clarity and flexibility in guidance for evaluation.

Next stepsThe King’s Fund is uniquely situated to develop a useful knowledge base for health related projects. Combining the existing knowledge base in evidence based practice with active networks developing practice based evidence promises could provide a living knowledge resource. This radical change in funded projects could add immensely to our understanding of health interventions.

The key steps:1. Build up networks between projects working in similar areas through conferences,

newsletters and reports to share practice based evidence.2. Prepare guidelines for evidence based practice in these key areas.3. Identify key stakeholders to disseminate best practice.4. Practice based evidence used in conjunction with evidence based practice could

identify effective interventions and guide public policy.5. Train further grant managers to facilitate evaluation and contribute to the knowledge

base.

References

Cobb, P.B. & Rubin, B.A. (2006) Contradictory interests, tangled power, and disorganized organization. Administration & Society, 38:79-112.

Craig, G. & Manthorpe, J. (1999) Unequal partners? Local government reorganization and the voluntary sector. Social Policy and Administration, 33:55-72.

Julnes, G, Mark, M, & Henry, G (1998) Promoting realism in evaluation. Evaluation, 4(4):483-504

Kreiner, G.E., Hollensbe, E.C. & Sheep, M.L. (2006) On the edge of identity: Boundary dynamics at the interface of individual and organizational identities. Human Relations, 59:1315-1341.

Kornberger, M & Brown, A.D. (2007) ‘Ethics’ as a discursive resource for identity work. Human Relations, 60: 497-518.

Legge, K (1984) Evaluating Organizational Change.London Academic Press.

Marchington, M & Vincent, S. (2004) Analysing the influence of institutional organizational and interpersonal forces in shaping inter-organizational relations. Journal of Management Studies, 41:1029-1056).

Milewa, T., Dowsell, G et al (2002) Partnerships power and the “new” politics of community participation in British Health care. Social Policy and Administration, 36: 796-809.

Morzinski, JA & Fisher, JC (1996) An evaluation of formal mentoring studies and a model for their improvement. Evaluation Practice, 17 (1):43-56

Pawson, R & Tilley, N (1997) Realistic Evaluation. Thousand Oaks,CA Sage

Sanyal, P. (2006) Capacity building through partnership: Intermediary nongovernmental organizations as local and global actors. Nonprofit and Voluntary sector Quarterly, 35:66-82.

Tools used by the partners for health team

Tool Aims Contribution Problem ConclusionEvaluation plan(using CMOs)

Clarify theory and specify outcomes

Introduced ideas of evaluation

Too difficult to use

Evaluation needed to be more flexible

Unify different approaches

Too rigid for the range of projects

User pathway Identify the whole user experience

Move away from focus on inputs to focus on client

Wikki Share knowledge

Under used.Lack of access to ITNo interest in knowledge sharingProjects too diffuse or in competition

Blog (Partners)Blog (team)Learning plan Move away from

hard “evaluation” to more acceptable “learning”Clarify links between project management and evaluation

One to one support

Individual support for projects

Wiki-Based Knowledge Exchange Among

Partners-For-Health Grant Recipients:A Discussion of Barriers

Richard Giordano

15 June 2007

Version 1.01. INTRODUCTION.There has been increased interest in the UK in network-based modes of organizing and

learning in the public services in general, and in both health care and public health in

particular. The purported benefits of network-generated knowledge formation is that it makes

explicit tacit knowledge that has been generated and tested in real, working settings, and that

such knowledge has the potential to inform best practice among practitioners. Moreover,

knowledge generation derived from work practice in networked settings is likely to be

sustained because it has a manifest social dimension driven by ongoing work and problem-

solving activities. Because of its social dimension, network-generated knowledge generation

potentially supports shared understandings of work practices in communities of practice that

display clear patterns of mutual engagement, joint enterprise and an emergent and shared

repertoire of action. [1] [14]1 The intention of the Partners For Health wiki was to encourage

the growth of a community of practice among its funded organizations: [23] The goal was to

create intra-organizational histories of joint learning that could result in the collective

identification of best practice derived.

Partners-For-Health established a wiki and encouraged, but did not require, participants to use

it to pose questions and to share experiences gained through practice in the field—both

practices that appear to work or show promise, as well as those that failed or had unintended

consequences. After about twelve months in operation, participants failed to use the wiki.

The central question in this section of the report is, “What barriers caused the wiki to fail?”

This section describes the use of a wiki to foster joint learning among ten Partners-For-Health

1 I have included citations in case readers want to follow-up on some of the ideas or issues in the report.

projects; outlines the reasons why knowledge exchange using a wiki failed; suggests a set of

approaches that may prove useful for supporting online knowledge exchange in the future;

and identifies implications for future management of such activities for the Kings Fund.

2. BACKGROUND AND RATIONALE FOR KNOWLEDGE EXCHANGE.The Partners-For-Health programme encourages its grantees to engage in reflective practice,

to apply what they learn to understanding the effects of their work, and to share experience

and lessons learned through their field work with others. The Kings Fund does not take a

simple evaluative approach such as “Did it work?”, but instead is interested in learning about

why a project succeeds or fails. To this end, Partners-For-Health programme encourages

grantees to collect and evaluate data and to construct theory that can be used by both the

Fund and the grantees themselves to build a body of knowledge of what works, for whom, and

under what circumstances or contexts. [21] The Kings Fund works in partnership with

grantees to use the experience of field workers to construct knowledge, and to make explicit

the organizational and contextual orientations that influence practice and knowledge.

There were twelve funded projects concerned with improving the sexual health, mental health

or end of life care of Londoners, particularly those in vulnerable populations, such as the poor,

the aged, the homeless, members of new immigrant communities or any combination of these.

The funded projects are encouraged to investigate practical, in situ methods of improving

services and evaluate the effects of those improvements.

The field work itself takes place in complex environments. The complexity arises because the

projects often have to cope with a variety of pre-existing social and health services—such as

existing hospitals, physicians, housing agencies, welfare agencies, and community-based

organizations (such as local churches) as well as unstable or emergent social contexts, such

as changing immigrant populations in London, or variations in government policy and policy

implementation. There are often varied cultural issues, particularly with immigrant

populations, that require of field workers delicate negotiation for gaining entry and building

trust with clients. In many communities, there is a manifest lack of trust with existing social

services, and many clients show an outright fear of being subject to these agencies. For the

field worker and agency that is intent on improving health outcomes in these environments,

these issued exist in combination—a client, for example, may have a mental illness, be a

member of a new immigrant group that stigmatizes mental illness, possibly face

homelessness, and lack trust in social services. The field worker in this case is faced with the

complex task of gaining entry, building trust with clients, building working relationships

between clients and social services, and building relationships across social services that is

client-centered. As they work in such environments, the Partners-For-Health programme asks

grant recipients to develop strategies to understand the effects of this work as well as the

circumstances that may influence those effects. [19]

Any one of these problems is difficult to solve, and the complex combination of them would

overwhelm any individual’s capacity to make sense of them. Because of the complexity of the

work, and the skills needed to carry it through, the Partners-For-Health programme

encouraged a process of joint learning through knowledge exchange. That is, the Programme

took the view that the generation of knowledge through a process of networked knowledge

exchange was a hopeful approach to creating shared understanding and the identification of

best practices. This belief was influenced not only by the recent interest in networked

knowledge in policy circles, but also by more fundamental studies in shared cognition and

learning that argue that knowledge generation and sensemaking takes place within a

community, not atomically within the heads of unconnected individuals. [13].

There was reason to believe that such knowledge-sharing network would emerge relatively

from the funded partners in a relatively unconstrained fashion: The practitioners under study

engage in similar practice, faced a common set of complex problems, and share a common

commitment to their clients. In an ideal world, this group would form into a community of

shared practice given the right tools and opportunities. [23]

3. THEORECTICAL ORIENTATION.The focus was on the sharing of procedural knowledge and collective sense making. The wiki

was intended to capture tacit (how-to) knowledge and sense making knowledge derived

through practice, as Brown and Duguid argue, “close to the point at which working life is lived,

[and] work done”. [6] Such knowledge affords an understanding—both at the Fund along with

its funded partners—of what works, for whom, under what conditions. It uncovers coping

strategies developed in use that overcome barriers and unpredicted situations. [4]. To see

this from another perspective, we may compare the funded partners with a group of scientists.

We typically know science from the formal publications and reports, but the practice of science

—how to get to the formal publication—comes from an understanding of how laboratory work

is actually done, and the tacit or “private” knowledge that grows as a result of bench

processes helps others, particularly novices, to understand how science is actually practiced.

[16] [17]

Shared knowledge derived from practice is not a commodity that can be packaged and

exchanged (such as a textbook), and nor is it a set of objective facts and independently-

verified causal explanations. Instead, knowledge is situated and has a community-based set

of meanings.[9]. In a networked setting, to learn and to exchange knowledge is to be seen as

being competent. That is, to be accepted into the group a member must be competent in

does (your competence is measured by your technical competence [15]), and to be seen as

capable of participating with others in a web of complex relationships. [8]. Knowledge and its

exchange therefore cannot be separated from the emergent social relations that it influences.

[20]. This argues, as Wenger and others have, that knowledge is strongly connected with

identity.

In a networked environment, identity is, at best, only partially under the control of the

participant. Texts contributed to a wiki by participants can be constructed and deconstructed

by other members in ways that are beyond control of participants, and can be used by others

to generate multiple identities or as a means of power and control. [15] [7] that have the

potential not only to keep a member from participating fully in the group, but can also have

real, tangible, and lasting economic or material consequences for the participant or the group

represented by the participant. There are a number of potential barriers. First, participants

may be reluctant to share tacit knowledge because such knowledge may be misinterpreted by

others, and this may affect their reputation in the group and beyond. Also, having fresh

insights is often seen as a status marker; but, there is a stigma attached to those in that

community that consume, rather than produce, new knowledge or techniques. Consequently,

an organization that is seen to consume but not contribute to a group can potentially be

stigmatized by the group. [5] Finally, being in possession of new techniques or knowledge

does not necessarily mean that one will share it, even if it increases a group’s status within a

group. Why? Because it gives one an edge over a competitor which is not easily given away

freely. This competitive edge is well known in commercial settings, but studies in the sociology

of science identify the intrinsic value that lab techniques and know-how have in scientific

research labs, especially those in competition with each other, and the reluctance of scientists

to share emergent knowledge within a community before it is formally published. [18] [12] [11]

There is evidence to suggest that non-profits, too, are in competition with each other for

funding, and are not likely either to share information that may adversely affect their own

identities or which will give a difficult-to-gain insight to a competing organization. [10] Non-

profits, as we have seen, often work in environments that are complex and where task

uncertainty is high. Some research has suggested that the higher the task uncertainty, the

higher the valuation placed on the resulting knowledge by researchers and practitioners. [5]

The higher the value of knowledge, the less likely it will be shared without some equitable form

of reciprocity. [3] It can be argued that the material and economic consequences of exchange

of tacit knowledge (or know-how) can be enormous for participants.

4. DATA GATHERING FOR THIS REPORT.In May, 2006, partners were brought together at the Kings Fund for a day-long workshop to

discuss what they considered to be the barriers to knowledge exchange. Participants reported

in both plenary sessions and in small workshops composed of six to eight members each.

About thirty-two representatives from eight funded projects (ten projects were invited)

participated in the workshop. Representatives from the Kings Fund facilitated the workshop,

and small group discussions were facilitated by the participants themselves who kept notes.

In addition, field notes from plenary sessions and workshops were collected by the author who

observed, but did not participate in, the plenary and small group sessions. These notes were

grouped into themes and serve as the basis of this report.

4. RESULTSThe results of our analysis of field notes suggest that economic considerations play a

dominant role in hindering knowledge exchange with the group under study, and the need for

learning how to exchange knowledge in electronic settings playing a lesser but consequential

role. The responses from this group also suggest that issues of identity are intertwined with

the economics of knowledge exchange.

4.1 The economics of knowledge exchange.The economics of sharing knowledge derived from field notes fall into two categories: The

costs of sharing knowledge; and, The impacts or payoffs of sharing knowledge. With respect

to costs, most participants argued that it takes time to compose a wiki entry and to read the

entries of others. They identified time is the most precious resource that they control, and they

manage their time carefully and judiciously. If they were going to compose an entry, they want

a tangible benefit from it—and many could not see the tangible results. A common refrain

was, “Writing something takes a lot of time, and I can’t see the benefit of this for me.” Indeed,

many did not want to go to a wiki (or a blog) to find an answer to a question because, as one

put it, “you’re confronted by threads that you trip over and which may, or may not, answer your

question. I don’t have the time for that.”

Some workshop participants considered a cost/benefit model in terms of time spent

composing an entry to a wiki and the benefits that would accrue from this. The payoff that

motivated the participants was an increase on perceived competence and reputation among

institutions and individuals “that matter”. There was barely any mention at all of acquiring new

skills and insights from others These groups that matter to the funded partners are the

professional medical and health communities, as well as agencies that affect policy. For

example, a representative comment was, “If I spend time writing, I have to consider my

audience and the impact. My audience is doctors. If I write, I need to write for doctors

because that’s what gets me my kudos.” Moreover, there was a concern that taking the time

to write should push forward the importance of the work with policy makers. A representative

comment in this regard was, “I’m concerned with advocacy, so anything I write has to put

advocacy on the map. I have to be careful about what I say and how I say it, or we won’t be

taken seriously.” This respondent did not see the value in gaining experiential knowledge or

insights from other practitioners, nor in sharing with peers—instead, the payoff was in gaining

the approval of “people who matter,” (her words) notably funders and policy makers.

Finally, respondents carefully considered the implications that sharing knowledge would have

on future funding. At issue were (a) what you shared and (b) how much you shared. For

many, this was a lose-lose wager. For example, if you exchanged knowledge of the practices

that did not have the intended effect (altruistically helping others avoid making the same

mistake), there was a palpable fear that you would be perceived as incompetent by both your

peer group as well as potential future funders; if you shared too little, then the peer group

would perceive you as simply consuming knowledge or hoarding what you know. Knowing

just what to share was seen to be fraught with uncertainty of its effects. A representative

comment was, “If we don’t get funded again, we may think ‘It must have been something we

said or haven’t said.’” This participant was concerned about funding from the Kings Fund as

well as other agencies.

4.2 Learning how to exchange knowledge.There was marked concern with knowing when to share and what to share, and this was

related to the judicious use of time identified in the previous section. Participants were also

unsure of the quality or relevance of their knowledge to the group. That is, they did not know if

their work would be relevant and useful to the group at large, and they feared that they may

waste the time of the other participants and thus reflect poorly on them. The issue was not a

fear of appearing incompetent to the group, but a general unfamiliarity with knowing what

information they held would be relevant to the group. For example, there was a general

ambivalence in the practitioners’ ability to decide successfully what would be valuable to the

group. For example, a representative question was, “How do we know what to say that’s

important?” There was widespread support for the notion that group norms needed to be

developed on notification (that is, when to post a message) and the identification of “important

findings” (what the proper content of a message should be). There was also a general lack of

enthusiasm in the small group settings for posting questions to the group at large, as this

seemed inefficient both for the sender and for the respondent. That is, the sender would get a

better response if they sent a question to a known expert or someone with known similar

experience. As one respondent said, “I can ask a question in public on a wiki, but I can also

just phone someone and talk it through in private.” Moreover, no one relished the though of

wading through a sea of questions to find one they could answer. Such an activity, in the

words of one participant, “is nice to do, but that’s not what I’m paid to do.”

5. CONCLUSIONThe evidence suggests that the major barrier in exchanging knowledge across organizations

in this group under study was the perceived economic or material impact on the one hand,

and the lack of norms and accepted group behaviour on the other. There was a general

consensus that the resource cost to contribute to the wiki was not offset by any real, tangible

benefit; that the resources can be used to better effect is they were directed at those who

could control future funding and resources; and, finally, that contributions may be used in the

future against funding decisions. We saw that issues of identity, particularly around the areas

of the group perception of competence, were also an important factor the militated against the

exchange of experiential and tacit knowledge. Finally, the respondents indicated that they had

little guidance on what the group believed was “important” and “relevant” information to share.

5.1 Relevance to design considerationsThis work surfaces some design considerations that focus on the social organization and

management of a group in an electronic environment that is intended to exchange knowledge.

We have seen that ease of use (a wiki) and shared practice (participants engaged in similar

practices and confronted similar problems) is not enough to trigger and sustain the exchange

of knowledge on its own. What is needed is a social scaffold and the generation of explicit

group norms to encourage knowledge exchange. These would include:

The creating of social interaction around emergent discoveries to help the group itself

establish group norms of behavior and group expectations. [5]. These norms and

expectations should be made explicit before the group engages a wiki.

Maintain group norms through multiple sessions that discuss norms, and what seems

to be working (or not working), and develop a governance structure that is seen to be

cost effective by the group... [14]

In preparation to engaging in knowledge sharing, develop norms of what to be shared,

how knowledge will be shared, and what is acceptable, with the use of examples to

make the norms explicit. [2] [22]

All of these point to the argument that while advanced information and communications

technologies, including social software such as a wiki, have helped to realize the a “knowledge

revolution,” it requires humans and their attendant social systems to realize it. [9]

5.2 . Implications for the Kings FundAll of these point to the argument that while advanced information and communications

technologies, including social software such as a wiki, have helped to realize the a “knowledge

revolution,” it requires people and their attendant social systems to realize it. [9] This has

implications for how the Kings Fund may manage knowledge exchange activities in wikis for

future projects. Some of the more important considerations include:

Grant managers should initiate processes and activities with funded partners that

engender (i) trust across partners, (ii) shared norms, and (iii) shared expectations. This

may be a role with which grant managers are unfamiliar, and may believe is outside

their areas of expertise. This may imply that third parties may be needed to play this

role, thereby driving up the expense of managing a contract.

Before a wiki can be established, participants need to meet face-to-face to establish

trust, norms and expectations. This is not a process that comes quickly or easily, but

may require a number of face-to-face meetings where participants learn about each

other, what they have in common, what their differences are, their areas of expertise,

and how they behave. It is unclear if participants have the time to devote to this, nor if

the expense can be easily justified, but this can be a by-product of a number of staged

face-to-face knowledge exchange workshops.

In the future, Kings Fund should establish an evaluative framework with realistic

expectations on what knowledge will be shared, with whom, when and under what

circumstances before establishing a wiki to support knowledge exchange. This will help

delineate the differences between an ongoing wiki and a face-to-face knowledge-

sharing workshop. Moreover, the Kings Fund and its partners should agree on tangible

products and benefits that accrue from participation in a wiki, and manage the wiki to

maximize the benefits.

6. ACKNOWLEDGMENTSThanks to Kate Batlin, David Naylor, Kate Hinds, and Steve Dewar.

7. REFERENCES

[1] Addicott, R., McGivern, G., and Ferlie, E. Networks, organizational learning and knowledge management: NHS cancer networks. Public Money and Management 26, 2 (April 2006): 87-94.

[2] Ardichvili, A., Page, V. & Wentling, T. Motivation and barriers to participation in virtual knowledge-sharing communities of practice. Journal of Knowledge Management 7, 1 (2003)64-77.

[3] Bell, D.G., Giordano, R. & Putz, P. Inter-firm sharing of process knowledge: Exploring knowledge markets. Knowledge and Process Management 9, 1 (“002): 12-22.

[4] Beunza D. & Stark, D. The organization of responsiveness: Innovation and recovery in the trading rooms of Lower Manhattan. Socio-economic Review 1 (2003): 135-164.

[5] Birnholtz, J.P. & Bietz, M.J. Data at work: Supporting data sharing in science and engineering. Group03, 9-13 November 2003. Sanibel Island, FL.

[6] Brown, J. S. and Duguid, P. Knowledge and organization: A social practice perspective. Organization Science 12, 2 (2001): 148-213.

[7] Carlile, P. A pragmatic view of knowledge and boundaries: Boundary objects in new product development. Organization Science, 13, 4 (2002): 422-455.

[8] Contu, A. & Willmott H. Re-embedding situatedness: The importance of power relationships in learning theory. Organization Science 14, 3 (2003): 283-296.

[9] Currie, G. & Kerrin, M. The limits of a technological fix to knowledge management. Management learning 35, 1(2004: 9-29.

[10] Ebrahim, A. Information struggles: The role of information reproduction of NGO-funder relationships. Nonprofit and Voluntary Sector Quarterly 31, 1 (2002): 84-114.

[11] Gaston, J. Originality and competition in science: A study of the British high energy physics community. U. of Chicago Press, Chicago, IL, 1973.

[12] Hagstom, W.O. Competition in science. American Sociological Review. 39, 1 (1974): 1-18.[13] Hutchins, E. Cognition in the wild. MIT Press , Cambridge, MA, 1995.[14] Jones, C. Hesterly, W.S. & Borgatti, S.P. A General theory of network governance:

Exchange conditions as social mechanisms. Academy of Management Review 22, 4 (1997): 911-945.

[15] Lamb, R. & Davidson, E. Information and communication technology challenges to scientific and professional identity. The Information Society. 21, 1 (2005): 1-24.

[16] Latour, B. & Woolgar, S. Laboratory life: the construction of scientific facts. Princeton University Press, Princeton, NJ, 1986.

[17] McCain, K.W. Communication, competition and secrecy: The production and dissemination of research-related information in genetics. Science, Technology and Human Values 16, 4, (1991): 491-56.

[18] National Academy of Sciences. IPR and research tools in molecular biology: Summary of a workshop. NAS Press, Washington, DC, 1996.

[19] Ogawa, T.T. & Bossert, S.T. . Leadership as an organizational quality. In M. Fullan (ed.), The Jossey-Bass reader on educational leadership. Jossey-Bass, San Francisco, 2000.

[20] Ǿsterlund, C. & Carlile, P. Relations in practice: Sorting through practice theories on knowledge sharing in complex organizations, The Information Society 21 (2005): 91-107.

[21] Pawson, R. Realistic evaluation. Sage: London, 1997.[22] Ryan, D. Getting the word out: Notes on the social organization of notification. Sociological

Theory. 24, 3, (2003): 228-254.[23] Wenger, E. Communities of Practice: Learning, Meaning and Identity. Cambridge

University Press, New York, 1998.