Embed Size (px)
Citation preview
Institutional Approaches to Curriculum Design
Institutional Story Template
Project Information
Project Title (and acronym) Technology Supported Processes for Agile and Responsive Curricula (T-SPARC)
Start Date October 2008 End Date May 2012
Lead Institution Birmingham City University
Partner Institutions N/A
Project Director Professor Stuart Brand
Project Manager & contact details
Professor Paul Bartholomew
Edge 610
City North Campus
Birmingham City University
B42 2SU
Project website http://blogt.bcu.ac.uk/tsparc/
Project blog/Twitter ID @TSPARC_BCU
Design Studio home page http://jiscdesignstudio.pbworks.com/w/page/36560187/T-SPARC%20Project
Programme Name Institutional Approaches to Curriculum Design (05/08)
Programme Manager Sarah Knight / Lisa Gray
0. Summary
The Technology Supported Processes for Agile and Responsive Curricula (T-SPARC) at Birmingham City University has sought to transform the institutional culture and practice of curriculum design and approval. Our intention has been to move from a position where curriculum design as a process is undertaken primarily as a prelude to an end-point approval event to one that embraces iterative collaborative design from which approval cascades. Furthermore, we have sought to develop technology that supports this approach within an online environment and to enrich the design process by making technology-based tools available to programme teams so as to broaden their opportunity to engage meaningfully with a wider range of stakeholders than do our extant processes.
In summary, we have created and deployed both policy and systems that allow for successful iterative collaborative design and ‘emergent approval’. However, our pilot technology offered a steep, and at times frustrating, learning curve for our (pilot) users. This has contributed to (though by no means fully) some cultural inertia in relation to full embracement of the potential of the new process and system by our pilot users.
Our piloting activity has provided a wealth of information in relation to how people use the new system and has allowed us to develop further our systems and processes. However, we can only claim partial success in the way the new infrastructure was used and we will now proceed to a second set of pilots in the 2012/2013 academic year, with a view to rolling out the new approaches across the University in the 2013/2014 academic year.
1. What are the headline achievements of your project?
Our ‘headline achievements’ can be broadly categorised as those that have emerged from the process of spending nearly four years on an institutional approaches to curriculum design project
and those that have become manifest as a product of our endeavours.
Institutional Approaches to Curriculum Design
Institutional Story Template
Achievements via ‘process’:
We enhanced our understanding of the drivers of the curriculum design process. Early in the project we undertook a Baseline Review so as to capture an institutional context to which the project could refer as an initial environmental analysis. This Baseline Review offered up the following themes as a representation of the lived experience of curriculum design (and programme approval) by our staff:
• Drivers for Design: • Coordination: • The availability of Information: • Relationships and mechanisms: • Stakeholders: • Constraints: • Compliance: • The Programme Director: • Holistic and distributed approaches: • Authenticity: • Textual representation: • Specialised language: • Audience: • Staff support: • Existing use of technology: • Time and space for design:
Subsequent engagement activities allowed us to explore further people’s opinion of the effectiveness and efficiency of curriculum design and approval processes. We did this through a series of workshops that included an activity whereby stakeholders annotated process maps of the extant workflow associated with curriculum design and approval. Those annotated process maps can be found here. As a consequence of engaging with process maps, we took the opportunity to explore various options for creating such representations – the story of this exploration can be found on our Project Blog.
Members of the T-SPARC team developed a new version of a model to describe stakeholder engagement. This model has been well disseminated within the sector and has allowed us to engage a variety of groups in conversation around what is meant by the term ‘stakeholder engagement’. It has also offered a philosophical framework to run the project itself.
As the project progressed, we invested in and developed the use of video techniques as a way of recording and representing the perceptions of stakeholders in every stage of the curriculum design life cycle.
Extensive use of video caused us to reflect on the associated data protection issues and as a consequence we collaborated with our institution’s Information Manager to develop a stakeholder-facing pro-forma to facilitate informed consent in relation to the sharing of video contributions.
Availability of the video hardware we had bought led to a range of demands for the equipment. We have always been keen to support such use because it allowed for the the technology to become more mainstream thus smoothing the way for wider use of video; and secondly, the purposes of this parallel use frequently added value to the curriculum design agenda. For example, our video units have been used to explore the perceptions of international students and accessibility as perceived by students with disabilities.
Institutional Approaches to Curriculum Design
Institutional Story Template
Achievements via ‘product’:
In addition to the products that have emerged as a consequence of our journey (the Stakeholder Engagement Model, the Baseline Review etc.), we have four main achievements:
1. A new process for curriculum design and approval, including ratification from the University’s Senate to conduct live pilots.
2. A new SharePoint-based infrastructure to support the new processes of curriculum design and programme approval.
3. Completed pilots, nine programmes across four faculties, that have used the process and system mentioned above.
4. A new extensive guide to the process of curriculum design that supports programme teams in their work.
The new process:
Our new process for curriculum design and approval dispenses with the conventional end-point focus on documentation and instead has the expectation that programmes will be designed iteratively with input from a variety of stakeholders, including those tasked with conferring approval. Furthermore, the act of iteration, with the input from stakeholders, creates an ‘audit trail’ of activity that is verified as part of the approval process. Textual representations of the course emerge from this iterative activity. Approval of a course is made on the basis of both the process of design (as evidenced in the ‘audit trail’) and the products of design (the final documentation) are found to be satisfactory. This verification of the quality of process and product takes place within an online environment.
Our overall aim was for better design to lead to better courses. This aspiration is elaborated upon in the multimedia clip offered below:
The new SharePoint infrastructure:
Our initial (proto) activity took place using out-of-the-box SharePoint 2007; this work allowed us to build a ‘wish list’ of features to support effective design and approval. Colleagues in CICT took this wish list forwards and used it to create a bespoke environment using SharePoint 2010. A diagrammatic representation of the system can be found here.
Reflexive commentary offered by Paul Bartholomew on: ‘The new system as an output, how the system supports the new process’.
A demonstration of the (first) pilot version of the software and its associated workflow can be found here.
Completed pilots:
Reflexive commentary offered by Paul Bartholomew & Sonia Hendy-Isaac on: ‘Better Design leads to Better Programmes’.
Institutional Approaches to Curriculum Design
Institutional Story Template
The following facultiesput forward programmes to be designed and approved via the pilot infrastructure described above. They were:
Birmingham Institute of Art and Design
Faculty of Education, Law and Social Sciences
Faculty of Health
Technology, Engineering and Environment
Overall, nine programmes were designed and approved through this piloting activity.
These live pilots have allowed us to develop further the interface programme teams use and a second series of pilots, beginning in the 2012/2013 will use an interface that has been enhanced as a consequence of the feedback we have received through our initial pilots.
The ‘Rough Guide to Curriculum Design’:
As we worked with programme teams on their design of curricula, it became apparent that the augmented opportunities for effective design were being under-utilised. As a consequence we committed to developing a guide to support staff in their work so that the opportunity for better design became a reality. We chose to inform this guide based on the areas of support that emerge from the (just-concluded) pilot activity. Consequently, the first draft of this extensive resource (running to 16000 words) is only just available (at the end of the funding period).
Reflective commentary on the timing of the Rough Guide to Curriculum Design – Paul Bartholomew and Sonia Hendy-Isaac in conversation.
2. What were the key drivers for undertaking the project?
The core project drivers (aims) for undertaking the project taken from the Project Initiation Document are listed below and a reflexive commentary offered:
1. Informing programme design activity through the enhanced provision of pertinent information.
2. Redesign of the ICT infrastructure that underpins the workflow of the curriculum design and programme approval processes.
3. Electronic support for course team dialogue during their programme design activity.
4. Electronic representation of programmes and underpinning evidence at (and leading up to) the point of approval.
Reflexive commentary from Paul Bartholomew on: ‘The core project drivers (aims) for undertaking the project’.
Although there have been some shifts in relative importance, these four aims have not changed throughout the duration of the project and pursuit of them has continued to drive our work.
They do warrant some expansion though:
Institutional Approaches to Curriculum Design
Institutional Story Template
Originally, we had intended Aim 1 to be about the provision of background resources to inform design choices (such as market information, progression and retention statistics etc) but as the project progressed we found that institutional imperatives around the provision of such information led to autonomous emergence of accessible data in these areas. More interestingly though, we found that programme teams needed significant support to make best use of the new opportunities afforded by our new processes. It was this insight that led to a decision to author the Rough Guide to Curriculum Design.
We initially intended to deliver Aim 2 through the deployment of three separate technologies – Moodle, to support team-based discussion; Mahara, to offer a representation of the design process (as an evidence-supported narrative account) and SharePoint as a document store for the finished documentation. As we explored the technologies, we found that SharePoint could serve all three roles. We therefore decided that the simplicity of using a single software product was the most elegant solution, and all development activity was funnelled into developing SharePoint.
For Aim 3, where we refer to the ‘course team’ - this has always meant the ‘wide course team’ – to include all those with a vested interest in the programme. In that context, supporting course team dialogue has been an exercise in effectively engaging (all) stakeholders in curriculum design.
Aim 4 represents the biggest cultural shift in our processes and the lived experience of curriculum design. In the extant process, only the products of curriculum design (the definitive documentation) have been subject to scrutiny; but in our new process, course teams need to additionally evidence a thorough design process that is inclusive of their stakeholders. It is this central premise that drives the use of our video technology, promotes wider dialogue and thus promotes more effective design. Once the threshold of expectation in relation to effective design practice was raised, most course teams needed support. The Rough Guide emerges from this context.
3. What was the educational/organisational context in which you undertook your project?
The T-SPARC project has been embedded within a wider project relating to the rejuvenation of ALL existing curricula. Where T-SPARC leads the way in developing new processes for the design and approval as we go forward, so the Redesign of the Learning Experience (RoLEx) project has been the vehicle to rejuvenate all of our exiting provision. A review of the RoLEx project is given below: Launched in July 2008, Birmingham City University’s RoLEx project has evolved through three iterations to become a set of principles embedded within institutional culture and practice to an extent that it no longer requires an explicit curriculum design project to ensure sustainability. This section reviews the RoLEx project, highlights some examples of its impact and introduces the follow-on initiative that will define and support emergent work.
Background When RoLEx was launched at the Learning and Teaching Festival of July 2008, some key principles of the initiative were shared; RoLEx was to be characterised by:
Alignment with the Learning and Teaching Strategy
Consultation with students
Consultation with employers
Courses becoming more attractive to potential students and employers
Reduction of assessment and marking load
Institutional Approaches to Curriculum Design
Institutional Story Template
Removal of duplication and reduction of complexity
A positive effect on working lives of staff
These principles have been applied through three iterations of RoLEx:
1. RoLEx 1 (2008/09): Using the opportunity afforded by the change in credit structure (from 12 to 15 credits per module) to redesign undergraduate curricula in accordance with the RoLEx principles outlined above.
2. RoLEx 2 (2009/10): Applying the RoLEx principles to the redesign of postgraduate provision.
3. RoLEx 3 (2010/11): Thematic in nature; all programme teams were asked to review provision with reference to improving the University’s response to the agendas of employability and assessment design.
RoLEx 1
As the first opportunity to evaluate the way in which the RoLEx principles could be enacted in practice, an evaluation group drawn from both internal and external agents comprehensively evaluated this first iteration. The group reported their findings under ten discrete headings, but two particular sections of the report warrant inclusion here as they relate to subsequent work.
Student engagement
“Many students who participated in RoLEx supported it, believing that it could enhance their learning experiences.
The students in the interview group reported that a large number of students did not know about RoLEx.
Variability in the students’ experience of consultation and involvement was noted. While the documentation presented to us indicated extensive and effective student engagement, the evidence of this was less strong from the student interview group in terms of both quality and quantity of engagement.
There has often been a focus on informing students (for example, documents sent by e-mail) rather than involving them (for example, through workshops with their course coordinators and staff team). Some students seemed to have been satisfied with very limited involvement which may have accorded with the way in which RoLEx had been framed for them.”
These findings provided further stimulus for the work that the Centre for Enhancement of Learning and Teaching (CELT) had started, leading to a series of initiatives to enhance the ways in which the University engages with its students:
The development of this new approach to curriculum design and approval (T-SPARC).
The development of a stakeholder engagement model – designed to clarify and share notions of ‘engagement’.
Stronger links with the Students’ Union including the (50%) secondment of a member of staff.
The Student Academic Partners scheme which, since 2009, has led to:
Institutional Approaches to Curriculum Design
Institutional Story Template
The employment of over 500 students as academic partners in development projects.
The involvement of over 200 staff across the university in partnership academic development work with students.
The generation of over 160 academic development projects.
A Student Academic Mentors scheme, piloted with over 60 student mentors employed in 2011/12. This will be expanded in 2012/13.
Selection of the University by the Higher Education Academy/Leadership Foundation for a Change Academy project in 2011/12. This aims to generate a wide range of employment opportunities for students on campus.
Employer engagement
“Most programme teams reported engagement with employers. Engagement was particularly well detailed where existing (pre-RoLEx) relationships were firmly established.
Most consultation appeared limited in scope, reported as mainly confined to the sharing of plans and the receiving of feedback. However, some co-creation of programme design was apparent.
Where co-creation of programme designs with employers was evident, this was often reported as being mainly located in the placement element of the programme.
Some staff reported that, had they had more time, they would have liked to consult more thoroughly with employers.
Employers were also used as a route to enable engagement with other stakeholders such as clients and service users.”
The limited engagement with employers as part of the curriculum design process was one of the factors that led to a renewed focus on employability through RoLEx 3.
Additionally, the stakeholder engagement model and the new processes for curriculum design and approval bolstered institutional expectations in relation to employer engagement in curriculum design and the T-SPARC processes provide a technology-based solution to support such work.
RoLEx 2
Because of the diverse nature of post-graduate provision; RoLEx 2 was far less centre-led with faculties asked to develop a targeted response to the RoLEx principles consistent with their faculty strategic plans. Faculty responses included:
Strengthening of development of academic skills.
Enhancements to approaches to Personal Development Planning.
Enhanced use of Technology Enhanced Learning and Teaching.
The introduction of new programme pathways.
Changes to titles of awards.
Enhanced use of simulation (Health and Business).
Institutional Approaches to Curriculum Design
Institutional Story Template
Development of internships.
Establishment of a post-graduate student societies.
Greater use of employers/companies as placement opportunities.
Enhanced use social media to support prospective students.
Enhancements to dissertation support.
Rationalisation of module portfolio.
RoLEx 3
This iteration of RoLEx sought further to drive change through ownership at programme team level – specifically in relation to employability and effective assessment design. Support for this approach was facilitated through a series of RoLEx 3 workshops attended by staff and students. These programme teams were selected by faculty management to ensure support was targeted (and thus aligned with faculty priorities) but this approach also facilitated direct communication to, and support from, CELT. This approach allowed for ‘bottom up development’ characterised by rapid access to innovation funds and direct support for programme teams to help them take ‘ownership’ of the change agenda.
Summary
RoLEx generated new appetites for learning and teaching development and raised considerably the profile of such work in the University. In addition to enhanced student progression and achievement rates, the outcomes of RoLEx include:
A step change in student engagement in all aspects of the academic process.
Refinement of the curriculum and a stronger alignment to employer needs.
A reduction in the number of modules and in unnecessary duplication.
Assessment that is rigorous, relevant and challenging but with over-assessment being avoided.
A real focus on employability in course design and delivery.
The development of a new 'programme team' approach, embracing staff and students working together on joint initiatives which is proving attractive and effective.
External recognition by QAA of not only the benefits of our student engagement initiatives but also of RoLEx itself as a “mechanism for systematic enhancement that engages staff and students”.
QAA auditors describing RoLEx as “a carefully planned, rigorously evaluated, sustainable mechanism for systematic enhancement”.
The principles of RoLEx, as a vehicle for enhancement, are now being embedded through the processes developed by the T-SPARC project – so the RoLEx principles will continue as that ‘sustainable mechanism’ signalled by the QAA as we move from the need for widespread redesign to more targeted and continuous refinements.
Institutional Approaches to Curriculum Design
Institutional Story Template
As mentioned above, the institutional approaches to curriculum design were commended by the QAA Institutional Audit of November 2010. It should be noted though that this audit visit required considerable institutional preparation and as such identification of our T-SPARC pilots became a secondary priority and were deferred by 12 months. Although the deferment of these pilots has meant that this project appears to be rather back-loaded in terms of outcomes, there are two significant benefits that occurred as a result of this:
We were able to make significant inroads in developing SharePoint 2010 prior to rolling it out with the pilot teams.
Commendation of our broad approach to curriculum design and approval by the QAA led to enhanced institutional confidence in what we are trying to achieve and this has led to excellent positioning in terms of sustainability for the project and its aims.
4. What was the technology context?
When the Project Initiation Document was authored it was envisaged that Mahara and Moodle would be used to complement SharePoint 2007’s document collation function, with Moodle supporting and informing team discussion and Mahara acting as a ‘virtual scrapbook’ to host and represent evidence of programme design processes. Once we began to develop our use of this technology, SharePoint 2010 became available to us. Augmented functionality of SharePoint 2010 over SharePoint 2007 contributed to the case for moving away from a multi-software solution to the use of just SharePoint for all three facets of our work. At that time, our institutional expertise on developing solutions with SharePoint 2010 was very limited so the project funded some extra capacity to bolster productivity and capability in the form of an external contractor. The contractor’s brief was to:
Get the new SharePoint 2010 servers online
Undertake any technical precursor work that would limit T-SPARC development work in the future
Develop basic workflows for programme design and approval
Develop basic functionality for programme design and approval
Support BCU staff in to bolster their ability to undertake future software development of SharePoint 2010 so as to secure institutional capacity to support the infrastructure beyond the funding period.
Having the capacity to do subsequent development work in-house has been instrumental to the progression and usability of the SharePoint system. Although the myriad of features within our system is outside of the scope of this review document, one of features of the technology warrants specific expansion:
About mid-way through the project, following a Skype call with The University of Southern Queensland, we decided to use web form collaborative functionality instead of the more commonly used Word document management system within SharePoint 2010. The University of Southern Queensland had experienced issues with users uploading different versions of the same Word documents with ‘track’ changes applied to the different versions. Using a web form format has meant that:
There is only one version of the truth.
Institutional Approaches to Curriculum Design
Institutional Story Template
Track changes are not used and thus do not lead to the creation of disparate versions of a document, each with a set of changes that need to be rationalised by the document’s primary author.
The university templates only exists in one location – updating the ‘system’ copy, updates the users’ copies
Collaborative working takes place through associated and linked forum discussions – this means that people can collaborate on a document without needing to ‘check-it-out’ first (and by so doing effectively locking it for editing by others).
One of the objectives set out in our Project Initiation Document (1a.) was:
‘Course teams will be able the use evidence of redesign (such as video clips from team meetings) which emerges from their work rather than having to write post-hoc reports of their meetings.’
To facilitate the collection of emergent evidence, a range of media recording equipment was purchased:
70 x Flip video cameras (products no longer in production)
30 x Sony MP3 recorder
3 x Voxur units (products no longer in production)
1 x Miituu unit (iPad 2/3 based software)
We have made use of this technology in a number of ways but would offer the following commentary:
We made it as easy as possible for programme teams to borrow equipment. We have supported a number of projects institutionally that were not related to programme design and/or approval; we have done so to facilitate the embedding further of the use of video as a medium for collecting and representing authentic stakeholder voices. We wanted to mainstream the use of the technology prior to rolling out the T-SPARC approaches to programme design and approval. That way people would, we hoped, feel they were using familiar tools rather than having an entire suite of new technologies to navigate along with the changes to the design and approval process.
We selected video and audio tools for the collection and representation of stakeholder opinions and narratives for two reasons:
1. Encouraging enhanced levels of stakeholder engagement into programme design and approval mechanisms needed to be perceived as being as light-touched as possible. When we first floated the idea with our stakeholders that they would be offering up accounts of the design process as part of programme approval – the expectation was that this would involve “writing documents about writing documents”. We therefore wanted to introduce methods that would not lead to the summary production of further documentation.
2. Audio and video (in particular) methods of recording and representing stakeholder voices have far higher levels of authenticity than do textual representations of them – even if verbatim transcripts are used (and mostly they are not), much of the information encoded within speech – cadence, intonation and gesture are lost making such accounts partial. Textual representations also, quite literally, rob the stakeholder of their voice. We have believed it to be very important, from a philosophical standpoint, to protect the stakeholder voice and to maintain its authenticity.
One of our most successful investments in technology has been the Voxur units – case studies in relation to their use can be found on our project blog here.
Institutional Approaches to Curriculum Design
Institutional Story Template
Voxur units are no longer being produced, after the company produced them ceased to trade, but developers who used to work there have set up a business developing software for the iPad 2/3, called Miituu. It is very similar (even augmented) in functionality but with a much more convenient form factor. Engagement of Corporate ICT The institution’s Corporate Information and Communication Technology department do not work for the project, but with the project. Although we have an emergent specification to work to, it has been a full time job for the T-SPARC Project Support Officer to collate business / system requirements and communicate these to CICT developers and to undertake system testing with users on a daily basis.
5. How did you approach the project?
The project took a grounded approach, taking its lead from the staff that had undertaken curriculum design and approval activity. The JISC’s insistence that projects spend at least nine months in review was, in our view, an excellent decision. We actually spent a full year listening to people’s ‘lived experience’ of curriculum design and this formative experience shaped much of what we subsequently did. During this review period, all projects were encouraged (quite robustly) to conduct a baseline evaluation; in our case, that created a tension because we did not want to identify any baseline measures, as these would inevitably become the metrics against which we measured ourselves at the end of the project. We thought such measurement would be problematical, as we would in effect be defining the parameters of success prior to our planned engagement with stakeholders. The compromise position was for us to undertake a baseline review thus avoiding the collection of baseline measures. This tension, and our approach to resolving it, is discussed below:
Reflexive commentary from Paul Bartholomew on ‘The differences between a Baseline Review and a Baseline Evaluation’.
The institution chose to locate the project within the Centre for Enhancement of Learning and
Teaching and this choice reflected the overall aspiration for the project to deliver processes that would lead to better curriculum design practice. The Project Manager was the University’s Head of Curriculum Design and during the project also became the University’s Head of Academic Staff Development. This role extension led to some interesting synergies, deficits in staff abilities to design curricula (in the most effective ways) could be rapidly addressed through the development of new education provision for staff. Indeed, during 2010 the Project Manager led the development of a new academic staff development Master’s programme – the MEd Learning and Teaching in Higher Education including new modules on assessment design, employability, technology enhanced learning and teaching and evaluation to address this very issue.
The Project Director was the University’s Director of Learning and Teaching and during the project
became the University’s Director of the Learning Experience – this broadened role has allowed for greater latitude and influence in matters relating to student engagement and built on already excellent practice as evidenced by a Times Higher Award and by commendations from the QAA.
The substantive posts and associated roles of the Project Director and the Project Manager
afforded the project team a seat at all of the key decision-making committees and groups within the University; this ensured excellent visibility of the project to key decision makers and allowed excellent visibility of associated projects and initiatives by the project team. Indeed, such was the
Institutional Approaches to Curriculum Design
Institutional Story Template
‘connectedness’ of the project team with the governance infrastructure of the University; the T-SPARC Project Board was thought to be redundant and (with the JISC’s permission) subsequently dissolved. The absence of a Project Board allowed for rapid decision making relating to the running of the project to be continually articulated with the wider needs of the University.
As already mentioned above, the views of stakeholders have been of paramount importance to the
project team – not just on the philosophical grounds we have already discussed but also for a more pragmatic reason. Technology-based systems are never just technology-based systems; the involvement of human beings in the use of technology becomes the most important factor in determining success or failure. If our user groups did not like the system we were developing for them, they would simply subvert it, break it, or ignore it. Thus, effective and credible stakeholder engagement has been fundamental to the success of our project. Our overall philosophy, as represented in our model discussed above, has stood us in good stead as we have made choices in relation to project direction and pace. For more information on our approaches, the postings on our Blog page tagged with the term ‘stakeholders’ can be found here.
At the start of the project, the development of a formal Project Initiation Document and a push from
the JISC towards waterfall methods of project management (such as PRINCE2) offered a useful opening structure to begin work and to produce an initial shared framework for collaboration with CICT. However, after a fairly short time, this way of working became too restrictive – the expectation to declare dates, identify prerequisite activities and to develop ‘work packages’ was not well-aligned with the needs of a project that was trying to deliver against a continually emerging set of requirements from our stakeholder groups.
However, we nonetheless recognised that we would need some form of framework through which
we could prioritise emergent work and through which we could communicate our requirements to CICT. As a consequence of this need we adopted an Agile project management methodology. Further details relating to this shift in project management can be found on our Project Blog here.
Our approach to evaluation has always been pragmatic and aligned with the philosophy of
evaluation for development – we have never seen the project as operating in isolation of the institution and have already secured its status beyond the funding period. As a consequence, our evaluation activity has been targeted around what we can learn from the project (so far) and how we can use such findings to continue to enhance our work as an embedded institutional initiative. Having assured sustainability of the project, this leaves us with some tensions in relation to timing – although we are drawing to an end of the funding period, our work on the project is ongoing and as such some of our pilots are still in the final stages of approval at this time of writing. Nonetheless, we have been able to conduct some evaluation work with them and this is expanded upon in the evaluation report authored in parallel with this document.
Our evaluation methods have followed the approach we have taken in the Baseline Review – we
have sought to understand the ‘lived experience’ of curriculum design, although this time we have used Nominal Group Technique instead of individual interviews to collect this data. A full account of that work can be found in the Evaluation Report.
6. What benefits has your project delivered and who are the beneficiaries?
Benefit Delivery Status Who Benefits Links to Evidence
Tangible Benefits for Learners
Better designed programmes Better learning experience
Programme designs are in place but are yet to run
Students and academic staff
N/A (Programmes are yet to run) though there is a perception that
Institutional Approaches to Curriculum Design
Institutional Story Template
programmes are better designed: Testimony from a CELT nominee
Better programme outcomes Programme designs are in place but are yet to run
Students, staff and the wider University through enhanced progression and retention
N/A (Programmes are yet to run)
Opportunities for better alignment with employer needs and thus more employable students
Technology based infrastructure has been designed and delivered
Students, employers and the wider University through better destinations data
Video demonstration of system (Version 1)
Influence over their experience – students have bolstered opportunities on the designs of their programmes
Technology based infrastructure has been designed and delivered
Students and academic staff
Video demonstration of system (Version 1)
Tangible Benefits for Staff
Supported design process New process includes embedded support
Academic staff Testimony from a CELT nominee
Augmented information in relation to programme design
Creation of the Rough Guide to Curriculum Design
Academic staff
More democratic approach to design process
Approval now requires evidence of wide team engagement in the design process
Academic staff Testimony from a CELT nominee
Better distribution of workload
New system to support design streamlines the workflow – this includes mechanisms for the distribution of tasks
Academic staff – particularly the Programme Director
Video demonstration of system (Version 1)
Business Process Improvements
More responsive/timely approval process
Iterative design process delivered
Academic staff, wider university
Centralisation of documentation – one version of the truth
SharePoint-based document management system integrated into new processes and system
Academic staff, administrative staff
Video demonstration of system (Version 1)
Single point of data entry Use of web form as an alternative to documents allows for multiple documents to be populated from single sources of data
Academic staff, administrative staff
Video demonstration of system (Version 1)
Tangible changes in
Institutional Approaches to Curriculum Design
Institutional Story Template
institutional policy/thinking/communications/approaches to change and continuous improvement
Holistically designed programmes
Technology based infrastructure has been designed and delivered
Academic staff, students and the wider university
Testimony from a CELT nominee
Streamlined process for re-approval
Technology based infrastructure has been designed and delivered
Academic staff, administrative staff, wider university
Video demonstration of system (Version 1)
Streamlined process for minor modifications
Technology based infrastructure has been designed and delivered
Academic staff, administrative staff, wider university
Video demonstration of system (Version 1)
Intangible benefits that you project has delivered e.g. around cultural change
New emphasis on design vs preparation for approval
New processes were approved for piloting by Senate in July 2010
Academic staff, students, employers, wider university
Video commentary; evaluation data (see report)
Wider understanding of what stakeholder engagement means
New model developed HE Sector Stakeholder Engagement Model
Wider and deeper understanding of the process of curriculum design
Wealth of artefacts and presentations have been offered to the sector over the past four years
Academic staff, the JISC, HE Sector
Rough Guide to Curriculum Design; Baseline Review; Design Studio
7. What outputs has your project produced?
The primary output of our work is a SharePoint-based infrastructure that supports curriculum design and approval. Overviews of its functionality can be found by following the links below:
o Diagrammatic representation with video talk through
o Video capture of a demonstration of version 1 of the software
In parallel with producing a system that ‘hosts’ the design and approval process we have created a resource – The Rough Guide to Curriculum Design to help programme teams work through their design activity more effectively. Sections of the Rough Guide are being embedded into the software to give ready access to targeted support at key points in the workflow.
Although a first version of the document has been produced, it has yet to have gone through a peer review process and is thus not yet shareable. However, in order to convey a sense of its focus the Table of Contents is included below:
Institutional Approaches to Curriculum Design
Institutional Story Template
Contents
Section One Page
Rough Guide Design and Approval Timeline 2 Rough Guide Introduction & Overview 5 Rough Guide Approach 8 Whose job is it anyway? Stakeholders and responsibilities 13 The Design Initiation Event (DIE) - Stage One 15 The Design Initiation Event (DIE) - Stage Two 15 Casting the DIE: Activities and outputs – Stage Three 16 Right, know what we need to do, so how do we do it? DIE Activities 18 The DIE is cast, what now? - Stage Four 20 What happens next? Module Development - Stage Five 23 Module Sign Off - Stage Six 27 Programme Specification and Student Handbook - Stage Seven 28 What happens next? – Stage Eight 29
Section Two
Design Challenges 30
Programme level:
Designing and evaluating effective induction
Designing for a widening participation agenda
Designing for disability
Designing for Personal Development Planning (PDP)
Designing for retention and progression
Designing for stronger student engagement
Designing for sustainability
Designing for the needs of international students
Embedding employability in the curriculum
Engaging employers in curriculum design and delivery
Engaging students in curriculum design
Engaging representatives of professional bodies in curriculum design
Seeking out effective practice across the University Module level:
Designing feedback mechanisms
Designing module assessments
Embedding employability in the curriculum
Engaging employers in curriculum design and delivery
Institutional Approaches to Curriculum Design
Institutional Story Template
Engaging students in curriculum design
Engaging with Information Literacy
Engaging with technology to support learning
Seeking out effective practice across the University
Writing good learning outcomes
We have produced a model, well received across the sector, that encourages more structured thinking in relation to conceptions of stakeholder engagement.
We have used our experience with video methods to create a data-protection proforma.
We undertook, and have shared, a multimedia resource that describes the ‘lived experience of curriculum design’ – our baseline review
We have offered a number of dissemination outputs through the course of the project:
Poster 1
Poster 2
Project Blog
Including sections specific to:
Accessibility
Communication
Data protection
Process mapping
Project management
Student Academic Partners
SharePoint
Stakeholders
Voxur units
Twitter Feed
YouTube Channel
Design Studio Page
Presentations at the:
JISC annual conference
JISC online conference
HEA annual conference
Institutional Approaches to Curriculum Design
Institutional Story Template
JISC Learning and Teaching Practice Experts’ Group
International Blended Learning Conference
8. Has delivering the project brought about any unexpected consequences?
Below are extracts from a review paper written by the T-SPARC Project Manager to inform discussion with the University’s Senior Management. It is written in the first person and offers a review of the recently conducted pilots using the new processes for curriculum design and approval:
This review emerges from my perspective of the T-SPARC project as its Project Manager in relation to:
The performance of the SharePoint PDAS (Programme Design and Approval System) as an interface.
The performance of a technology supported workflow for: o Programme Design o Programme Approval
The effectiveness of the various roles that groups and individuals had been allocated.
The performance of the SharePoint PDAS as an interface
The decision to use SharePoint as the primary technology to support this project emerged from a wish to use existing BCU technologies to:
Facilitate easier adoption through familiarity
Work within an environment that could be supported by CICT
To avoid additional recurrent license fees for new products
Although the T-SPARC project initially specified the use of SharePoint in conjunction with Moodle and Mahara, a single interface to support the entire programme design and approval workflow was deemed as being a more elegant solution.
Using SharePoint has required some adaptation of the software leading to a bespoke ‘solution’ – the nature of this type of development has made us subject to the cycles of bugs and fixes that characterise any software development work.
The primary issue users have had with the technology can be summed-up as “The SharePoint interface is not intuitive”. I can certainly empathise with this viewpoint and have experienced that too. The version of the interface we used for our pilots could be described as being ‘flat’ – the initial view gives little clue as to what a user should do and doesn’t seem to reflect a staged approach to programme design.
The project team have responded to emergent user needs in two ways:
The development of changes to the interface to reflect the staged nature of programme design and to improve navigation between the programme, modules, discussions and supporting files. Some language has been changed to make clearer the expectations of the process.
Institutional Approaches to Curriculum Design
Institutional Story Template
The development of a ‘Rough Guide to Curriculum Design’. This document takes programme teams through the programme design sequence, shares the course design philosophy that underpins each stage and (crucially) explains how the work of programme design teams should articulate with the PDAS system.
The performance of a technology enhanced workflow
For programme design:
The PDAS interface gives good support for team-based discussion and broadly, wide-ranging discussions in relation to the wording of learning outcomes and the wording and structuring of assessment has been well facilitated. I believe that this has led to very well thought-through modules. The overall process has led to unprecedented levels of learning and teaching support for programme teams.
Certain beneficial features of the process and the software have not been well exploited. Stakeholder engagement in curriculum design remains relatively low – the involvement of students remains somewhat tokenistic compared to our aspirations, the involvement of employers has been fairly superficial and often done ‘off-system’ (though sometimes to good effect). The involvement of externals in the design process has been patchy but where it has occurred early enough has been impactful.
The nudge feature of the system was over-used leading to saturation of inboxes and inefficiency for some people involved.
For programme approval:
Broadly, the system has not been successful in supporting approval in an effective and efficient manner. Although there is no doubt that some of the interface issues have inhibited engagement, other factors also underpin this. Briefings for those supporting the approval process were inadequate – this meant that conflicting conceptions on the nature and purpose of programme approval were not resolved and those who had responsibility for supporting approval did not understand their role as required by the system or were not able to carry them out.
Specifically:
The staged nature of certain sign-offs (philosophy and aims) was not well appreciated by some Programme Directors and this delayed the development of modules.
Some externals were not identified early enough in the process and as a consequence were unable to participate in the design of the course. This led to expectations for the PDAS system to perform outside of specifications at the approval stage.
The PDAS system builds definitive documentation as a consequence of approval not for approval. The absence of the expected set of Word documents prior to approval led to frustration for some and led to their limited engagement in the intended process.
Of course problems, where they arose, were addressed through other means such as face-to-face meetings, email correspondence and telephone conversations – but the system itself, as used by the users, did not offer the easy solution for them that we had hoped for.
The effectiveness of the various roles that groups and individuals had been allocated
Institutional Approaches to Curriculum Design
Institutional Story Template
Programme teams:
For the most part Programme Directors did a good job of engaging with the design and approval process. Other programme team members were less engaged and in some cases were much less well informed than their respective Programme Directors.
Academic Registry representatives:
AR Representatives did not contribute to the design processes directly, though they did offer support in response to direct questions posed by telephone or email.
AR Representatives did not embrace the opportunity to engage with the online representations of designed programmes and instead waited for definitive (MS Word) documents to be built before offering a view.
CELT Representatives:
CELT Representatives engaged fully with the new process across all pilots and offered a very considerable level of support to programme teams. On the PDAS system, all modules were subjected to thorough scrutiny resulting in changes to many modules – particularly in relation to the wording of learning outcomes and the structuring of assessments.
The building of close relationships with programme teams, catalysed by the discourse supported by the new process, led to a good deal of adjunct staff development activity. These enhanced relationships with programme teams have persisted beyond the programme design activity and has led to transformed levels of engagement with academic staff development by those teams.
External Representatives:
External Representatives undertook some influential activity on one of the programmes – shaping the programme philosophy and contributing to the making explicit of assessment regimes. The other programmes saw much less external engagement – mainly because there were long delays in getting them on the PDAS system. For some of the programmes, externals were not added onto the system at all, necessitating all documents to be built by the system and then (manually) emailed to the external.
Students:
Students were barely used as part of the design process. Although some programme teams collected some student data, it was never clear as to how the data collected explicitly influenced design. Truly empowered engagement by students in a way that might shape the design of the programme did not occur to a significant extent.
Lead Academic Assessors:
Engagement on the system by Lead Academic Assessors fell well short of the aspirations of the original design brief. LAAs simply did not have enough time to fully commit to the demands of the role as defined in this pilot activity. Their engagement was frustrated by an unintuitive interface.
Conceptions of what ‘artefacts’ were to be approved varied across LAAs with varying tolerances to ‘signing-off’ on modules based on the data held in the web-forms. A fundamental mismatch in relation to expectations was revealed here. At least one LAA had anticipated that
Institutional Approaches to Curriculum Design
Institutional Story Template
the system could and should yield documentation for subsequent review rather than see documentation as cascading from the act of ‘approving’ web forms. The PDAS system has been actively designed for approval to be conferred on a completely different set of artefacts than is the case with the extant processes.
T-SPARC Project Team:
The T-SPARC project team offered a good, but ultimately inadequate, level of support for those involved in the process. Difficulties with securing access to some groups (Externals, LAAs, Registry) meant that briefings were inadequate and misunderstandings on the nature of the process were not addressed. I think there was a tacit assumption that experienced panel chairs had expertise in facilitating programme approval activity and were thus fully prepared as Lead Academic Assessors. I think it was further assumed that support would only be needed in relation to interface familiarisation. I think there were large differences in philosophical positions between one of the Lead Academic Assessors and the T-SPARC Team and that there was insufficient opportunity to deal with these tensions.
Reflective commentary: The review offered above shares some specific tensions that emerged from us using the new system with live pilots. We mustn’t forget that this was the first time the technology-based system, the new curriculum design processes and the new approach to programme approval had been tried. It was inevitable that tensions would emerge – indeed, revealing those tensions was one of the primary reasons for conducting pilots. One of the most disappointing features of the pilots was the way in which stakeholder engagement in curriculum design fell well below the potential afforded by the infrastructure we put in place. Although we saw a step-change in relation to engagement with CELT and across a good proportion of the course team, student and employer engagement remained at a stubbornly low level. However, we do now have our Rough Guide to Curriculum Design and it does offer some very direct support in relation to how to carry out effective stakeholder engagement and the guide will be available for the next series of pilots.
9. How will the project be developed further/sustained?
Given the tensions described above, the new approaches and system will not be mainstreamed this coming academic year; instead we have Directorate support to undertake a second stage of pilots. This is a sensible decision and will allow us to continue to improve both our overall approach to design and approval and the new technology-based system that supports it. It is a mark of the depth of support that we have that the T-SPARC Project Support Officer, up until now employed on a fixed term contract, has just been offered a permanent contract to support the ongoing work and subsequent rollout. All being well with our second round of pilots, these new approaches will begin a rollout across the institution in the 2013/2014 academic years.
10. Summary and Reflection
Institutional Approaches to Curriculum Design
Institutional Story Template
If we consider the four project aims, declared in the Project Initiation Document and not changed since, the T-SPARC project HAS been successful: Project Aim 1: Informing programme design activity through the enhanced provision of
pertinent information. The Rough Guide to Curriculum Design is a comprehensive resource that addresses this very aim. Combined with our approach to embed sections directly in the PDAS (SharePoint) system, our reuse of legacy multimedia artefacts relating to programme design, bolstered academic staff development activity and parallel responses from our Marketing and Academic Planning departments, this aim has been well served.
Project Aim 2: Redesign the ICT infrastructure that underpins the workflow of the
curriculum design and programme approval process. We have invested much of our effort in designing and delivering a system that manages the workflow of curriculum design and approval, and although we need to do more (and already have) in optimising the interface and briefing stakeholders, the system does allow for collaborative working; does facilitate communication on SharePoint and through into people’s ‘inboxes’; does ensure data is only inputted once; does ensure there is only one version of the truth; does host and archive evidence of design as a process; and does dynamically build and host definitive documentation. Project Aim 3: Electronic support for course team dialogue during their programme
design activity. The PDAS system boasts a fully functioning discussion engine that supports discussion at the programme, module and sub module level (for example dedicated forums for learning outcomes, assessment etc), these discussions articulate fully with active directory services allowing for users to subscribe to threads or to ‘nudge’ users to ask them to engage. Although the cultural practices around the use of this technology in relation to wide stakeholder engagement need to develop – the infrastructure does delivers to specification. Project Aim 4: Electronic representation of programmes and underpinning evidence at
(and leading up to) the point of approval. Again, the PDAS system fully delivers against this specification. All discussions and uploaded files such as video and audio clips, links to external sites, minutes of meetings are archived on the system and effectively create a robust audit trail of the design process. Furthermore, the design product – the definitive documentation is available to all involved in dynamic draft form as a series of web forms and as a final version in MSWord or PDF format once sign-off has occurred. Of course, with a project such as this, the meeting of design specifications is not enough. Success is assessed in relation to the declared perceptions of the stakeholders/users and by that measure we have some way to go. Scrutiny of the evaluation report (submitted in parallel with this document) shows that when users (from eight of the nine pilot programmes) were asked about the most significant factors (positive or negative) that impacted on their lived experience of the pilot activity, four of the top five factors (from forty-two) were ‘negative’. They were:
1. Clear (human) facilitation of the whole process was essential 2. Immaturity of the technology
Institutional Approaches to Curriculum Design
Institutional Story Template
3. Immaturity of the process
4. Discussions around design were valuable
5. Differences in conception as to what ‘approval’ was all about
Factors 1-3 emerge as a function of the pilot nature – we had never facilitated such a process before and underestimated the degree to which programme teams needed support. On reflection, we had made the assumption that programme teams knew how to design curricula and would ‘just’ use the system to do it. However, we had raised the expectation in relation to effective design to such an extent that the programme teams felt lost as to how to meet these new demands and needed much more structure and guidance. Factor 4 was the one positive factor of the top five – programme teams felt they had collaborated much more fully within the team and with CELT. They had many more conversations, these conversations led to more robust design and overall the teams felt that they had a very well thought through course as a consequence. Factor 5 is an outlier and has prominence because all members from one course team thought it was the most important factor. This related to their expectations of a light touch process because they felt their course did not need to change substantially; in fact, the level of scrutiny the new processes brought to bear on the course meant that those conferring approval did push for more change than the programme team thought their course warranted. Factors 1, 2, 3 and 5 can all be addressed through a much more robust approach to briefing programme teams prior to their embarking on this new process. For the second set of pilots to be run in 2012/2013, it has already been agreed that the briefing event will be a full day. It has also been agreed that student representatives (where a programme is being reapproved) will also be invited to the briefing event; we hope their inclusion will ‘pump prime’ their engagement in the redesign of their course.
Lessons learned – reflections of the Project Support Officer:
Technical lessons:
Don’t let the system allow people to make mistakes:
Users have stated in some instances that they feel the system needs to be more user-friendly, or that they need to be better trained on how to use it. Some users have expressed concern that they are attempting to complete actions such as attaching evidence of engagement to module documentation, or posting comments to discussion forums, and they are unable to complete the task. It is felt that with V2 of the PDAS system, it is essential to remove the ability for users to make these mistakes by streamlining the interface in such a way that the option to fail is not available.
Too much on-screen clutter– simplify the interface:
Users have reported that they feel the interface is clunky and unintuitive to navigate. The removal of additional unused buttons is on the development list but they are yet to be removed. Three members of the pilot programme teams have volunteered to work with project staff and internal PDAS developers over the summer to facilitate streamlining of the interface.
System alterations, save them up, manage stakeholders:
Many alterations are suggested on a daily basis from users. Project staff seek to collate these suggestions, look at how they integrate with project philosophy (don’t clash with ‘stuff’ we are trying
Institutional Approaches to Curriculum Design
Institutional Story Template
to encourage users to do, or not to do) and prioritise the ones that get the green light into an order for CICT staff to develop. Deployments of major changes are saved until after the pilot phase so as not to confuse users with interface updates while they are trying to complete their programme design work.
Human Process Lessons
Managing expectations early in the process:
More emphasis is needed in the early stages of supporting programme teams, specifically in relation to the philosophy of what they are doing, not just how they should do it. The problem space of curriculum design needs to be fully understood before the PDAS solution will be accepted/embraced.
Users struggled to differentiate between the new process and the new system:
A number of users seemed to conflate the use of the system, with the new iterative design process. New demands in relation to doing a ‘better design job’ were seen as being demands of the technical system - a failure of the system to allow them to do what they wanted. In fact, the system was performing to specification making it difficult for old approaches to curriculum design to be undertaken. Although the new approaches emerged from our stakeholder groups, these particular people had not been part of that engagement process – they were living with somebody else’s solution to a problem they hadn’t fully explored!
Briefings for users are critical and need to be done by the right people:
All user groups need to be appropriately briefed on new processes and systems, for the piloting, the various stakeholder groups had their briefings tailored to their specific needs. For reasons relating to ownership of systems and processes, it is felt that certain stakeholder groups were not briefed as well as they could have been had the project team been involved in the briefings
Nudges (SharePoint notifications) are not always used appropriately:
Staff need training on how to use the nudge functionality appropriately – its not enough to build the system and show them how to use it, they need to understand how the system relates to the process and how their actions have an effect on all other stakeholders.
Lead Academic assessors don’t have the time to engage in the design process:
Where the LAAs were expected to act in a formative development capacity, that aspiration has proven to be unrealistic. As we move into phase 2 of piloting it is predicted that the CELT nominee will take more responsibility for monitoring of the development of the programme and design process, and act in a reporting capacity to the LAA should they feel any matters need clarification. LAAs would be asked to sign off on the design process at certain fixed points.
Concluding remarks of the Project Manager:
Overall, the technical parts of the process have been broadly successful and we have delivered a system and process that successfully addresses all of the project aims. Now that this infrastructure is in place, we must support the change in cultural practice to align with new expectations and opportunities. After all, as we all know, cultural change does not always move at the same pace as technical development!
