Available online at www.sciencedirect.com

www.elsevier.com/locate/ijproman

International Journal of Project Management 26 (2008) 251–257

Teaching methods for international R&D project management

Blazenka Divjak, Sandra Katarina Kukec *

University of Zagreb, Faculty of Organization and Informatics, Pavlinska 2, 42000 Varazdin, Croatia

Received 24 January 2008; accepted 24 January 2008

Abstract

This paper presents a case study of teaching project management in Croatia in two different learning environments and points out twomain paradigms upon which both courses are designed. These two paradigms are most effective learning is working in real-life situations

and teach them what you promised and students will respond with effort. We point out similarities in teaching methods but also the dif-ferences in motivation and achieved levels of knowledge and skills. Implementation of e-learning has been made according to the setobjectives for each course. Follow up of students’ satisfaction level and personal development has regularly been assessed.� 2008 Published by Elsevier Ltd and IPMA.

Keywords: International projects; R&D projects; Teaching methods; Learning outcomes

1. Introduction

We write this paper from the point of view of teachingthe following target groups: professionals studying in lifelong learning programs and students at postgraduate stud-ies. The accent is on teaching these target groups interna-tional R&D project management.

At the beginning we set two paradigms for teaching pro-ject management.

1. Most effective learning is working in real-life situations

Group work on an actual real-life project is the mainteaching instrument since groups act as project teams andthe project ideas are close to their field of professionalinterest.

Additionally, playing different roles in the project lifecycle and learning to cross disciplinary borders enable stu-dents to gain self-assurance in project management. Theroles which one can take on in a project can be in a wide

0263-7863/$34.00 � 2008 Published by Elsevier Ltd and IPMA.

doi:10.1016/j.ijproman.2008.01.003

* Corresponding author. Tel.: +385 42390825; fax: +385 42201162.E-mail addresses: blazenka.divjak@foi.hr (B. Divjak), sandra.kukec@

foi.hr (S.K. Kukec).

range from strategic decision makers, project leaders andscientific coordinators to project managers, team members,reviewers or sponsors. Inclusion of interdisciplinary knowl-edge is needed, like data mining, decision making, financeand accounting, mathematical graph theory, modelling,ICT, etc.

2. Teach them what you promised and students will respond

with effort

This includes defining very clear learning outcomes andintroducing quality culture in the whole process of teachingand learning. The responsibility for the quality of thecourse is shared between teachers and students. What ismore, learning outcomes of the course have to be synchro-nized with the learning outcomes of the whole program andthe assessment must be proven to measure the learningoutcomes.

Additionally, there is also a constant follow up on thesatisfaction of participants through questionnaires, pub-lishing of evaluation results and concrete steps takentowards improvement.

Teaching must have a strong support in ICT, whichmust be available constantly. Additionally to the teaching

Analysing

Applying

Understanding

Remembering

Creating

Evaluating

Fig. 1. New version of Bloom taxonomy.

252 B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257

program an on-line course is offered to the students so theycan review the lectures taken on the course. The on-line

course is structured as a self-paced course and it providesfurtherance of knowledge and skills since it is accompaniedwith case studies and assessment. Lectures for postgradu-ate students are provided in a form of blended learningand for the lectures for project management training work-shops an on-line course has been created as an after-courseactivity, which the workshop participants can complete tostrengthen their knowledge even after the workshop hasended.

The lectures have to be organized in order to supportgroup work activities. This form of teaching requires smal-ler teaching groups of up to 25 participants that form smal-ler project groups of 4–5 students that work together ontheir project. The support to the groups is provided fromthe lecturer and at least one or ideally two assistants whoare available to groups during the lectures and afterwards.Through group work the students develop their interper-sonal skills.

The added value for each participant that this form of lifelong learning brings, besides the knowledge about projectmanagement, can be the benefit of both personal and insti-tutional partnership between the participants in one lecturegroup.

Finally, we can identify some crucial problems thatoccur when trying to implement project management at uni-

versities. Firstly the scientists are used to thinking ‘‘scientif-ically” instead of ‘‘project oriented” thinking. Secondly, theuniversity management has to be convinced that theyshould change their organization from a traditionally orga-nized functional organization to a more adaptable projectorganization.

2. Problems and challenges

The Higher Education has become part of a global shiftto a new way of creating and using knowledge. The newway is focused on solving problems and is sensitive to cus-tomers needs. It strives for quantity as well as quality. Itcuts across interdisciplinary boundaries. It is enlivened byapparently infinite quantities of instantly accessible infor-mation. Accountability, QA, league tables and perfor-mance indicators have become permanent entities in theHE lexicon [1].

Traditionally professors and scientists in Croatia areused to perform scientific and teaching activities and notresearch projects. Therefore they are not achieving highresults in international R&D project competition arena.

We tried to find out what the critical factors for success-ful participation in international R&D projects are. Inorder to answer that question, in 2006 we conducted a sur-vey among the existing and potential project coordinatorsand partners at 30 faculties at the University of Zagreb.The complete results of the survey can be found on theweb page of the Reference Centre for E-projects in Croatia[2]. Here we point out that the survey shows that the three

main obstacles for raising performance in internationalR&D projects are the following: lack of project manage-ment skills; legal obstacles and obstacles in the accountingsystem; and lack of administrative support to scientists.

The survey also provides an answer to the questionWhich approach would mostly help raise absorption capacityfor international projects in Croatia. The highest rankedwere the following: organizing training workshops in inter-national R&D project management, web portal with all thenecessary information and case studies, and financial(accounting) support for preparing and managing projects.

In order to solve some of the existing problems we havemade the following steps. The mentioned portal, ReferenceCentre for E-projects in Croatia [2], maintained by our pro-ject team, tries to provide accurate and up to date informa-tion in Croatian and in English. Further, our project teamat the Faculty of Organization and Informatics (FOI) hasconducted more then 30 training workshops in interna-tional project management for almost 500 participantsfrom 5 Croatian universities, SMEs, public institutions,NGOs and companies. At the same time several coursesin project management at different study programmes havebeen introduced.

Furthermore, several institutions of higher education inCroatia have incorporated project management course intheir study programmes at different levels in the last twoyears.

Here we will describe our experiences from the projectmanagement training workshops and the course entitled‘‘Project cycles in research and development” which is setat the doctoral (PhD) study of information science at FOI.

3. Learning outcomes, assessment methods and student

workload

Paradigm Teach them what you promised and students

will respond with effort includes defining very clear learningoutcomes. In order to define learning outcomes for thecourse at the doctoral study we used Bloom taxonomy [3]and synchronized them with the learning outcomes of thewhole doctoral program which is research oriented. Theoutcomes have been identified at all six levels of the Bloomtaxonomy. The modified (new) version of Bloom taxonomyis shown in the Fig. 1 taken from [4]

B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257 253

Here are the learning outcomes:

� Comprehension of the role and techniques of projectapproach in science and development, and the under-standing of the characteristics of scientific research (risksinvolved in scientific project, importance of researchers’inventiveness, international character)� Understanding and application of project management

methods for application and management of R&D(research and development) projects� Analysis and identification of project success criteria in

each phase of the project cycle� Synthesis, design and presentation of main components

in an international R&D project application and themastery of required terminology (mostly in English)� Development of team work skills, but also development

of leadership skills and partially skills essential for pro-ject documentation management� Evaluation of R&D projects in respect to the application

of project management methods and, in a reasonablyreduced range, in respect to scientific relevance of theproposed research.

For the course ‘‘Project cycle in research and develop-ment” at the doctoral study of Information sciences wedeveloped a student-centred learning model (SCL) andwe use a virtual learning environment (VLE) open sourceMoodle to support the model. SCL model has been provenas a very beneficial one for students in the doctoral study,which also supports the conclusion of Allan G. [5] in hisarticle: ‘‘. . . SCL methods lead to deeper understandingand learners are more satisfied.” We also collected empiri-cal evidence from our work with students to support thatstatement.

For example there were 12 students in the doctoral studygroup in academic year 2006/07 and we ask them to eval-uate the teaching and learning methods on the scale from1 to 5, where 1 is very poor and 5 is excellent. There were10 answers out of 12 with the mark 5 (excellent) and 2answers out of 12 with the mark 4 (very good).

Quality culture in the whole process of teaching andlearning proclaims that the assessment must be proven tomeasure the set learning outcomes and we devoted consid-erable time and effort to prepare, discuss and evaluate that.In the following table we map learning outcomes andassessment methods. Some learning outcomes are assessedby several assessment methods and the level of achievementis verified at each assessment event according to Bloomtaxonomy.

In the project management training workshop for life-long learners there is no need to prepare such sophisticatedstructure of learning outcomes and no need to map themwith assessment methods. What is important is that thegoals have to be very clearly set and communicated to par-ticipants at the beginning of the workshop. In most casesthe goals and themes have been very precisely preparedin advance with the beneficiary groups or interested parties

that requested the workshop to be held. Since these groupsare very heterogeneous and some aspects cannot be fore-seen in advance, several minor changes are often necessaryand can be built in the goals on the spot according to thesuggestions of the participants. On the one hand, thoseinterventions motivate the participants to collaborate morefreely with trainers and other participants and give themfeeling of certain control of teaching and learning processand on the other hand, trainers collect very valuable infor-mation about participants’ background knowledge, moti-vation to participate and opportunity to make creativechange of content and methods to meet the participantsgoals. From our experience from around 50 workshops,participants highly appreciate that approach and what ismore this approach is very much in accordance with theproclaimed SCL method.

The doctoral study as a whole is furnished with creditsand The European Credit Transfer System (ECTS) is used.In this system 1 ECTS covers 25–30 h of work of an ‘‘aver-age” student. For completing the course ‘‘Project cycles inresearch and development”, students earn 9 ECTS points.Here is the calculation of estimated student workloadaccording to students’ activities in the frame of the course.

Student workload has been verified through the diary ofactivities that each student has to keep in the Moodle

throughout the course duration and exanimation period.In the last academic year we collected 12 diaries and eachconsisting 300–500 words. Students keep record of theirworkload and also analyze freely their expectations andachievements in the course. The estimated workload of 9ECTS was verified through diaries.

There is also a constant follow up on the satisfaction ofstudents through anonymous questionnaires and informalinterviews, publication of evaluation results and specificsteps taken towards course improvement. In the question-naire given to the students at the end of the course 10out of 12 students answered that the measurement wasexcellent and two that it was very good.

In the case of training workshops, unfortunately, we donot attached credits but instead of credits we issue certifi-cates of completion with number of working hours whichparticipants put into their portfolio. There has been an ini-tiative to change that since many participants in the train-ing workshop would like to use that informal education aspart of their specialist’s postgraduate study. Namely, inCroatian educational system there are two kinds of post-graduate formal education – doctoral study which isresearch oriented and specialist study, where students con-centrate on high level professional problems.

4. Effective learning through real-life activities

Group work on an actual real-life project is the mainteaching instrument, since groups act as project teamsand the project ideas are close to their field of professionalinterest. At the beginning of the course groups are formedaccording to research niche defined by each student. This

254 B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257

year we use Work programme for ICT in FP7 as a basis forfinding a niche. In that process of analysis, students alsodevelop strategic and operational planning abilities.

Additionally, playing different roles in the project lifecycle and learning to cross disciplinary borders enables stu-dents to gain self-assurance in project management. Theroles which one can take on in a project can be in a widerange from strategic decision makers, project leaders andscientific coordinators to project managers, team members,reviewers or sponsors. Inclusion of interdisciplinary knowl-edge like data mining, decision making, finance andaccounting, mathematical graph theory, modelling, ICT,etc., is also needed and of essential value to the projectteam.

Therefore, we introduced several team work activitieswhich are hands on.

The activities resemble the following:

� find your own niche in strategic objectives of differentR&D programmes,� define your project idea and analyze it with an appropri-

ate tool (for example SWOT analysis, problem tree,needs analysis, etc.),� define objectives, deliverables, activities, indicators,

risks, etc. by using Logical Framework Matrix (LFM),� design the major work packages and provide planning

(PERT, Gantt, etc.) for them,� construct the overall budget and distribute resources

according to objectives and deliverables,� complete the project application form and foresee the

management of the project,� evaluate the project according to the actual evaluation

rules and procedures and negotiate an evaluation reportwith other evaluators, etc.

The above mentioned activities are ‘‘building bricks”

that build the course. Some of these activities are alsoincorporated into training workshops depending on theparticipants’ preferences and goals of the workshop. In thiscase there are more limitation in time, since the trainingsare intensive and short cycled. On the other side, if we wantto keep attention and involvement of participants wehave to use teaching methods that engage participants. Inorder to do so we use all of the above mentioned activitiesaccept completion of application and evaluation of pro-jects. Instead of that we focused more on previously men-tioned activities and exercises. In evaluation forms, whichwe collect after each workshop, such hands on activitiesare always very highly graded. Additionally we provideon spot help and guidance to groups, which means, thatbesides the main trainer there are always one or two assis-tants that help the trainer in advising and coordinatinggroup activities. There are also some problems we encoun-tered in group work activity. In their comments in evalua-tion sheets, workshop participants usually mentioned thattimeframe for such group work must be longer since theyprofit most from such activity.

At the doctoral course we also use the activity of analyz-ing a given research article, which is published in a journalwith good citations, in order to enhance student under-standing of scientific methods and presentation of theresearch project results. Additionally students receive aninsight into current research in the field of projectmanagement.

5. ICT support for teaching and learning project

management

Our goal is to introduce new technology in teaching andlearning of project management but in order to enhance theprocess of learning and not just to belong to the trend.Therefore, in different teaching environments (formal andinformal) we analyze the needs for information and com-munication technology (ICT) and use those ICT toolswhich, according to our experience and relevant literature(for example [6–8]) maximise the learning effects. (seeTables 1 and 2).

Generally speaking, e-learning or computer enhancedlearning can be defined as a learning process enhanced byICT. It supports collecting, analyzing and applying infor-mation appropriately and comprises different teachingmethods, for example information management, creativethinking, critical thinking, problem solving and collabora-tive learning [9].

We distinguish four alternatives of e-leaning accordingto [10] and they are given in Table 3.

The process of strategic planning of e-learning imple-mentation is described in [11,12].

We use a virtual learning environment (VLE) opensource Moodle, at the doctoral study and teach the courseas blended (hybrid) learning, combining classroom teach-ing and face to face collaboration with learning, collabora-tion and interoperability supported by LMS.

In the training workshops we teach with the support ofICT but we encourage workshop participants to use openself-paced courses to deepen and broaden their knowledgeand skills.

Additionally, the self-paced on-line course is providedwithout mentoring system as an open source of informa-tion and training on our web domain without restrictionfor all interested users. We evaluate the self-paced coursein three groups, each group had 15 trainees. Traineespointed out that the materials used and navigation inself-paced course are good enough, but still think that for-ums with trainers and mentoring system will improve theachievements, since content is rather demanding. In Table4 we summarize the implementation of e-learning in differ-ent learning environments.

Let us emphasise that we regularly assess the quality oflearning and e-learning, since studies show that ‘‘. . .drop-out students (non-completers) reported to have signifi-cantly lower satisfaction with e-learning than studentswho successfully completed (completers or persistent stu-dents) the same e-learning courses.” [13].

Table 1Map of learning outcomes and assessment methods

Learning outcome Assessment method

Comprehension of the role and techniques of project approach in science and development, and theunderstanding of the characteristics of scientific research

� Oral exam, application presentation and‘‘defence”

� Project evaluation based on set rules andregulations

Understanding and application of project management methods for application and management ofR&D (research and development) projects

� Oral exam, application presentation and‘‘defence”

� Team work on specific methods during andbetween lectures� Project application writing� Presentation of theoretical and practical aspects

of each application sectionAnalysis and identification of project success criteria in each phase of the project cycle � Oral exam, application presentation and

‘‘defence”

� Team work on specific methods during andbetween lectures� Writing the project application� Presentation of theoretical and practical aspects

of each application section� Project evaluation based on set rules and

regulationsSynthesis, design and presentation of main components in an international R&D project

application and the mastery of required terminology (mostly in English)� Writing the project application with summary in

English� Oral exam, application presentation and

‘‘defence” of proposal� Presentation of theoretical and practical aspects

of each application section� Construction of on-line course on FP7 project

application in Moodle� Project evaluation procedure with summary in

EnglishDevelopment of team work skills, but also development of leadership skills and partially skills

essential for project documentation management� Team work on specific methods during and

between lectures� Writing the project application� Team work: self-evaluation reports and report

on workloadEvaluation of R&D projects in respect to the application of project management methods and, in a

reasonably reduced range, in respect to scientific relevance of the proposed research� Project evaluation procedure based on set rules

and regulations

Table 2Student workload estimated in hours

Activity Studentworkload inhours

Lectures 30Scientific paper analysis, essay 30Design of a project application

Research of information sources and previous project

applications, writing the application, team work

60

Evaluation of projects/On-line course design 30Research/analysis/synthesis Presentation, literature for

the oral exam90

Total: 9 ECTS 240

B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257 255

For example, in the last academic year, doctoral stu-dents, almost unanimously evaluated with excellent gradesthe material distribution through VLE Moodle. They alsoappreciated the contribution of e-learning towards commu-nication between students and teacher. Furthermore, they

were positive towards contribution of VLE to collabora-tion among students mediated by e-learning (7 answersfrom the questionnaire were that communication wasexcellent (grade 5), 2 answers very good (grade 4) and 3answers good (grade 3)) but the main improvement theysaw in the teacher accessibility (11 excellent (grade 5)answers and 1 very good (grade 4)). Finally, they foundthat VLE contributed to the transparency of work andachievement of learning outcome. Minor problems wereaddressed by very few students that did not have enoughpre-experience with ICT.

One of the students put in her learning diary that shelearned a lot about collaboration participation in thecourse, even though she has been in formal education foryears. Another interesting remark in another student’sdiary was that the student found international project man-agement very interdisciplinary and therefore very good fordemonstration of useful integration of different theories,techniques and tools, that students learn before at under-graduate and graduate level.

Table 3E-learning alternatives adopted according to [10]

Alternative Description Technologies

Face to face teaching Teaching (mostly lectures) in classroom � Not used, except for example text processor (Word) for thepreparation of teaching

ICT supported teachingand learning

Technology is used mostly to improve traditional teachingmethods

� Presentations (PowerPoint)� Multimedia CD-ROM-s� Web portal for courses with hypermedia (courseware)� Testing programmes (self-evaluation quizzes and knowl-

edge tests)� E-mail and mailing lists� Forum

Blended, (mixed,hybrid) learning

Combination of traditional teaching in classrooms andtechnology enhanced teaching

� LMS-Learning Management Systems� CBT/WBT-Computer/Web Based Training� Videoconference

Fully online learning Teaching and learning is entirely technology enhanced;there is no face to face teaching (lectures)

� Course (programmes, workshops, tutorials) are presentedthrough Internet (as Web portal or LMS)� Videoconference

Table 4Summary of the implementation of e-learning in different learningenvironments

Level/type oftraining

Alternative of e-learningapplied

Evaluation fromstudents

Workshop ICT supported Very positivePostgraduate

doctoral studyBlended Very positive

Self training Fully on-line Positive

256 B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257

6. Similarities and differences between a workshop and a

course

Training for international project management, pro-vided through one day or several day workshops has a dif-ferent scope, goals and motivation of trainers and students.Participants of workshops usually form more heteroge-neous group of learners than doctoral students do and theirmotivation is more intrinsically and practically oriented.They are usually not very interested in activities on scien-tific paper analysis and scientific methods, but would preferthe real practical knowledge and experience transfer.

Furthermore, they are prepared to take in more specificproject details and facts since they already have some pre-knowledge of project management or experiences in real-life projects and what is even more important they havetheir own project idea, which they want to realize afterthe course. The potential project that would rise out fromtheir project idea motivates them strongly to be concen-trated and highly interactive in the workshop.

At the same time generic (transversal) skills, which theyhave to develop, are essentially similar to those expectedfrom PhD students. However the level (according toBloom) they would achieve in a two day workshop is con-siderably lower than when PhD course is in question.Therefore it makes sense to continue their education infor-mally with the help of on-line self-paced course. We alsooffer post workshop help desk in order to answer addi-tional dilemmas that workshop participants could have.

Additionally, workshops have been proven as good oppor-tunities to find partners and collaborators for projects.

The size of the groups in the training courses and thePhD course are quite similar and vary from 12 to 25 stu-dents. That is one of the reasons we can successfully incor-porate team work activities, case studies and achieve highlevel of interactivity, collaboration, personalized approachthat are the key factors for student-centred learning. Addi-tional communication channel for student to student andstudent to teacher (trainer) collaboration is open throughLearning Management System (LMS). This communica-tion is proven rather successful in terms of quality of learn-ing but also very time demanding on the teacher’s side.

Although there are some differences between teaching apostgraduate study in project management and holdingworkshops on the same topic, some of the teaching andlearning methods can be applied to both teaching groupsas results of paradigms set at the beginning.

7. Conclusion

Following two paradigms (Most effective learning is

working in real-life situations and teach them what you

promised and students will respond with effort) we developedtwo courses on project management, one in a formal andthe other in an informal learning environment.

Both of them are furnished with considerable amount ofICT support and specially designed fully on-line course.We found out that students in both environments speciallyappreciate interactivity and hands on activities, even iftheir motivation for completing set tasks is quite different.Special emphasis is put on development of generic skillsand team work and in that respect both groups need initialencouragement from the teacher (trainer). Introduction ofe-learning has been made according to the objectives wewould like to achieve. Students’ satisfaction and personaldevelopment is being regularly assessed.

Further development can be made in international col-laborative work during international project management

B. Divjak, S.K. Kukec / International Journal of Project Management 26 (2008) 251–257 257

course throughout cooperation on PhD study level. In thatrespect students will have first hand experience in skills thatare important like international partnership, cultural differ-ences, etc. It can also be enabled through e-learning toolsand methods.

References

[1] Ramsden P. Learning to teach in higher education. London and NewYork: Routledge; 2003.

[2] Reference Centre for E-projects in Croatia www.projekti.hr Univer-sity of Zagreb, Faculty of Organization and Informatics (19 January2007).

[3] Bloom BS, Krathwohl DR. Taxonomy of educational objectives: theclassification of educational goals, by a committee of college anduniversity examiners. Handbook I cognitive domain. New York:Longmans, Green; 1956.

[4] Anderson LW, Krathwohl DR. A taxonomy for learning, teaching,and assessing: a revision of bloom’s taxonomy of educationalobjectives. New York: Longman; 2001.

[5] Allan G. An enquiry to student-centred learning. Project managementconference: excellence in teaching, learning and assessment. Bourne-mouth University; 2003.

[6] Rodriguez D, Sicilia MA, Cuadrado-Gallego JJ, Pfahl D. E-Learningin project management using simulation models: a case study basedon the replication of an experiment. IEEE T Ed 2006;49(4):451–63.

[7] Motschnig-Pitrik R, Mallich K. Effects of person-centered attitudeson professional and social competence in a blended learningparadigm. Educ Technol Soc 2004;7(4):176–92.

[8] Levy Y. Comparing dropouts and persistence in e-learning courses.Comput Educ 2007;48(2):185–204.

[9] Bates AW. Managing technology change: strategies for college anduniversity leaders. San Francisco: Jossey-Bass Publishers; 2000.

[10] Zemsky R, Massy W. Thwarted innovation: what happened toe-learning and why. A final report for the weatherstation project ofthe learning alliance at the University of Pennsylvania in cooperationwith the Thomson Corporation; June 2004. p. 51.http://www.irhe.upenn.edu/Docs/Jun2004/ThwartedInnovation.pdf.

[11] Begicevic N, Divjak B, Hunjak T. Decision making model forstrategic planning of e-Learning implementation. Fourth EDENresearch workshop, research into online distance education ande-learning: making the difference. Castelldefels: Spain; 2006.

[12] Begicevic N, Divjak B. Validation of theoretical model for decisionmaking about e-learning implementation. J Inform Organ Sci2006;30(2):171–84.

[13] Levy Y. Comparing dropouts and persistence in e-learning courses.Comp & Educ 2007;48:185–204.