1 Shaping the future of psychology through developing and assessing graduate attributes using collaborative learning

PROJECT OUTCOMES AND RATIONALE Project Outcomes The outcomes of this project are twofold: (1) The refinement and validation of a self-assessment and peer-assessment measure of psychology graduate attributes that can be used for formative and summative assessment in undergraduate psychology courses around Australia, and (2) The development of a semi-structured problem-based collaborative learning approach to tutorials that can assist in the development and assessment of psychology graduate attributes. By achieving these outcomes, this innovative project will provide both a validated assessment tool and a highly effective inquiry-based approach to learning that will transform the assessment and development of graduate attributes in psychology undergraduate students Australia-wide. The project directly targets the “Assessment and Promotion of Student Learning” ALTC priority with particular emphasis on: (1) developmental, diagnostic and summative assessment and feedback to students, (2) innovative models of assessment and reporting student achievement developments and (3) building on ALTC completed projects related to student assessment. This project makes a significant contribution to the scholarship of graduate attributes in Australian higher education. It is vital to develop an evidence base and rigorous assessment tools in the domain of graduate attributes as Australian universities and professional bodies seek to develop outcomes-based quality assurance and accreditation processes for higher education. The recent historic changes to the accreditation and competency standards of psychologists in Australia have brought into sharp focus the importance of developing undergraduate psychology courses that: (1) integrate the assessment of graduate attributes in the curriculum, and (2) incorporate assessment practices and learning activities designed to foster students’ development of graduate attributes. However, no psychometrically valid self assessment and peer assessment measures of psychology graduate attributes exist. Thus, undergraduate psychology courses have no clear method of tracking student development and achievement of graduate attributes. Furthermore, no rigorous and systematic evaluation of curriculum design has been undertaken to determine the effectiveness of teaching approaches in fostering students’ development of graduate attributes. This project directly addresses these gaps and builds on previous ALTC funded projects on curriculum design of undergraduate psychology courses and graduate attributes2-4. The project is also informed by two large sample pilot longitudinal studies (see headings “Preliminary findings on self and peer assessment of graduate attributes” and “Preliminary PBL-CLA research” under the section titled “Approach”). Therefore, the team has piloted the measures, learning materials, data collection methods, and project management protocol to action the project.

The three project outcomes will be achieved through the strong, vibrant, and longstanding partnership between Deakin University, Queensland University of Technology (QUT), and Macquarie University. In particular, team members Karantzas (Deakin University), Hazelwood (QUT), and Fitness (Macquarie University) have worked together for five years in scholarly and capacity building activities servicing the discipline of psychology in Australia. Rationale Previous ALTC reports by Cranney3 and Lipp4 highlight that psychology has been slow to apply education principles to the identification, assessment and curriculum implementation of graduate attributes in undergraduate programs. In mapping a national vision for the future of undergraduate psychology, Cranney3 canvassed stakeholders including psychology students, psychology accreditation bodies, and universities regarding the competencies and attributes of psychology graduates. Thematic analysis of stakeholder responses identified six psychology graduate attributes: 1.Knowledge and Understanding of Psychology, 2. Research Methods in Psychology, 3. Critical Thinking Skills in Psychology, 4. Values in Psychology, 5. Communication Skills, 6. Learning and the Application of Psychology. Cranney3 argued that the uptake and implementation of these psychology graduate attributes by Australian undergraduate programs required that two specific actions be undertaken. These included: (1) the development of strategies and resources to support the development and assessment of graduate attributes, and (2) the implementation of evidence-based teaching practices to yield effective ways of fostering students’ professional development and consolidation of graduate attributes. In identifying these gaps, both Cranney3 and Lipp4 argue for the importance and urgency for undergraduate psychology programs to invest in the rigorous development of curriculum design and assessment of graduate attributes to enhance student work readiness and learning outcomes. This urgency to adequately develop and assess the graduate competencies of psychology students has been significantly heightened by the recent and rapid overhaul of psychologist accreditation in Australia as part of the introduction of the Federal Government’s Health Practitioner Regulation Law (2009). As part of the reform, the newly formed Psychology Board of Australia (PBA) will administer the registration and accreditation of psychologists, ensuring that only those who are suitably qualified and display the attributes necessary to be a competent psychologist will be registered. In doing so, the PBA incorporated the recently developed psychology graduate attributes, formulated as part of previous ALTC funded projects 3,4. The Australian Psychological Accreditation Council (APAC) holds the responsibility for ensuring that psychology courses embed and assess graduate attributes as part of their curriculum. Despite the emphasis on these psychology graduate attributes by professional bodies and more recently, academic institutions, there exists no psychometrically validated measure of psychology graduate attributes. Rather, approaches to the

2 assessment of graduate attributes have involved either: (1) surface curriculum mapping approaches, “stand-alone skills assessments or GA checklists [such as whether students were exposed to or practised graduate attributes]”2; or (2) the use of electronic media such as e-Portfolios without considering how the implementation of these methods will result in the accurate assessment of attributes2. Moreover, these approaches fall short of assessing students’ development of graduate attributes over the course of their academic journey. Indeed, such assessments, irrespective of whether they are formative or summative in nature, are frequently unable to inform academics whether course or unit material enhances students’ learning outcomes and development of graduate attributes at any level. Further, there is an increasing emphasis on developing institution-wide systems for monitoring individual student learning progress and acheivement of graduate atrtibutes in Australian higher education2. However, a review of the literature reveals no valid measures of generic graduate attributes in any discipline, let alone psychology graduate attributes. Therefore, the development of a self assessment and peer assessment measure of psychology graduate attributes is the first outcome of the proposed project. While historically in higher education, external ratings of students’ performance have been viewed as being more precise than self assessment, there is in fact considerable evidence confirming the validity of self assessment as an accurate measure. Reviews of the literature have established that self appraisal is less subject to halo effects than is external ratings5. There also exists compelling evidence that self assessment demonstrates significant measurement convergence with observational measures and interview assessments5. Further, the validity of self assessment is increased when students know their assessment will be cross-checked or other informants will provide peer assessment5. Importantly, for the proposed project, the development and implementation of self and peer assessment is significant for two reasons – the role of these forms of assessment to enhance both learning and students’ capacity for self–regulation regarding graduate attributes. There is now considerable evidence that engaging students in self and peer assessment is a powerful method that enhances their learning6,7. In addition, self and peer assessment provides students with practice in monitoring and critically evaluating their own performance6. This practice fosters students’ capacity for self-regulation – the ability to assess the quality of their own work and evaluate its impact – an important skill underpinning the development of graduate attributes. Nicol8 argues that self and peer assessment of graduate attributes should be embedded within courses to ensure the development of learner self-regulation. Therefore, the inclusion of both self and peer assessments of graduate attributes in the proposed project will optimise the validity and reliability of the assessment process and enhance both the learning of students and their capacity to carefully reflect and judge their development across graduate attributes. Preliminary findings on self-and-peer assessment of graduate attributes. Work from our own team has already made significant progress into developing valid self and peer measures in assessing graduate attributes. In a preliminary study funded by a Deakin University Strategic Learning and Teaching Grant (STALG) in 2008, we developed and piloted a self-report measure of self and peer assessment of graduate attributes drawing on the descriptions of graduate attributes articulated by numerous Australian higher tertiary institutions. The measure was administered to two samples of 200 third year psychology undergraduates. Statistical analysis to determine the factor structure of the measure (i.e., the themes captured) and the reliability of the measure revealed a valid and reliable 13 item scale that represented three broad domains of graduate capabilities: (1) the use of critical thinking to synthesise material and develop solutions, and (2) teamwork and communication skills, and (3) an understanding of the issues and ethics associated with a graduate’s industry and profession. These same factors were found across both samples of students and across self-assessment and peer-assessment versions of the measure. However, given that the measure was developed just prior to the dissemination of Cranney’s3 psychology graduates attributes, our preliminary measure requires refinement and validation to ensure that the items capture all six psychology graduate attributes specified by Cranney3. These refined self and peer assessment measures can then be used by institutions Australia-wide to assess psychology students’ progress on these graduate attributes over their undergraduate course. Embedding Graduate Attributes in Psychology Curriculum through Problem Based Learning This project will not only assess students’ attainment of graduate attributes, but will develop, trial and implement inquiry-based learning activities to ensure students’ progress in developing graduate attributes. Barrie2 persuasively argues that “graduate attributes are most effectively embedded in assessment when they explicitly underpin all teaching, learning and assessment activities”9. Moreover, these authors contend that if assessment is not integrated as part of learning activities and outcomes, then neither teachers nor students see attributes as important outcomes of study. Rather, graduate attributes are perceived as outcomes that are not central to discipline content, and as less important than content-based learning outcomes. What is required is the development of curriculum and assessment procedures that move away from traditional content-transmission models to inquiry-based learning curricula, pedagogies, and assessment practices2. Similarly, we argue that curriculum design that embeds the development and assessment of graduate attributes through inquiry-based approaches (i.e., problem-based learning [PBL]) is required. We contend that PBL approaches can facilitate curriculum design such that a relationship is established between discipline content, graduate attributes and learning outcomes in explicit ways, to assist students to become aware of the connections between course content and graduate competencies. Despite the plethora of studies investigating the efficacy of PBL approaches in developing learning outcomes and competency development10 no Australian research has applied and evaluated PBL approaches to the development of

3 graduate attributes in undergraduate students of any discipline. Further still, undergraduate psychology teaching in Australia does not employ PBL-like approaches in the undergraduate education of students, focusing more on traditional content-transmission models. This is supported by Lipp4 who found that despite awareness regarding the need to develop the graduate attributes of students, the majority of psychology undergraduate courses do not provide adequate opportunities for the development and assessment of graduate attributes in the classroom. Yet, tutorials provides the ideal context in which psychology graduate attributes can be cultivated and assessed. In this environment, students have the opportunity to work for discrete periods of time in small groups to complete discipline relevant and workplace relevant activities and tasks.

Studies of small group activities (whether they are highly structured or less structured such as in PBL) note that students often “progress in parallel”11 rather than interacting with group members on tasks. Other studies have found that students divide learning activities or assessment tasks into components so that each group member works on a separate part of the activity11. Thus students often engage in, at best, co-operative rather than collaborative learning processes11. Hence, traditional small-group tutorial activities offer insignificant opportunities to develop students’ analytical, research and interpersonal skills desired in the psychology graduates of today and tomorrow. To fill this gap, the second project outcome is to develop a semi-structured, problem-based collaborative learning approach (PBL-CLA) to tutorials to foster graduate attribute development, and opportunities for formative and summative assessment of these attributes in undergraduate psychology students. Collaborative Learning Approach to Tutorials: Developing Psychology Graduate Attributes through Shared Cognition and Student Engagement While traditional tutorials can often include group work, the efficacy of group tasks is significantly enhanced if learning opportunities are designed to incorporate collaborative learning12. In a comprehensive review of the outcomes of collaborative learning approaches, Prince10 presents strong evidence for the effectiveness of collaborative learning in increasing academic achievement and skills linked to graduate attributes. Specifically, the collaborative learning approach involves the “mutual engagement of participants in a co-ordinated effort to solve the problem together”13. Moreover, collaborative learning opportunities need to be structured such that group members not only work together, but develop a shared understanding of the problem, and engage in social interaction skills to regulate each others’ understanding and cognitive change toward deeper learning14,15. Roschelle14 derived the concept of the “Joint Learning Space” in which students are asked to explicitly agree on joint goals, methods and solutions to a given problem. The result of the Joint Learning Space is to encourage students to discuss and find solutions to the problem, and to gather and analyse knowledge to establish a shared meaning of the problem at hand. Moreover, within this tutorial context, both formative and summative assessments of graduate attributes can be embedded thereby providing the opportunity for students and their peers to monitor and reflect on each other’s progress in the development of graduate attributes.

As a means of maximising student engagement in collaborative learning during class time, we propose a semi-structured PBL-CLA to tutorial work based on Roschelle’s14 concept of the Joint Learning Space and constructive alignment6. The PBL-CLA provides an innovative way of training psychology students in the necessary research, analytical and interpersonal graduate attribute skills and to improve student learning outcomes.

Preliminary PBL-CLA research. This type of approach to teaching undergraduate psychology tutorials has already been successfully trialled by our team as part of the Deakin STALG project funded in 20091. Assessment of students’ graduate attributes after undertaking fortnightly tutorial activities demonstrated that the tutorial program significantly improved students’ development on the graduate attributes of critical analysis, teamwork, and professional practice. Cumulatively, 46% of the change in student graduate attributes was due to the tutorial program. Students also reported that the semi-structured PBL-CLA enhanced their learning over the course of the tutorial program. Finally, the tutorial program improved student engagement by increasing students’ intrinsic motivation over time and enhancing their understanding of graduate attributes.

Thus, we have encouraging preliminary evidence that this method of embedding and assessing graduate attributes explicitly highlights graduate attributes as important learning outcomes of psychology. Moreover, the PBL-CLA encompasses real world activities where students engage with one another and support each others’ learning. In doing so, the PBL-CLA constructively aligns discipline content with learning outcomes and assessment activities that support the development of psychology graduate attributes. This project will build on our preliminary research in order to provide evidence-based methods for the embedding and assessment of psychology graduate attributes which can then be implemented across psychology undergraduate courses nationally. Our approach, which includes our research design and dissemination plan, describes how our project objectives will be demonstrated.

VALUE AND NEED FOR THE PROJECT The outcomes of this project are of national significance to industry bodies and universities involved in the effective training and development of psychology students and the assessment of graduate attributes. The value and need for this project is underscored by the enthusiastic endorsement of the project by major professional bodies (APS, APAC. PBA, Graduate Careers Australia), all of whom have agreed to be part of a reference group in steering this project.

Currently, there exists no reliable and validated measure to assess students’ progress across the six psychology graduate attributes identified by Cranney3. Consequently, there is an urgent need for measures assessing graduate attributes to be

4 carefully developed and validated using rigorous methods. The development of such measures needs to provide meaningful formative and summative feedback to both teachers and learners regarding students’ consolidation of graduate attributes.

The development of the GAP-Q will provide the sector with the first self and peer assessment measure of psychology graduate attributes that can be used for both the formative and summative assessment of students’ graduate competencies. This easy to use measure has the potential to be widely used by undergraduate psychology courses around Australia as a means of tracking students’ development of graduate attributes. Furthermore, the development of this measure is likely to have wider impact than solely in the discipline of psychology, providing the basis for other sectors to develop discipline specific versions to assess graduate attributes in other social science and health science domains.

The PBL-CLA tutorial program is the first in Australian undergraduate psychology to integrate the assessment of graduate attributes with curriculum development and learning outcomes. More broadly, this project is the first to integrate PBL with graduate attribute assessment. This tutorial program addresses the problems and needs outlined by previous ALTC studies that have investigated the discipline of psychology3,4. Specifically, the PBL-CLA tutorial program addresses the need for undergraduate psychology courses to provide adequate opportunities for the development and assessment of graduate attributes in the classroom. These attributes not only assist students in the workforce, but provide important foundational knowledge and skills for students’ successful navigation of postgraduate studies in psychology22. Currently, these opportunities are lacking in undergraduate psychology courses around the country. Therefore the tutorial PBL-CLA represents a highly valuable and innovative method for training psychology students in the necessary research, analytical and interpersonal graduate skills while increasing student engagement and improving student learning outcomes. In doing so, this approach provides solutions to past problems in embedding graduate attributes into psychology curricula.

Moreover, the introduction of a PBL-CLA approach to tutorials provides an innovative method for teaching in undergraduate psychology that can be used as a template for dissemination and implementation across university courses around the nation. Therefore, the use of PBL in undergraduate psychology is novel, with no Australian courses using PBL approaches in the teaching of undergraduate psychology students. Our dissemination strategy will ensure that this teaching method is made widely known and available to all undergraduate psychology programs around Australia. In particular, the national symposium and website will provide clear and tangible instructions on how to implement the PBL-CLA and embed the assessment of graduate attributes in the curriculum of psychology undergraduate units and courses.

APPROACH AND DISSEMINATION Approach The project will be conducted over two years and will involve three phases. Phase 1, conducted in year 1, will involve the refinement and piloting of the self and peer assessment versions of the Graduate Attribute Measure (termed the Graduate Attributes of Psychology Questionnaire, GAP-Q), and the refinement and piloting of materials for the CLA-PBL tutorial program at all three institutions in which the program will be evaluated. Phase 2, conducted in year 1, will comprise a large-scale psychometric validation study of both versions of the GAP-Q, with data collected across all year levels of the undergraduate psychology programs of all three participating institutions. Phase 3

Phase 1: Piloting and Refinement

, conducted in year 2, will involve the administration of the CLA-PBL tutorial program in a psychology unit at all three universities. Graduate attributes will be assessed longitudinally using the GAP-Q over the tutorial program. Our evaluation of this work is integrated within each phase of the project, as is the independent evaluation. Below we describe the project procedures and our evaluation of the project. The independent evaluation is described in the section titled “Evaluation framework.”

Piloting the GAP-Q Questions from the Graduate Attributes Questionnaire (GAQ) developed in our previous STALG project will be refined to make them relevant to psychology students. A number of new questions will be developed to capture the additional graduate attributes identified by Cranney3 and endorsed by the PBA and APAC. The revised measure (both self and peer assessment versions) will be piloted with undergraduate students undertaking psychology at all three universities. We will pilot these items to evaluate their clarity and ease of comprehension and to test for biased responding. The feedback received from the pilot participants will be used to further refine the measure before administration as part of phase 2.

Development and Piloting of the semi-structured CLA-PBL Materials The sequence of key activities for the piloting of the CLA-PBL materials are outlined in Figure 1 and described below.

Figure 1. Sequence of Activities for the piloting of GAP-Q and development of materials for phase 3

The tutorial workbook, tutor manual and adjunct materials will be developed, piloted, and evaluated within the first half of year 1. Given that all three psychology units consist of related but distinct content about relationships and communication, the workbook for each unit will need to be designed so that it aligns with the learning materials and outcomes of the unit. The

Activity 7 Activity 6 Activity 5 Activity 4 Activity 3 Activity 2 Activity 1 Development of CLA-PBL materials for all three units

Piloting of materials with tutors at each

institution

Refinement of materials

Piloting of materials with a small group of students at each

institution

Final refinement CLA-PBL

materials for all three units

Tutor feedback session

Student feedback session

5 tutorial workbook and teaching materials developed for the unit HPS304 The Social Psychology of Relationships at Deakin University will be adapted in developing the tutorial workbooks for QUT and Macquarie University. Once developed, the tutorial workbook and adjunct materials will be trialled with the tutors participating in the tutorial in the role of the student, with a member from the research team acting as the facilitator. The facilitator and tutors will then have a de-briefing session about the content of the workbook, the timing of the activities and the extent to which the workbook assists students in developing the graduate attributes. Once this is done, a small group of students will also be invited to trial one tutorial in which tutors will alternate in facilitating the tutorial. Again, a de-briefing session will follow in which the same issues will be discussed, but in this iteration the feedback will be from the students’ perspective. This information will be collated and the project team will review the pilot feedback and determine any required amendments to the materials before rolling out the tutorial program in year 2. Phase 2: Psychometric Assessment of the Psychology Graduate Attribute Measure The GAP-Q developed in the piloting phase of the project will be administered online to psychology undergraduate students and then statistically evaluated using factor analysis techniques to identify the key themes that represent psychology graduate attributes. The sequence of key activities for this phase of the project are outlined in Figure 2 and described below.

Figure 2. Sequence of activities for phase 2 – GAP-Q development

The GAP-Q developed in the piloting phase of the project will be administered online to approximately 450 psychology undergraduate students (150 from each institution). The sample size was determined through adhering to guidelines required in conducting the statistical analyses described in the data analysis section below. Once university ethics approval is obtained, students will be recruited from semester units by placing an invitation to participate in the study on the announcement boards of unit websites. Permission will be requested to place the invitation online by contacting all unit chairs of the three universities undergraduate psychology programs in semester 2, 2011. The website will include a web link to the questionnaire, where students will be able to access the anonymous online questionnaire. The online questionnaire will remain live for 2 months to ensure that the desired sample size is reached.

Phase 2: Data Analysis Upon conclusion of the data collection period, the data will be subjected to different statistical analyses – each with a specific purpose. To assess how well each item in the self and peer assessment versions of the GAP-Q discriminates between students’ competence on the various graduate attributes, Item Response Analysis (IRA) will be used. IRA is a technique that examines the consistency of scores for each respondent on items in a factor. For example, if an individual scores high on all items but one measuring critical analysis, then IRA will detect this item as an item with poor discrimination. To explore the factor structure of both versions of the GAP-Q we will use Confirmatory Factor Analysis (CFA). CFA is a method used to assess if a measure captures the themes or constructs (termed factors) hypothesised to exist as part of a self-report instrument. Therefore, CFA will be used to determine whether the GAP-Q is comprised of six factors representing the six psychology graduate attributes. Our sample size meets this criterion enabling us to conduct CFA and produce non-biased results16. To ensure that each graduate attribute factor is reliable we will conduct Reliability Analysis (RA). RA is used to determine if all items constituting a factor consistently measure the graduate attribute of interest (i.e., all items comprising the factor of critical analysis measure critical analysis and share relationships with all other items belonging to the factor). These methods will allow use to enhance the reliability and validity of the measure enabling us to accurately assess the psychology graduate attributes. Phase 3: Curriculum Embedding and Assessment of Graduate Attributes Phase 3 will take the form of a longitudinal study across three year levels. As part of this study design, undergraduate students enrolled in first, second and third year psychology studies will undertake the tutorial program in year 2 of the project. Each participating institution will deliver the tutorial program at one year level. Specifically: • First year psychology students enrolled in the foundational unit PYB007 Interpersonal Processes and Skills at QUT will

undertake the tutorial program during semester 1. • Second year psychology students enrolled in PSY250 The Psychology of Human Relationships at Macquarie University

will undertake the tutorial program in semester 2. • Third year students enrolled in HPS304 The Social Psychology of Relationships at Deakin University, will undertake the

tutorial program in semester 2. This implementation plan will achieve the following important outcomes. Firstly, the mix of institutions captures the

diversity of Australian higher education. Deakin University, a multi-campus institution, is one of Australia's most progressive universities and a leader in distance education, providing a mixture of on campus and off campus learning across most of its university courses. QUT is part of the Australian Technology Network (ATN), a coalition of five Australian universities that share a common focus on the practical application of tertiary studies and research and aim to produce graduates who are closely aligned to the needs of industry and the wider society. Like Deakin, QUT is a multi-campus institution; however,

Activity 5 Activity 4 Activity 3 Activity 2 Activity 1 Recruitment of students

across institutions Students anonymously fill in GAP-Q online

Analysis of the factors underlying the measure

Analysis of the reliability of each factor

Analysis of each item

6 studies in psychology are undertaken on the Kelvin Grove campus whereas studies in psychology at Deakin are delivered across campuses. Macquarie University

Secondly, implementing the method in three different institutions allows us to include students from different demographic and geographic locations around Australia. Thirdly, we are able to test the effectiveness of the method by varying tutors, course material and departmental course structure which have been found in past education research to influence teaching interventions17. Finally, it affords us the opportunity to implement the program and collect data from all three undergraduate year levels in a single year.

is a single campus institution that provides traditional face-to-face teaching and is placed in the top 10 tertiary institutions in Australia by the Times Higher Education World University Rankings.

An outline of the sequence of the key activities of phase 3 conducted at each institution are presented in Figure 3 and described below the figure.

Figure 3. Sequence of activities for phase 3 – PBL-CLA tutorial program fostering graduate attributes

The implementation of the tutorial program at each institution will take the form of six, two-hour fortnightly tutorials that will be conducted on the even weeks of the targeted teaching period. At the end of each fortnightly tutorial, students will be administered the GAP-Q alongside other control variables outlined below to determine how the fortnightly tutorial activities impact on students’ learning and development of graduate attributes, and to track changes in students’ self assessment of graduate attributes. Thus, a total of six assessments of graduate attributes will be collected from each year level, providing three year levels of longitudinal data. The total cohort numbers are estimated to be 1,100 students, with approximately 600 first year undergraduate students at QUT, 200 second year undergraduate students at Macquarie, and 300 third year undergraduate students at Deakin University. Based on the response rate obtained during the trial of the tutorial program as part of the STALG project at Deakin, we anticipate approximately 50% of students at each institution to participate in the evaluation. Based on our own experience in conducting longitudinal studies and past evidence on attrition rates, we expect approximately 30-40%18 of participants to drop out of the evaluation over the tutorial program. Therefore, we expect to recruit and retain a total of approximately 420 students as part of phase 3.

On campus administration of CLA Tutorial 1

In the first tutorial (T1), students will be informed about the ALTC project and the tutorial program. During this tutorial, students will be provided with a copy of the Plain Language Statement and invited to participate in the study. Irrespective of their participation in the study, all students will undertake the tutorial program. Interested participants will complete a set of questionnaires which include the collection of background information about the participant, and data regarding a series of baseline and control measures. Participants will be administered measures of: learning styles (Revised Study Process Questionnaire [R-SPQ-2F19]), academic motivation (Academic Motivation Scale [AMS20]), and the GAP-Q self assessment validated from phase 2.

Tutorials 2 to 6 At the beginning of tutorials 2 to 6, students will be randomly divided into five small groups each consisting of five students who will sit facing each other. Students will remain in their randomly nominated group throughout the tutorial program (tutorial 2 to tutorial 6). The tutor will then ask each group to select a group leader and to open their workbooks and commence reading the background information for the given case study for the week. Students will then work through the interactive activities outlined in the workbook (i.e., read case notes of a couple or family in crisis, score psychological tests, read transcripts) and there will be a time limit in which to develop a course of action to assist the couple/family in question, and provide justification for this course of action. The workbook will drive students’ movement through the development of their course of action and justification. The workbook will consist of case notes, tasks involving reflective questions regarding the problem, activities around the development of concept maps to synthesise ideas, and case formulations to assist students’ decision-making processes, critical analysis and capacities to work effectively and collaboratively as a group. The workbook tasks will also guide students in referring to their lecture notes, learning modules and reading materials that accompany the units as a means of aligning the practicum with the content of each unit. Most importantly, in fostering collaborative learning, the workbook will comprise a series of questions that guide the groups in developing shared meanings, goals and strategies in working through the problem-based task.

The focus on developing a workbook that fosters collaborative learning is designed to improve students’ critical analysis and problem-solving skills and foster a deeper understanding of the lecture content and reading material. At the end of the tutorial, each group of students will be required make a five minute presentation to the rest of the class regarding their case conceptualisation and course of action. The presentation requires groups to provide a justification for their solution and

Activity 4 Activity 3 Activity 2 Activity 1 • Assessment of graduate attributes

using GAP-Q, demographics and control variables in the first tutorial

• Tutorial program to students outlined in first tutorial

• Administer the PBL-CLA approach in all subsequent tutorials. • Measure how each tutorial fosters

graduate attributes, motivation, PBL self assessment, teamwork

•Analyse quantitative data •Analyse qualitative data (self-report & focus groups)

•Conduct student focus groups

•Conduct tutor focus groups

7 audience members (students and the tutor) are given the opportunity to ask questions of the group. Thus, students are exposed to the process of peer review. The entire process from commencing the tutorial to completing the group presentations will take 1.5 hours. Our previous trials indicate that this provides students with sufficient time to complete their responses to the brief set of quantitative and qualitative questionnaires administered at the end of the tutorial.

Data regarding the extent to which the tutorial program enhances students’ learning motivation and learning experiences regarding graduate attributes will be collected at the end of each tutorial (tutorial 2 to tutorial 6). Participating students will spend 15 minutes completing two self-report measures – a tutorial-based version of the GAP-Q (self assessment) in which students are asked the extent to which tutorial activities fostered the development of psychology graduate attributes, and a validated measure of intrinsic motivation (Intrinsic Motivation Inventory [IMI21]). Students will also answer two open-ended questions regarding: (1) whether the tutorial facilitated the students’ learning of discipline content, and (2) how activities helped them in developing each of the six psychology graduate attributes.

Furthermore, in tutorials 3 and 6, group members will complete the peer assessment version of the GAP-Q. Students in each group will be randomly assigned to a group member on whom they will provide the peer assessment of graduate attributes. Upon completing the GAP-Q (peer assessment) students will pair up to discuss how their self assessment compares to their fellow peer’s assessment of their graduate attributes. This process of peer review will further help students in making judgments regarding their competencies6 which can improve their capacity to self-regulate in developing these attributes. The administration of the peer assessment of graduate attributes at the end of tutorial 3 and 6 is a strategic decision. Peer assessment at the end of tutorial 3 is timed to ensure that peers have sufficient knowledge of each other to provide an accurate assessment and feedback to fellow students. Furthermore, the discussion between pairs of self and peer raters in tutorial 3 will provide formative feedback midway through the tutorial program to complement student’s self assessment on the graduate attributes. The peer assessment in the final tutorial (tutorial 6) not only provides students with summative assessment from peers, but also feedback regarding how the peer has perceived change in the student’s demonstration of graduate attributes over time.

Focus Groups Post Tutorial Program At the conclusion of the tutorial program in each unit, tutors and students will be invited to participate in two small focus groups at each institution. The purpose of the focus groups will be to obtain tutor and student accounts regarding students’ engagement in deep learning, changes in deep learning, and development of graduate attributes. The focus groups will run for 1 hour each.

Phase 3: Data Analysis To assess changes in students’ progress towards developing their graduate attributes across the six tutorials (tutorial 1 to tutorial 6) we will employ Latent Growth Curve Modelling (LGCM). LGCM is a statistical method that calculates linear and non-linear rates of change over a period of time. As a result, LGCM will allow us to assess how students’ competencies in terms of their graduate attributes change over the course of the tutorial program. To control for contextual variables that may influence findings such as: tutor, campus, or institution, we will use Multi-level modelling. Multi-level modelling is a statistical technique that accounts for the effects of contextual variables and partials out these contextual effects in the analysis of the variables of interest. This allows us to accurately calculate the how much of the change in students development of graduate attributes is due to the PBL-CLA tutorial program and how much is due to other factors in tutorial delivery (e.g., tutor, campus, institution).

The qualitative data will be subjected to thematic discourse analysis in order to uncover common themes in students’ engagement with, and evaluation of, the tutorial program. The coding of themes will be undertaken by members of the research team during which time category themes and coding instructions will be developed. This coding scheme, along with the transcripts, will then be given to two independent raters to re-code the qualitative data. The inter-rater reliability of coders will then be calculated (Cohen’s Kappa). This method of coding and analysis will be conducted to ensure the accuracy and robustness of the derived themes from the transcripts. These analyses will also be subjected to further post-hoc analyses by institution, campus, and tutor. Dissemination Dissemination will be woven throughout the entire project facilitated through meetings with a project reference group who themselves are connected to the partner tertiary institutions, industry and professional accreditation bodies. In addition to this, our dissemination strategy consists of four specific components. These dissemination plans include: (1) The development of a workshop regarding psychology graduate attribute assessment to be run at the Australian Psychological Society national conference; (2) The presentation of findings at national and international conferences and in national and international peer reviewed journal publications; (3) The development of a website that provides resources to staff regarding curriculum design and assessment of psychology graduate attributes. This site will also provide psychology students with resources pertaining to graduate attributes; (4) A national symposium on psychology graduate attributes.

Therefore, these strategies ensure that dissemination will involve: teaching and learning practitioners, course leaders/co-ordinators, psychology Heads of Department, national accreditation bodies, and psychology students. This will ensure that all key stakeholders will be engaged in the dissemination process. Each of the four dissemination strategies is outlined below.

8 1. Psychology graduate attribute assessment workshop at the Australian Psychological Society (APS) conference The project team will submit a proposal to conduct a workshop on the GAP-Q, its application and scoring methods at the APS national conference. The purpose of this workshop will be to disseminate the GAP-Q and best practice around the assessment of psychology graduate attributes to academic staff from psychology departments around Australia. This will be an outcome of year 1 of the project. 2. The presentation of findings at national and international conferences and in national and international peer

reviewed journal publications The project team will disseminate the study findings to the broader academic community in the form of national and international conference presentations in the areas of psychology and education (e.g., APS National Conference, Higher Education Research and Development Society of Australasia Conference, the International Conference on Psychology Education) and submitted for publication in a peer-reviewed journal (e.g., Journal of Educational Psychology, International Journal of Educational Research or journal of similar impact or esteem). These forms of dissemination will be outcomes of year 1 and 2 of the project. 3. Staff and student website on curriculum design and assessment of graduate attributes The website will consist of a staff and student section. The staff section will include downloadable guides that will contain advice and strategies pertaining to best practice regarding the embedding of graduate attributes in curriculum design and assessment of psychology graduate attributes. This site will feature examples of learning materials from the three institutions’ CLA workbooks. The site will also include a research repository clearinghouse. The clearinghouse will include a copy of the ALTC reports for years 1 and 2 of the proposed study, previous ALTC reports pertaining to curriculum development and assessment of psychology graduate attributes, and ALTC project reports on graduate attributes in general. This section of the website will also include a description of the GAP-Q, a copy of the self and peer assessment measure and guidelines on how to administer and score the measure. Advice on how to implement the measure in both formative and summative assessment will also be provided.

The student section of the website will include information about the psychology graduate attributes and the relevance of graduate attributes to the competency standards of practicing psychologists. The site will give students the opportunity to complete the GAP-Q online and will receive immediate feedback on their self assessment on the psychology graduate attributes. This feedback will provide the student with their scores on the various graduate attributes and commentary on their strengths and areas for improvement. The site will be promoted through online advertisements placed on the APS website, various national psychology teaching websites, the Heads of Department and Schools of Psychology Association (HODSPA) network, and through the psychology student societies associated with each tertiary institution around Australia. The site will be designed to collect data regarding site usability and accessibility such as number of hits on various web-pages and the number of times a given resource was downloaded. The staff and student website outcomes for year 1 will be limited to information regarding the GAP-Q. The inclusion of content regarding strategies of best practice in embedding graduate attributes in curriculum design and assessment of psychology graduate attributes will be an outcome of year 2. 4. National symposium on psychology graduate attributes. The project team will organise a Heads of Department and Schools of Psychology Association (HODSPA) symposium on psychology graduate attributes. Heads of department, associate heads of department, course co-ordinators and unit chairs/co-ordinators from all psychology departments from around the country, members of the Australian Psychological Society, the Psychology Board of Australia and the Australian Psychological Accreditation Council will be invited to attend the symposium. The symposium will coincide with a major national psychology conference such as the HODSPA annual meeting thereby ensuring that the symposium will be well attended by key stakeholders. The symposium will be organised as a pre-meeting activity. The theme of the symposium will be on graduate attribute curriculum design and assessment. During this symposium the project team will present the findings of the ALTC project and provide the opportunity for staff from psychology courses around the country to attend and share practice experiences regarding the curriculum design and assessment of graduate attributes. The symposium will also provide the opportunity for the project team to receive feedback about the GAP-Q and the PBL-CLA from stakeholders from around the country. The symposium will culminate in the development of a set of recommended national practice and assessment initiatives of psychology graduate attributes which will be disseminated to all psychology departments via the HODSPA network and uploaded on the project website. The national symposium will be an outcome of year 2 of the project.

PROJECT MANAGEMENT The project management will highly structured to ensure effective staff management, financial control, and project evaluation in executing the project vision and goals. The project management will involve five groups of personnel – the project team, a project manager, a financial controller, a reference group, and an evaluation consultant. Project vision and goals: Clear project vision and goals have already been established as outlined in the project outcomes and plans for dissemination. However, execution of this vision and goals will be further refined at the beginning of the project through a meeting with the project team, project manager, reference group and evaluation consultant. The project team, manager and reference group

9 will work with the consultant to establish the scope of the evaluation, how the stakeholders will be consulted, and how the evaluation findings will be integrated into the project at each phase to address the goals and vision of the project. Staff management: The project team have each committed to distinct roles for the project. In the first project meeting, all project personnel will provide a role description outlining how they will action their role and provide timelines regarding their execution of key tasks. The project team and project manager will conduct phone conferences bimonthly to ensure the project is being actioned in a timely manner. The project team, project manager and reference group will meet every six months (4 face-to-face meetings over the project) to ensure that industry stakeholders have input in the vision, reporting and project outcomes. The specific contributions of each personnel group are outlined below under “Project Team and Anticipated Contributions”. Financial control: Over the entire period of the project, Deakin University has committed the School of Psychology financial controller to oversee the weekly financial management of the project. Independent evaluation:

Project Team and Anticipated Contributions

An independent consultant will be employed to carry out an evaluation of the project. The role and tasks of the consultant are outline in the “Evaluation Framework” section.

Dr Gery Karantzas (Deakin University) will lead the project team and will be responsible for the development and piloting of the psychology graduate attributes questionnaire (GAP-Q). He will also be responsible for the piloting and development of the PBL-CLA materials for HPS304 The Social Psychology of Relationships at Deakin University. This will include the development of the content of the cases to be used in the tutorials, the content of the workbooks and tutor manuals and meeting with the project team and reference groups regarding refinement of materials. He will also provide advice and support materials to Dr Hazelwood and Professor Fitness in developing the PBL-CLA tutorial materials for the units at QUT and Macquarie University. Dr Karantzas will lead the quantitative data analysis and report writing of the project. He will also organise and convene the national symposium on psychology graduate attributes and co-ordinate the project team in the design and development of the website.

Ms Susie Macfarlane (Deakin University) will assist in the development of project materials and will be responsible for the training of tutors in conducting the PBL-CLA for on campus tutorials at all three institutions. She will also conduct the tutor and student focus groups and be responsible for leading the qualitative analysis of focus groups. Ms Macfarlane will also assist with the project write-up and have input in the development of the website.

Associate Professor Greg Tooley (Deakin University) will provide advice regarding the development and piloting of the GAP-Q and the materials for the PBL-CLA. He will co-convene the national symposium on psychology graduate attributes with Dr Karantzas and assist in formulating the program and inviting relevant parties to attend the symposium. He will also assist in the project write-up and lead the development of content for the staff section of the website.

Associate Professor Alex Mussap (Deakin University) will provide advice regarding the development and piloting of the GAP-Q and the materials for the PBL-CLA. He will assist with the project write up and take a leadership role in the content development for the website and be responsible for the development of student section of the website.

Dr Zoe Hazelwood (Queensland University of Technology) will be responsible for the piloting and development of PBL-CLA materials for PYB007 Interpersonal Processes and Skills at QUT. This will include the development of the content of the cases to be used in the tutorials, the content of the workbooks and tutor manuals. She will also co-ordinate the tutors and conduct regular meetings to receive feedback from the tutors while also providing support to tutors in the delivery of the tutorial program. Dr Hazelwood will assist with the write up of the project.

Professor Julie Fitness (Macquarie University) will be responsible for the piloting and development of PBL-CLA materials for PSY250 The Psychology of Human Relationships at Macquarie University. This will include the development of the content of the cases to be used in the tutorials, the content of the workbooks and tutor manuals. She will also co-ordinate the tutors and conduct regular meetings to receive feedback from the tutors while also providing support to tutors in the delivery of the tutorial program. Professor Fitness will also assist with the write up. Project Manager The day-to-day management of the project will be the responsibility of the project manager. The project manager will be supervised by Dr Karantzas. The project manager will organise project meetings, including project team and reference group meetings and ensure ongoing communication between team members and the reference group. They will also be responsible for organising the data collection through all phases of the project (1 to 3), the cleaning and screening of data and conducting preliminary descriptive analysis for subsequent use by the project team. The project manager will also be involved in the organisation of all aspects of the national symposium on psychology graduate attributes and will liaise with web developers regarding the psychology graduate attributes website. They will also be involved in the formatting of materials for the PBL-CLA tutorial programs for the three universities. Financial Controller The School of Psychology at Deakin University will provide up to half a day a week

Reference Group

of time release for the School’s financial controller to financially manage the project. The financial controller will provide monthly reports to the project team regarding project expenditure and manage the payment of the project manager, evaluation consultant and all project expenses.

10 The commencement of the project will see the appointment of a Reference Group that will provide expert advice at the commencement of the project and feedback at the commencement and conclusion of each of the three phases of the project and on the resources developed during the piloting phase. The reference group will comprise of Dr Dale Holt, Dr Nicholas Vidouris, Professor Brin Greyner, and Mr Bruce Guthrie. Dr Dale Holt is the deputy director of the Institute of Teaching and Learning at Deakin University and has successfully conducted numerous ALTC projects and acted as a reference group member on many other teaching projects – both national and international. Dr Vidouris is the senior manager of science and education of the Australian Psychological Association (APS), and member of both the National Psychology Education and Training (NPET) Reference group of the APS and the Australian Psychology Accreditation Council (APAC). Professor Greyner is the Chair of the Psychology Board of Australia. Mr Bruce Guthrie is the acting executive director and research manager at Graduate Careers Australia (GCA). Evaluation Framework An independent consultant will be employed to evaluate the project during each of the three stages, thereby providing feedback to the project team throughout the project. The project team will work with the consultant to establish the scope of the evaluation, how the stakeholders will be consulted, criteria for assessment, and how the evaluation findings will be integrated into the project at each phase. Specifically, the evaluation will: (1) survey the uptake of the GAP-Q by psychology departments and schools around Australia; (2) evaluate the effectiveness of the implementation of the tutorial program in developing graduate attributes at the three institutions; (3) the management of input between the project team and the project team and the reference group; (4) appraise the dissemination plans associated with the project, and (5) assess the quality of the project’s contribution to the evidence base regarding the embedding and assessment of graduate attributes in psychology courses of Australian universities. The evaluation consultant will provide formative feedback through a series of brief reports at each phase of the activity (including attendance at the national symposium) and a summative evaluation report at conclusion of project.

REFERENCES 1. Karantzas, et al. (2010). Developing face-to-face and online tutorial-based collaborative learning: Evaluating an approach

to enhance student engagement and the development of graduate attributes. STALG report. Deakin University. 2. Barrie, et al. (2009). Key issues to consider in the renewal of learning and teaching experiences to foster graduate

attributes: The National Graduate Attributes Project. Syd: ALTC. 3. Cranney, et al. (2008). Designing a diverse, future-oriented vision for undergraduate psychology in Australia. Syd: ALTC. 4. Lipp, et al. (2006). Learning outcomes and curriculum development in Psychology. Syd: ALTC. 5. Robins, et al. (2007). Handbook of research methods in personality psychology. NY: Guilford. 6. Biggs. (2003). Teaching for quality learning at university (2nd ed). London: Open University Press. 7. Nicol, Macfarlane-Dick. (2006) Formative assessment and self-regulated learning: a model and seven principles of good

feedback practice, Studies in H. Ed, 31, 199-218. 8. Nicol. (2010). The foundation for graduate attributes: developing self-regulation through self and peer assessment,

Quality Assurance Agency for Higher Education: Graduates for the 21st Century: Integrating the Enhancement themes. 9. Loacker. (2000). Self Assessment at Alverno College. Wisconsin: Alverno College Institute Productions. 10. Prince. (2004). Does Active Learning Work? A Review of the Research. Jrnal of Engine. Ed., 93, 223-231. 11. DIllenbourg, et al. (1996). The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds.), Learning

in Humans and Machine: Towards an interdisciplinary learning science. (Pp. 189-211). Oxford: Elsevier. 12. Blundel. (2001) Collaborative learning through the art of writing. London: Routledge. 13. Roschelle, Teasley. (1995). The construction of shared knowledge in collaborative problem solving. In O'Malley, C. E.,

(Ed.), Computer Supported Collaborative Learning (pp. 69-97). Heidelberg: Springer-Verlag. 14. Goodnow et al. (2002). Extending studies of collaborative cognition by way of care giving situations. IJBD, 26, 16-25. 15. Resnick, et al. (Eds.) (1991). Perspectives on socially shared cognition. DC: APA. 16. Finch, et al. (1997). Sample size & nonnormality on the estimation of mediated effects in latent models. SEM, 4, 87-107. 17. Neuhauser. (2002). Learning style & effectiveness of online and face-to-face instruction. Am J Dist Ed, 16, 99-113. 18. McKnight, et al. (2007). Missing Data: A Gentle Introduction. NY: Guilford. 19. Biggs, et al. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British J of Ed Psych, 71, 133–149. 20. Vallerand, et al. (1993). On the assessment of intrinsic, extrinsic, & amotivation in education: Evidence on the concurrent

and construct validity of the Academic Motivation Scale. Ed & Psych Meas, 53, 159-172. 21. Deci, et al. (1994). Facilitating internalization: The self-determination theory perspective. Jn of Personality, 62, 119-142. 22. Kiernan, et al. (2010). Professional education of psychologists using online problem-based learning methods: Experience at Charles Sturt University. Australian Psychologist, 43, 286-292.

1

BUDGET Budget Stage 1/Year 1 Budget Stage 2/Year 2

ALTC $

Other $

Total $

ALTC $

Other $

Total $

PERSONNEL Research Fellow/Project Manager (.8 EFT) 53,495 53,495 Karantzas (Project leader .2 EFT) 16,767 16,767 Macfarlane (Team member .2 EFT) 15,178 15,178 Tooley(Team member .1 EFT) 11,119 11,119 Mussap (Team member .1 EFT) 11,119 11,119 Hazelwood (Team member .2 EFT) 16,703 16,703 Fitness (Team member .2 EFT) 28,247 28,247 Financial controller (.1 ET) 7,278 7,278 Tutor payments (Basic Tutorial) 21,945 Tutor payments (Repeat Tutorial) 14,498 Teaching relief 54 @ 98.85 5,338 5,338 Sub total 58,833 106,411 165,244 95,276 106,411 201,687 PROJECT SUPPORT Tutorial workbooks 3,927 Tutor manuals 374 Questionnaire booklets 4,590 Tutorial adjunct materials 871 Project team conference calls 605 605 Reference group meeting (Melb) - catering 316 316 Reference group meeting (Melb) - travel & accomm

2,070 2,070

Focus group transcription 1,065 Sub total 2,991 2,991 13,818 13,818 PROJECT ACTIVITIES Consultant - project team interviews 1,000 1,000 Consultant - dissemination evaluation & report 2,000 4,000 Website development & maintenance 4,240 2,120 National symposium - catering 62 x 82.52 5,116 National symposium - travel 5,000 National symposium - accomm 1,500 National symposium - program printing 106 Sub total 7,240 7,240 18,842 18,842 ATTENDANCE AT ALTC EVENTS 3,000 Sub total 3,000 3,000 INSTITUTIONAL OVERHEAD LEVY 7,206 12,794 Sub Total 7,206 12,794 20,000 Total per Stage/Year 79,270 106,411 185,681 140,730 106,411 247,141 ALTC Other Total TOTAL PROJECT BUDGET 220,000 212,822 432,822

2 Budget Justification (Please note: Some costs are estimated based on average anticipated costs)

Budget Stage 1/Year 1 Budget Stage 2/Year 2 Item ALTC

$ Other $

Total $

ALTC $

Other $

Total $

PERSONNEL Research Fellow/ Project Manager

RF (Grade 1 Step 2) $52,240 @ .8 + 28% oncosts per wk x 2 years 53,495 53,495

Karantzas Project leader

Karantzas (Level C Step 1) $83,833 @ .2 EFT per wk x 2 years 16,767 16,767

Macfarlane Team member

Macfarlane (Level B Step 3) $75,887 @ .2 EFT per wk x 2 years 15,178 15,178

Tooley Team member

Tooley (Level D Step 3) $111,190 @ .1 EFT per wk x 2 years 11,119 11,119

Mussap Team member

Mussap (Level D Step 3) $111,190 @ .1 EFT per wk x 2 years 11,119 11,119

Hazelwood Team member

Hazelwood (Level B Step 5) $83514 @ .2 EFT per wk x 2 years 16,703 16,703

Fitness Team member

Fitness (Level E Step 2) $141,240 @ .2 EFT per wk x 2 years 28,248 28,248

Financial controller HEW 7 (Class 7, Step 6) $72,784 @ .1 EFT per wk x 2 years 7,278 7,278

Tutor payments (Basic Tutorial)

Tutor payments - 22 tutorials (12 QUT, 6 Deakin, 4 Macquarie) @ $98.85 x 2 hours each x 5 weeks

21,945

Tutor payments (Repeat Tutorial)

Tutor payments - 22 tutorials (12 QUT, 6 Deakin, 4 Macquarie) @ 65.90 x 2 hours each x 5 weeks

14,498

Teaching relief 54 hours @ $98.85 (Basic Tutorial) 5,338 5,338 Sub total 58,833 106,411 165,244 95,276 106,411 201,687 PROJECT SUPPORT

Tutorial workbooks 1,100 tutorial workbooks @ $0.17 x 21 pages double-sided 3,927 Tutor manuals 22 Tutor manuals & materials $0.17 x 100 pages double-sided 374 Questionnaire booklets

450 Questionnaire booklets @ $0.17 x 12 pages double-sided x 5 tutorial assessments

4,590

Tutorial materials Tutorial adjunct PBL material 1,100 students / groups of 5 = 220 copies @ $.09 x 44 pages single-sided

871

Team meetings Team conference call meetings - 6 bimonthly meetings @ $0.84 per minute x 120 mins x 2 years

605 605

Reference group meetings

Reference group meetings catering - 10 @ $15.80 per person x 4 meetings

316

316

Reference group travel

Reference group meeting travel & accommodation for interstate members Hazelwood ($400 flight + $150 accomm), Fitness ($335 flight + $150 accomm) x 2 meetings x 2 years

2,070 2,070

Transcription Student and tutor focus group transcription - 6 focus groups @ 60 mins each @ $2.958 per minute

1,065

Sub total 2,991 2,991 13,818 13,818

3

Budget Stage 1/Year 1 Budget Stage 2/Year 2

Item ALTC $

Other $

Total $

ALTC $

Other $

Total $

PROJECT ACTIVITIES

Evaluation Consultant

Interviews with team members regarding project team processes & practice $1000 per day x 2 (1 day in year 1 & year 2)

1,000 1,000

Attendance & evaluation of symposium $1000 (year 2), evaluation of website over 2 years $2000, final evaluation report 2,000 (year 2)

2,000 4,000

Website development & maintenance

Web programmer (HEW 4, Step 1) @ $212.00 per day (incl. on costs) x 20 days (20 days year 1, 10 days year 2)

4,240 2,120

National graduate attributes symposium

Catering - 62 delegates (6 research team, 4 reference group, 28 Heads of Department/School, and 28 Associate Heads Teaching & learning @ $82.52 day delegate package

5,116

Domestic flights @ $500 x 10 personnel (6 project team & 4 reference group) *flight costs factor in the event that symposium is held in conjunction with an interstate national psychology conference.

5,000

Accommodation @ $150 per night x 10 personnel (6 project team & 4 reference group) *accommodation costs factor in the event that symposium is held in conjunction with an interstate national psychology conference.

1,500

Printing symposium program for 62 delegates @ $0.17 x 10 pages double- sided

106

Sub total 7,240 7,240 18,842 18,842 ATTENDANCE AT ALTC EVENTS

Sub total 3,000

3,000 3,000

INSTITUTIONAL OVERHEAD LEVY

Sub Total

7,206 12,794

Total per Stage/Year 7,206 12,794 20,000 79,270 106,411 185,681 140,730 106,411 247,141 TOTAL PROJECT BUDGET ALTC Other Total 220,000 212,822 432,822

4 Budget Justification (continued)

A large proportion of the budget pertains to personnel costs, with the employment of a project manager/research fellow, tutors, and teaching relief. These are essential costs for this time intensive project. The project manager will co-ordinate the day-to-day running of the project, will assist with the convening of meetings with team members and the reference group, and will be central to the achieving many milestones of the project. Given that the PBL-CLA is administered during the tutorial programs at each institution, the employment of tutors is essential to ensure that the approach is implemented in year two. The development of the various materials for the PBL-CLA requires that Karantzas, Hazelwood, and Fitness, receive teaching relief to devote time to the construction of these materials which include the workbook, adjunct decision-making materials for students, and the tutor manuals. These are time intensive activities, thus teaching relief is another central personnel cost. All project support items are central funds for the project’s success. The need to develop and disseminate workbooks, adjunct decision-making materials to students and evaluate student progress through the learning activities requires the printing of numerous documents multiple times over throughout the course of the project. The team meeting and reference group costs are necessary to ensure the smooth and timely running of the project. Given that Hazelwood and Fitness are interstate team members, travel costs have been factored into the project to attend meetings. The central role of these team members makes it important that they have face-to-face meetings with team members and the reference group. The project activity costs are essential costs and have significant implications regarding the project dissemination plans and evaluation of the project. Much of these costs relate to the national graduate attributes symposium – a dissemination activity of national significance that brings together the project team and reference group will Heads of Departments and teaching leaders in psychology Australia-wide. The project evaluation costs span years 1 and 2. This is because the evaluation process will provide both ongoing evaluation (which can feed into the project team’s management of the project) and a summative evaluation that will provide data regarding the efficacy of the entire project.

Timeline Project Timeframe Activities & Outcomes Nov. 2011 – Dec. 2011 • Ethics application submission and approval Jan. 2012 – Mar. 2012 • Refinement of GAP-Q and development of PBL-CLA tutorial materials April 2012 – May 2012 • Piloting, review of feedback and refinement of GAP-Q and PBL-CLA tutorial materials June 2012 – Aug. 2012 • GAP-Q data collection across all three institutions, printing of PBL-CLA materials (“Choose

your own adventure workbook”, tutorial manual, adjunct tutorial materials) • GAP-Q data analysis and report write up • Staff and student web-site design development and construction

Sept. 2012 – Oct. 2012 • GAP-Q workshop APS national conference • Conference presentation (e.g., APS conference, National Teaching Leaders Forum) • Upload of GAP-Q information of staff and student web-site, internal web-site testing and

compliance Nov. 2012 • GAP-Q information published on staff and student web-site

• Web-site presence disseminated to all Australian universities offering undergraduate psychology courses via HODSPA network email list and APS email list.

Dec. 2012 • Writing up of GAP-Q journal paper Jan. 2013 – Feb. 2013 • Training of tutors in PBL-CLA tutorial approach at QUT Mar. 2013 – June 2013 • Implementation of PBL-CLA tutorial approach to undergraduate students at QUT

• Administration of evaluation questionnaires after each tutorial • Conduct tutor and student focus groups and transcription • QUT Data entry and analysis • Training of tutors in PBL-CLA tutorial approach at Deakin & Macquarie Universities

July 2013 – Oct. 2013 • Implementation of PBL-CLA tutorial approach to undergraduate students at Deakin University and Macquarie University

• Administration of evaluation questionnaires after each tutorial • Conduct tutor and student focus groups and transcription • Deakin and Macquarie University data entry and analysis • Conference presentation (e.g., HERDSA national conference, International Conference on

Psychology Education) Nov. 2013 • Upload of PBL-CLA information on staff web-site (staff web-site developed and tested in year 1)

• National Graduate Attribute Symposium • Writing up of PBL-CLA journal paper

Qualifications and Experience of the Project Team

This project brings together a strong and vibrant team with the necessary skills and knowledge in teaching and research to achieve the proposed outcomes. Cumulatively, the team has been awarded numerous teaching awards including ALTC Citations and conducted innovative teaching and research in student learning outcomes. The team consists of a blend of early career and experienced academics holding various positions of leadership in teaching and learning. The team has a longstanding history. Over the last two year, the project leader Dr Karantzas has worked with colleagues Macfarlane, Tooley, and Mussap on innovative tutorial approaches to teaching undergraduate students. Karantzas, Hazelwood, and Fitness have worked together for five years in scholarly and capacity building activities servicing the discipline of psychology in Australia. All three have worked collaboratively as office bearers on the national executive of Psychology of Relationships Interest Group of the Australian Psychological In their respective roles, the trio have organised national conferences, workshops and seminars on the topic of relationships, and in doing so, have maintained a strong interest in the professional development of professionals working in the area of psychology.

Dr Gery Karantzas (PhD) is a Lecturer in the School of Psychology and has developed a strong research and teaching record having received two dissertation awards (the Australian Psychological Society Psychology of Relationships Interest Group Thesis Award and the La Trobe University Research Thesis Merit Citation Award) and an International Study Program Award (Deakin University). He has attracted over $500,000 in internal and external competitive grant funds. These include the Australian Research Council, beyondblue and Academy of Social Sciences in Australia. His grant experience, project management, and expertise in research design and statistics are evident in his capacity to develop and carry out high quality research and evaluation in a timely manner. The research output from these grants has resulted in 25 referred journal and conference proceeding publications and numerous conference presentations. Moreover, Gery has consistently demonstrated the ability to organise and carry out capacity building and dissemination events at national and international levels. He has served on numerous national and international conference scientific and organising committees and organised an international roundtable on couple and family relationships. The event brought together some of the most highly regarded national and international scholars, policy makers and practitioners in relationships. As an outcome of this event Gery is co-editing a handbook on couple and family relationships to be published by Wiley-Blackwell in 2011. These are just some examples of Gery’s ability to carry out high quality scholarly activities.

Gery is an enthusiastic teacher that has a strong interest in teaching innovation. In 2003, he developed an entire first-year tutorial program using online multimedia technologies to engage students in learning and to track learners’ reflections over the course of the program. He developed knowledge in collaborative learning during his tuition of 3rd year advanced developmental students at the University of Melbourne. During this period he undertook specialist training to operate a multimedia collaborative learning space (CLS). Gery is the current unit chair of HPS304 The Social Psychology of Relationship. In 2009, he along with colleagues Tooley, Mussap and Macfarlane was awarded funding through the Deakin University Strategic Teaching and Learning Grant Scheme to investigate the role of collaborative approaches to learning in fostering graduate attributes in undergraduate students. Gery’s leadership and innovation has resulted in the receipt of three teaching awards: (1) A School of Psychology Service Award for Research into Teaching; (2) The Faculty of Health, Medicine, Nursing, and Behavioural Sciences (HMNBS) Innovation award; (3) a Deakin University Award for Teaching Excellence. In 2010 he was inducted into the Deakin University Teaching Leaders Forum and the Faculty of HMNBS Teaching Leaders Group. His role in both these groups is to provide leadership in teaching and learning policy and practice at a university and faculty level.

Ms Susie Macfarlane (BSc. (Hons.), GCHE) is a Research Fellow in Scholarly Teaching and Learning in the Faculty of HMNBS at Deakin University. Susie has a major leadership role in teaching and learning

academic development, and teacing research at the Faculty level. Susie was recognised with a 2008 Vice Chancellor’s Award for Outstanding Achievement in Teaching and Learning and was awarded an ALTC Citation in 2009. She is a member of the Deakin University Higher Education Research Group, serves on the Committee of the Deakin University Teaching Leaders Forum and is a member of the Faculty of HMNBS Teaching Leaders Group. Susie is a passionate educator who engages students in active and meaningful learning experiences that have great significance for them as individuals, and that result in high quality learning outcomes and student evaluations. Prior to undertaking her role as a research fellow, Susie was a subject coordinator of a 1200 student compulsory first year unit. In this role, Susie has undertaken major curriculum redevelopment, redesigned all assessment tasks, implemented innovative, student-centred assessment tasks and tutorials, and developed synchronous interactive online tutorials. Susie is an active researcher, having successfully obtained over $150,000 of external funding and published several peer-reviewed publications in the areas of health promotion and education. Susie is currently undertaking a PhD on the psychometric properties of the Australasian Universities’ Survey of Student Engagement (AUSSE). Associate Professor Greg Tooley (PhD) is the Head of the School of Psychology and, for the past three years, has been the Director of Teaching and Learning. In this capacity, Greg led the redevelopment of the entire School’s Teaching and Learning redevelopment plan. Among other things, this program has produced a new and highly successful workload allocation model, online availability of the entire undergraduate program of the School, and a Deakin University Psychological Society of more than 800 members. In the process, the School has improved dramatically in terms of its Student Evaluation of Teaching performance. The school went from being one of the poorest rated teaching Schools in the University to be the highest rated (in 18 months) and have maintained that status for 4 consecutive semesters now. Greg has won several Teaching and learning awards, including Faculty of HMNBS post-graduate supervisor of the year (2002) and, most recently, Vice Chancellor’s Award for Outstanding Achievement in Teaching and Learning and ALTC Citation. He is a member of the Teaching Leaders Forum and the Higher Education Research Group. Greg is an active researcher who regularly attracts external funding, publishes in quality international research journals and has supervised a number of HDR students to successful completion of their candidature. Dr Alexander Mussap (PhD) is the Associate Head of School Teaching and Learning in the School of Psychology at Deakin University. Alex has an interest in developing and evaluating approaches to education, particularly approaches that promote the development in our students of a professional identity, one that integrates knowledge and practice in Psychology. His teaching has been recognised through the receipt of the Deakin University Psychology Society’s Lecturer of the Year Award and the Vice Chancellor’s Award for Outstanding Achievement in Teaching and Learning and an ALTC Citation. Alex is an experienced researcher in the areas of clinical vision, body image, and culture. He has been awarded three first-authored ARC Discovery Grants totalling $359,000 to conduct research in these areas, has published 2 (B1) book chapters and 26 (C1) articles in international refereed journals of high standing, and has made numerous conference presentations of his research. Alex has a strong track record of research supervision. He has supervised an international Postdoctoral Research Fellow, 22 HDR students (8 as primary supervisor; 14 as associate supervisor, with 7 completions to date), 14 Honours students as primary supervisor, and 44 students of the 4th-year Graduate Diploma of Psychology as primary supervisor.

Dr Zoe Hazelwood (PhD) is an Early Career Academic who has demonstrated leadership in both teaching and research. Zoe is responsible for coordinating the first Faculty-wide first-year interpersonal communication unit within the Faculty of Health at QUT and is a previous ALTC Citation Award winner for her teaching of large inter-disciplinary units. Specifically Zoe’s teaching focuses fostering self-reflection in students thereby contributing to students capacities to integrate their university experience with their real-world experiences. She is perfectly placed to act as QUTs representative on this ALTC grant application. She has demonstrated high quality leadership in all aspects of academic scholarly activity having been a

longstanding office bearer on the national executive of Psychology of Relationships Interest Group of the Australian Psychological Society and organised national conferences in relationships. She has edited numerous conference proceedings and is an associate editor of the Journal for Relationship Research. She has published a number of articles in the area of relationships and in 2007 received Australian Psychological Society Psychology of Relationships Interest Group Thesis Award.

Professor Julie Fitness (PhD) is currently Chair of Academic Senate and previously the acting Dean of the Division of Linguistics & Psychology and Head of the Discipline of Psychology at Macquarie University. She is responsible for coordinating the second year elective into the Psychology of Human Relationships. Julie has demonstrated outstanding service and leadership in the areas of teaching and research. In her role as chair of Academic Senate, she is responsible for implementing policy regarding teaching and learning university-wide. Julie’s high level service and teaching roles mean that she is ideally placed to act as Macquarie University’s representative on this ALTC grant application. Julie is a distinguished researcher having published over 60 peer-reviewed papers and book chapters, four edited books, and is on the editorial boards of some of the most prestigious journals in the areas of relationships and social psychology. In 2008, she was awarded the Australian Psychological Society Psychology of Relationships Interest Group Research Award.

UNIVERSITY AUSTRALIA

Office of the Deputy Vice-Chancellor (Academic)

Geelong Waterfront Campus Geelong Victoria 3217 Australia

Ref: F2008j0045 Telephone +61 352278507 Facsimile +61 352278536 dvca@deakin.edu.au

4 April 2011 www.deakin.edu.au

Dr Carol Nicoll Chief Executive Officer Australian Learning and Teaching Council PO Box 2375 Strawberry Hills NSW 2012

Dear Dr Nicoll

Letter of endorsement

On behalf of Deakin University I am pleased to endorse the joint Innovation and Development grant application led by Dr Gery Karantzas at Deakin University to be conducted in partnership with Queensland University of Technology and Macquarie University titled 'Shaping the future of psychology through developing and assessing graduate attributes using collaborative learning'.

This project seeks to explore an important issue in Australian higher education, that is, how best to assess and foster graduate attributes in undergraduate psychology students and enhance their learning outcomes. These outcomes relate to specialist knowledge of the discipline, the application of this knowledge, critical thinking, research and communication skills. This project will refine a measure to assess psychology graduate attributes, and develop an innovative problem-based, collaborative learning approach to enhance development ofthese attributes and improve learning outcomes. The project has significant ramifications for the discipline Australia-wide.

This is a very strong proposal, lead by a very strong team that has the skills and competencies to carry out this project to the highest standard.

For these reasons, I am delighted to endorse this proposal without reservation. I trust that the ALTC will see the value ofthis proposal.

Sincerely

Professor Joe Graffam Pro Vice-Chancellor (Academic)

March 25, 2011 Director, Grants Scheme Australian Learning and Teaching Council PO Box 2375, Strawberry Hills NSW 2012 . Dear Sir/Madam, This letter is to confirm that the School of Psychology at Deakin University is pleased to endorse the joint Priority Project grant application with Queensland University of Technology and Macquarie University titled “Shaping the future of psychology through the developing and assessing graduate attributes using collaborative learning”. The project leader from our institution is Dr Gery Karantzas and the School of Psychology will support Dr Karantzas’ involvement in the project through the appropriate infrastructure support and teaching relief as required throughout the course of the project. The School of Psychology at Deakin University is delighted to lead this collaborative endeavour into the development and assessment of psychology graduate attributes, and strongly supports this important project. Sincerely, Associate Professor Greg Tooley Head of School, Psychology Faculty of Health, Medicine, Nursing and Behavioural Sciences

School of Psychology

Faculty of Health, Medicine, Nursing and Behavioural Sciences Melbourne Campus at Burwood 221 Burwood Highway, Burwood Victoria 3125 Australia Telephone 03 925 17365 Facsimile 61 + 3 + 924 46858 email: hos-psych@deakin.edu.au www.deakin.edu.au

Deputy V ic e-C hanc ellor, P rovos t

MACQUARIE UNIVERSITY NSW 2109 AUSTRALIA

P hone +61 (0) 2 9850 4891 Fax +61 (0) 2 9850 4892 Email provost@vc.mq.edu.au

25th March 2011 Director, Grants Scheme, Australian Learning and Teaching Council, Level 14, 300 Elizabeth Street, Surry Hills, NSW, 2010 Dear Sir/Madam, This letter is to confirm that Macquarie University is pleased to endorse the joint Priority Project grant application with Deakin University and QUT titled “Shaping the future of psychology through developing and assessing of graduate attributes using collaborative learning”. The project team member from our institution is Professor Julie Fitness. Professor Fitness is Chair of Academic Senate and is responsible for coordinating the elective unit entitled “Psychology of Human Relationships” within the Faculty of Human Sciences at Macquarie University. Given Professor Fitness’ current role as Chair of Academic Senate and her many years of experience as a distinguished teacher and researcher in the psychology of relationships, she is well placed to act as Macquarie University’s representative on this ALTC grant application. The overall aim of this project is to explore an important issue in Australian higher education: that is, how best to assess and foster graduate attributes in undergraduate psychology students, and to enhance learning outcomes. This project will benefit from Professor Fitness’ experience in fostering both scholarship and self-reflection in her undergraduate students, and in contributing to students’ capacity to integrate University-derived understandings of relationship issues with their real-world experiences. Macquarie University is delighted to partner with Deakin University and QUT in this collaboration. In my role as Provost at Macquarie University, I strongly support Professor Fitness’ participation in this project and commend this proposal to you without reservation. Yours sincerely,

Professor Judyth Sachs Deputy Vice-Chancellor (Provost)

Human S c ienc es

Department of Psychology

MACQUARIE UNIVERSITY NSW 2109 AUSTRALIA

P hone +61 (0)2 9850 7111 Fax +61 (0)2 9850 8062 Email psy@psy.mq.edu.au

25th March 2011 Director, Grants Scheme, Australian Learning and Teaching Council, Level 14, 300 Elizabeth Street, Surry Hills, New South Wales, 2010. Dear Sir/Madam, This letter is to confirm that the Department of Psychology at Macquarie University is pleased to endorse the joint Priority Project grant application with Deakin University and Queensland University of Technology titled “Shaping the future of psychology through the developing and assessing graduate attributes using collaborative learning”. The project team member from our institution is Professor Julie Fitness and the Department of Psychology will support Professor Fitness’ involvement in the project through the appropriate infrastructure support and teaching relief as required throughout the course of the project. The Department of Psychology at Macquarie University is delighted to be a part of this collaborative endeavour into the development and assessment of psychology graduate attributes, and strongly supports this important project. Yours sincerely,

Associate Professor Mike Jones, Acting Head of the Psychology Department