PROBLEMS OF EDUCATION IN THE 21 st CENTURY Vol. 75, No. 1 ...oaji.net/pdf.html?n=2017/457-1488299643.pdf · Tatjana Koķe, Nora Jansone-Ratinika, Rudīte Koka Rīga Stradiņš University,

PROBLEMSOF EDUCATION

IN THE 21st CENTURYVol. 75, No. 1, 2017

54

ISSN 1822-7864

FORMATIVE ASSESSMENT ON AN INDIVIDUAL AND AN INSTITUTIONAL LEVEL ON THE WAY TO QUALITY CULTURE IN HIGHER EDUCATION

Tatjana Koķe, Nora Jansone-Ratinika, Rudīte KokaRīga Stradiņš University, Riga, Latvia

E-mail: [email protected], [email protected], [email protected]

Abstract

Several issues are always on the agenda in higher education, most notably quality assessment methods and whether this methodology has a direct influence on teaching, and the improvement of teaching quality by including effectiveness, efficiency, reliability, responsiveness as well as empathy. On the one hand it is a logical and a justified movement towards the standardisation of quality assessment procedures, but on the other hand, integration of an individualised approach, which would allow assessing one’s performance in essence and providing solutions for different higher education practice specifics, which we can see both in higher education institutions internationally as well as locally.The purpose of quality assessment is to define must-do and must-have things, so that higher education institutions would integrate emerging tendencies.Transforming assessment is crucial for the performance of the entire institution, both in planning and in managing the everyday work of the institution, as well as by applying assessment methods to the academic staff and students individually. By acknowledging the results, additional decisions should be made, e.g., what strategies and methods should be used to promote teacher motivation, what should be done to achieve better study results, what do better performing students need, and how to approach those students who are performing poorly. This research aims to explore the formative assessment experience at the Rīga Stradiņš University (hereinafter RSU), which is crucial to nurture quality culture in higher education. The formative approach in quality allows you to build and strengthen the understanding of quality within the organisation showing that quality assessment is quality forming and altogether it is a continuous process.Key words: quality assessment, quality assurance, quality culture, higher education, formative assessment.

Introduction

Education sector has a major impact on global socioeconomic processes implementing changes in different fields and aspects. The requirement to continuously develop education services for the purposes of implementing different quality measurements can be explained by the fact that the field of higher education both directly and indirectly has an impact on regional economic indicators both in Europe and in the rest of the world,1 and higher education is one of the influencing factors on human quality of life (Loughborough University, 2016).

By providing its main function – knowledge creation, knowledge diffusion and knowledge transfer, the field of higher education makes a large contribution to the development of knowledge economy, which not only improves financial indicators, but also promotes innovation in the system (Donlagic & Fazlic, 2015; Heraud, 2016).

Higher education quality measurement requirements are no longer determined only by the Bologna Declaration, or successive national reports on the subject from 2001 until 2015,2 or the European Network for Quality Assurance in Higher Education (ENQA) created in 2003

PROBLEMSOF EDUCATIONIN THE 21st CENTURYVol. 75, No. 1, 2017

55

ISSN 1822-7864

for the monitoring of education quality, nor any successive routine evaluations, accreditation and auditing procedures determined by local national regulatory frameworks of the countries. Rather, higher education quality assessment requirements are made necessary because of the international education practice and the daily life of the education process – market expansion and diversification, which increasingly offers potential students their options according to their needs rather their opportunities (Sursock, 2015). Quality assessment aims at defining must-do and must-have things so that higher education institutions can integrate the emerging trends.

The cornerstone of continuous growth is frequent assessment of internal and external quality as well as promotion of result based innovation. In the search for the optimal model, our research focuses on comparison of internal and external quality aspects both on a regional and a European level, by engaging EU institutions and other organisations. External evaluation of initiatives provide universities on various levels with the opportunity to present their activities to external experts, acquiring independent analytical reports about the compliance of the institution’s priorities, with their desired progress, identifying significant correlations in the implementation of their activities. The openness of universities towards such research activities are a testament to their courage, which directly correlates with their orientation towards further development. The cooperation project initiated by the Ministry of Education and Science of the Republic of Latvia with International Bank for Reconstruction and Development (World Bank) should be mentioned as a good example of such a target activity. It is aimed at carrying out a study that would evaluate management of Latvian universities on their way to offering excellent quality of higher education in Latvia both to the Latvian society and to international students.3 RSU, also represented by the authors, was one of the seven universities that saw experts from major European universities arrive in June 2016. The final report, which describes each university’s profile and provides specific recommendations for improvement is issued in September 2016. Experts have drawn their attention to the fact that Latvian universities have shown positive progress by implementing democratic principles, which allow administrative and academic staff as well as students to participate in the management of the universities, although in the overall performance it is recommended to redirect future priorities from crisis policy to growth and sustainable development, which allow to pay more attention to the efficiency of the study process and research.4

The Concept of Quality in the Field of Higher Education

Quality assessment methods are of high topicality in the agenda of higher education institutions, especially the question of whether methodology used for internal and external assessment has a relevant influence on teaching, and the improvement of teaching quality by including effectiveness, efficiency, reliability, responsiveness as well as empathy. On the one hand it is a logical and a justified movement towards the standardisation of quality assessment procedures, but on the other hand integration of an individualised approach, which allow assessing one’s performance in essence and providing solutions for different higher education practice specifics, which can be seen both in higher education institutions internationally as well as locally.

Quality assessment and provision methods of higher education institutions often have specific primary emphasis and are focused on improving or regulating the organisational processes (in the case of process oriented quality management approaches), the assessment of the outcomes of activities (in case of assurance or evaluations approaches) or on the development of individual abilities (in case of quality development through professional training approaches). One of the primary issues in the field of higher education is promoting understanding on quality not just by analysing isolated factors and by determining criteria. System-wide analysis, which studies the level of the individual in deep, reviewing the institutional scope, is important for quality assessment (Ehlers & Schneckenberg, 2010; Sursock, 2015).

Tatjana KOĶE, Nora JANSONE-RATINIKA, Rudīte KOKA. Formative assessment on an individual and an institutional level on the way to quality culture in higher education



56

ISSN 1822-7864

The conceptually simple, yet frequently used ‘3P’ model looks at education as a complex system, where ‘Presage’, ‘Process’ and ‘Product’ variables interact with each other. Criteria used in these three dimensions have been categorised in more complex quality assessment models, of which each can be used for the evaluation of specific quality aspects (Gardiner, 1997; Gibbs, 2010).

Quality assessment process in higher education is characterised by:Formalised internal and external assessment procedures for the evaluation of various

planned processes;Evaluation criteria determined as a result of valid research results and communicated

on all levels;Accumulation of data acquired as a result of assessment and their interpretation

communication on various levels of the institution;Targeted follow-up procedures ensure various quality assessment and implementation

phases.In general, quality in higher education institutions is assessed based on these categories

(European Network for Quality Assurance in Higher, 2009): Policy and procedures for quality assurance – quality policy and quality standards

of the programmes to be introduced in higher education institutions in the implementation and assessment process;

Approval, monitoring and periodic review of programmes and awards – students have the right to make informed choices. A quality feature of higher education is full-fledged content, containing information on the potential study opportunities and execution of the programme, as well as the involvement of current students in the ongoing development of quality (Bukley, Soilemetzidis, & Hillman, 2015). Research across countries shows that students are becoming more demanding and are beginning to adequately question the value for money they receive (Department for Business, Innovation and Skills, 2016).

Assessment of students` performance – one of the key concepts of student performance assessment is directed at independent deep learning promotion, which has been acknowledged as a major factor of quality and an effective learning process. Directed independent learning is a process, in which students are taken through the curriculum content, education and assessment, and supported by staff and the learning environment, and students are encouraged to play an active role in their learning experience – either on their own, or in collaboration with their peers, staff and other students jointly engaging in purposeful learning activities within their study programmes, with staff playing a different role as ‘imparters of knowledge’ (Thomas, Jones, & Ottoway, 2015). Despite the fact that this principle has been prescribed and postulated in the teaching methods of higher education across various sources, it is not absolute in real time (Bukley, Soilemetzidis, & Hillman, 2015). One of the directed independent deep learning enhancing instruments is formative assessment – the road to communication and effective inception, design, quality assurance and enhancement promotion, which traditionally refers not only to students, but also to any other stakeholders in the field of higher education: students, potential students, and parents; academic and administrative staff, curriculum designers and educational developers; senior managers, professional bodies employers and others (Neves & Hillman, 2016).

Quality assurance of teaching staff – higher education institutions across the world pay more and more attention to teaching excellence and carrying out specific activities to ensure constant high-quality teaching resulting in high level learning (The Business Innovation and Skills Committee, 2016). This necessity is based on student opinion polls across various regional and thematic dimensions, which proves that students consider continuous teaching staff knowledge and teaching skill



57

ISSN 1822-7864

development to be one of the key aspects of learning quality. Students also appear to value structure in the way they are taught and assessed, reacting positively to large numbers of assignments and a good balance between summative and formative assessment methods (Neves & Hillman, 2016).

Learning resources and student support – the compliance of content and format of learning resources with study results is an important factor contributing to the quality of education in the digital era and at a time when learning is changing and is becoming part of the polemic more and more (European Commission, 2013a). A key indicator is to provide support for students both directly from the academic staff and the system order in general. Active engagement of students is also crucial to the overall experience. Students themselves recognise when they do not put enough effort, but when providing opportunities to engage with their peers and with staff can have a major impact on students feeling supported and getting the most out of their experience (Neves & Hillman, 2016).

Information systems – purposeful and efficient digitisation and prudent automation of data and processes has become the norm in education quality, though further development opportunities along with the rapid development of technology have become nigh endless.

Public information – it is equally essential to create the university’s public profile which should include quantitative and qualitative up to date, impartial and objective information on the programmes and other additional activities.

It should be noted that despite universities being organisations and state regional institutions that provide education, being international and inclusive creates challenges for developing universal quality standards and guidelines, which serve as effective assessment metrics. The pillars of education quality no longer are to be based on local activities, but rather encompassing aspects of the global spectre, which require a uniform framework and elasticity. In practice, it was proved when carrying out the Internationalisation Study at the RSU in 2014 and 2015.5 The Study focused on the pedagogical process with international students to acquire data and formulate arguments for the improvement of the study process, promoting the professionality of all students, and strengthening the university’s prestige and international brand. International student mobility in the acquisition of education is ever increasing, and this forces education systems and service providers to think about the attractiveness and compliance with international quality standards (European Commission, 2013b).

Strategic planning and quality assessment should be mutually inclusive as well as complementary processes, allowing to evaluate the internal process compliance of universities with external processes and global trends. Assessment in higher education institutions is at the core of quality ensuring, which also encompasses all strategically important segments – finances, human resources, physical environment and digital infrastructure, learning environment, administrative processes, the study process etc. Exact identification, estimation and fulfilment of stakeholder needs is at the core of a successful improvement of quality. Various studies have analysed stakeholders across various sections, generally including students, academic and administrative staff, parents, employers, partners, local communities, etc., combining internal or core stakeholders with external or secondary and indirect stakeholders (Kasetwar, 2008; Maric, 2013; Kettunen, 2014).

Quality management and development activities should have an impact on the learning and teaching process of the professional behaviour of stakeholders, incorporating new values, skills and attitudes. Frequently the introduction of a quality system is believed to be a natural outcome that improves teaching and learning, but what often is overlooked is the fact that quality as a result and a process is the negotiation process of the stakeholders, participating in the end result of the educational situation. Mutual agreement and investment across various stages promotes the fact that the stakeholders involved in the educational process act as competent quality managers in the education environment. This competence is most precisely




58

ISSN 1822-7864

characterised by the concept of quality literacy (Ehlers, 2007). There are four primary quality components-dimensions included in this concept:

Quality knowledge: This dimension includes theoretical and instrumental knowledge about today’s quality opportunities, most prominent approaches, strategies in teaching and ensuring the assessment, improvement and management of the educational process.

Quality experiences: This dimension shows the ability to purposefully apply the known quality strategies in action.

Quality innovation: This dimension includes abilities, which extend available tools and pertaining strategies. Modification of integrated quality strategies and creation of new innovations for the achievement of goals is important at this level.

Quality analysis: This reveals the ability to analyse individual results and quality development process contextually. It allows actors to evaluate various goals, set tasks and achieve them by considering the needs of various stakeholders. Critical analysis requires the ability of differentiation and reflection of existing knowledge and experiences in the light of quality development challenges (Ehlers, 2007).

The sequence of quality skill dimensions can change in practice, however it is important to keep in mind the cycles and the principle of purposeful actions.

The concept of literacy includes both subject knowledge, i.e. understanding, and processual action – competence. In regard to the interpretation of quality culture, misleading understanding and directly involved quality literacy heterogeneity is demonstrated by the differing understanding among the stakeholders of higher education. The most common conviction among academic and often administrative staff is the belief that they are responsive executioners rather than masters of quality standards. There still exists the opinion that top-down is an adequate direction of quality assessment, which is communicated as an internal regulation or must-do. Almost all stakeholder groups – academics, administrative staff, management, students, and quality control managers often demonstrate the belief that quality criteria and standards are an administrative bureaucracy, which has little effect on daily work and is distant to the performers. Opinion polarity can be observed also on the question of purpose and necessity of quality assessment. Some of the stakeholders believe that the aim of quality politics and system is its existence, because it is required by external normative regulations on European and national level. The other half is represented by the so called ‘evaluative thinking’ or understanding of the quality culture and respectively the assessment as a way of thinking or way of philosophy and action throughout the all that allows to identify needs, define goals and reflect on the completion of each task with the purpose to improve the quality as much as possible (Hoidn & Karkkainen, 2014; Earl & Timperley, 2015). It is humanly understandable that both opinions represented are not homogenous, since in definite situations the stakeholders are the subject of assessment by themselves. The fact that students and representatives of academic staff more often re-evaluate the necessity to evaluate the quality of the education process and assessment at various stages is a positive trend. In addition, they also strive to initiate formalised and collegial daily quality assessment activities to acquire analytics and to improve the ability to evaluate pros and cons of their own and peer’s performance, as well as constructively improve the professional performance (Sursock, 2011; Vettori, 2012; Neves & Hillman, 2016; Kearney, Perkins & Kennedy-Clark, 2016).

Quality literacy as a goal and preferable result introduces the discussion on the concept of quality culture. Holistic quality culture forms process-oriented quality management and an element of synergy in quality literacy.

Quality Culture as Quality Implementation and Assessment Plan

Ever since the beginning of the 60s, the concept of culture in literature debates has acquired an ever increasing and more prominent role in various science fields, including



59

ISSN 1822-7864

individual and institutional aspects of higher education by emphasising the ideology of diversity, increasing interest in organisation management in an institutional and disciplinary context, the importance of symbolic, emotional or irrational components as well as criticising quantitative methodology in management and assessment etc. (Clarck, 1972; Schein, 1985; Maassen, 1996).

The concept of culture is primarily associated with patterns of human activity and the symbolic structures that give them their meaning (Williams, 1983), which in various contexts acquires the importance of the umbrella term. At the core of the quality culture idea is the emphasis on change from quality control, where rule and quality management implementation is key, to a cultural level that assimilates the gradual introduction of individual and institutional standards and regular improvement at all levels. Even though the basic principles can be described relatively easily, still fundamental empirical studies on the phenomenon of the concept and its role in implementation and types of actions are needed (Ehlers & Schneckenberg, 2010). Whereas it is difficult to date a specific moment of change, in general it marks the beginning of a new era and gradual alienation from approaching quality as something mechanistic and top-down by moving towards a new understanding that quality development is a question of organisation culture creation, based on shared values, accepted meaning systems (Pettigrew, 1979) in organisation’s operating philosophy (Ouchi, 1981), necessary competences and new professionality as a result of it all. A lot of support has been provided by the intensive work with quality control systems and standards of accreditation and procedure development over the past few decades (Bollaert, et al., 2007), which has allowed the approach to move from control to change, from introduction to development, from standard execution to innovation and look at control management systems, instruments, competencies, individual and organisational values in a structured framework of holistic culture elements. This kind of change in thinking requires cautious evaluation of not only visible and tangible factors, but also those whose identification is not as simple, whereas their impact on behaviour and performance is considerable. These factors (communication types, shared beliefs, values, symbols and rituals) can be compared to implicit unwritten rules, which have never been formalised still are known at all levels and generally observed.

In a more general way one of the organisational culture qualities is the teaching culture. On an institutional level one of the quality aspects is gradual teaching culture formation, which is coherent with the basic principles of a democratic education by promoting mutual interdependence based on the development of basic principles and self-organised action of organisations as educational institutions (Kokare, 2011; Kools & Stoll, 2016). In regard to the concept of learning culture, it is important to understand that learning at the university cannot be applied only to the study process – in a sustainable organisation every employee learns to reach individual progress and form the overall results of the organisation. In return, the organisation is responsible for providing support and teaching its community. To achieve the above-described goals, the interaction of such quality culture elements as communication, cooperation, top-down and bottom-up combination, respect and promotion of staff autonomy that passive policy receivers would turn into active contributors is of primary meaning (Davies, Douglas, & Douglas, 2007; Harvey & Stensaker, 2008; European University Association, 2012).

The nurturing of quality culture in higher education is focused more on structural management and cultural psychological component interaction in a constant education improvement process.

Leadership and communication are considered to be key quality culture forming factors in the structural-managerial and cultural-psychological dimensions. The role of leaders is associated with their ability to localise resources, clarify roles and responsibilities, form partnerships and optimise human and process management. Whereas adequate communication is one of the prerequisites in promotion of quality strategy and policy propagation, as well as evaluation of achieved results, identification and rallying of staff values and beliefs. Quality culture mechanisms include increasing staff loyalty and commitment, shared ownership,




60

ISSN 1822-7864

empowerment and continuously renewable and perfected knowledge (Bendermacher, Egbrink oude, Wolfhagen, & Dolmans, 2016).

According to the European University Association quality culture is an organisational culture that intends to enhance quality permanently and is characterised by two distinct elements: on the one hand, a cultural-psychological element of shared values, beliefs, expectations and commitment towards quality, and on the other hand, a structural-managerial element with defined processes that enhance quality and aim at coordinating individual efforts. The explanation shows aspects of two categories, which have an impact on quality culture. The so called hard aspects include quality management, strategies and processes, whereas soft aspects include values, beliefs, commitment etc. (European University Association, 2006). Despite varying structural elements, structural-managerial and cultural-psychological levels should be linked via communication, participation and trust on the individual and institutional level (European University Association, 2010).

The current heterogeneous understanding and challenge to fully integrate the idea of the culture in the practice of universities help to explain the concept of quality culture that has been relentless ever since the 90s and in fact currently experiencing a new wave of popularity in European higher education.

Despite the popular philosophy and developed support network in practice, the understanding of quality culture as one in which everyone is responsible for quality in the organisation, not just quality controllers, is frequently misunderstood among various members in the field of higher education (Harvey & Green, 1993). Academic staff in particular have been very reluctant to engage with management schemes and procedures, which are found to be overly bureaucratic and demotivating (Morley, 2003). The concept of quality culture is one answer to this problem, complementing the structural dimension of quality assurance (quality management handbooks, process definitions, instruments, tools) with the dimension of values of an organisation, relating to the commitment of its members, the underlying values, skills and attitudes (Ehlers, 2009).

An important step in developing the concept’s key principles was achieved in the context of EUA’s Quality Culture project,6 which was launched in 2002 to assist universities in their efforts to develop and embed an internal quality culture as well as to encourage the dissemination of existing best practices in the field of quality assurance. From this perspective, it was found that a quality culture cannot be simply equated with the institutional quality assurance system – although the system forms an important part of it – it being built on the values and practices that are shared by the institutional community and have to be nurtured on many levels and by various means at the same time.

Quality culture interpretation shows two distinctions, which to an extent demonstrate consecutive stages. One of those is quality culture as a tool for reflection (European University Association, 2010). On an individual level it is a reflection of their own and colleague performance as well as abilities and possibilities of development. At the same time, on an institutional level reflection framework is granted by the study of strategies and policies; tools and practices; principles and underlying assumptions. The other aspect reveals self-reflection on a growth perspective as a prerequisite for analysis, and the road to the next step – enhancement (European University Association, 2012).

Despite increasing emphasis on research and the necessity to strengthen it in practice, the concept of quality culture in the works of education theorists and practitioners is being debated extensively( (Shepard, 2000; Europena University Association, 2007; Ehlers & Schneckenberg, 2010; Bendermacher, Egbrink, Wolfhagen & Dolmans, 2016). Many authors and organisations fill the concept of quality culture with a wide array of contents that shows diverse emphasis, but in general share universal motives (Ehlers & Schneckenberg, 2010). Quality culture is explained as a process, a combination of rules and regulations, based on institution stakeholder knowledge, skills and attitudes (Ehlers & Schneckenberg, 2010). Targeting to the field of higher education, it is described as academic quality culture – the system of values and beliefs of higher



61

ISSN 1822-7864

education representatives that allows to build a quality-based approach, unified standards and to represent them in the work of education (Clarck, 1972). Even though quality culture is believed to be the element of an organisation culture, they are mutually consequential and reversibly connected (Bendermacher, Egbrink, Wolfhagen & Dolmans, 2016). Quality culture cannot be seen as something that an organisation has, rather what that organisation is, thus becoming a matter of identity (Ehlers & Schneckenberg, 2010).

The clash can still be identified in that quality culture in practice is not associated with an individual’s identity a priori, and among various stakeholder target groups the assessment initiative of quality is falsely presumed as doubts of employee’s compliance with standards, respectively, his value and loyalty towards the organisation, which is rooted in the understanding of assessment quality control as a primary purpose. The shift of emphasis in the understanding promotes the creation and maintenance of a purposeful quality culture, based on the idea of quality culture not as a constant status but rather than a relentless and shared quality forming a formative process.

Formative Approach to the Assessment in the Process and Outcomes: RSU Case

Formative assessment in higher education is one of the forms of study result evaluation; however, it reveals only a narrow formative approach in the context of quality culture. It acquires a new meaning, when we expand the concept of the ‘formative’ not only by reducing it to student performance evaluation on an individual level, but also attributing it to the provision of quality at an institutional and even global scale. Formative approach to quality allows to build and strengthen the understanding of quality within the organisation, showing that quality assessment is quality forming and altogether it is a continuous process.

To reveal the above-described approaches empirically, the paper contains insight into the implementation and continuous development of formative approach at individual and institutional level in the higher education institution of Latvia – RSU.

RSU7 is a modern, prestigious university recognised in Europe and in the world, with the human being at the core of its value system. The university provides opportunity for specialists in health care and social sciences to become valued experts in their field. The knowledge, skills, attitude and competences acquired throughout the study process are compatible with the requirements, human traditions and professional environment set out by Europe and the world, forming a stable basis for life-long learning. RSU implements up-to-date approach to ensure high level study and research quality focusing on continuous development of the university. The quality policy and strategy of the university include the following values: person, team, result and loyalty, and it is envisioned that all stakeholders should participate in the achievement of these values. By implementing the quality policy, additional principles are considered, which are all based in quality culture and ideas of formative approach:

Student-centred approach – involve students in the development of the study process, promote the equality among academic staff and students and as well as cooperation creating a modern study, research environment and infrastructure;

Partnership – strengthen an interdisciplinary approach in health care and social sciences;

Quality – maintain and develop a management system that would implement a comprehensive and result-based quality development approach.

RSU was the first university in Latvia that acquired the international quality management certificate of European certification organisation Bureau Veritas Quality International already in 2002. In 2009, as a part of the requalification, it acquired ISO 9001:2008. This is only one of a formal testaments proving the continuous development of RSU and its’ efforts to develop quality standards, which improve the prestige of the university, allowing students and staff to join its ranks and become part of it.




62

ISSN 1822-7864

To assess current achievements and determine growth opportunities in the quality monitoring, RSU Student Council initiated the application to acquire Peer Assessment of Student Centred Learning (hereinafter – PASCL) in February 2016. Upon examination of RSU self-evaluation report and observations at the site, PASCL representatives concluded that RSU is a student-centred university, which actively involves students in the study improvement process, ensures communication at all levels, and generally is distinctly oriented towards a frequent evaluation of its achievements and improvement of activities. Recommendations covered by the PASCL expert report (Dimovska, Skadborg, & Szabo, 2016) helps RSU to work on different directions more systemically (improvement of e-study environment functionality and content, development of study programme quality indicators, evaluation process and content update, evaluation unification, improvement of transparency and objectivity etc.) to improve identified things in the organisation and move towards the development of high quality culture.

Higher education specialists in several universities have studied changes in the assessment policy and practice in the field of the higher education, indicating the necessity and direction of transformations: to improve potential for student learning, increase student satisfaction, improve value for money, increase suitability of assessment methods to the outcomes of 21st century higher education, fairer representation of student achievement, greater confidence in academic standards (Ball, Bew, & Bloxham, 2012).

Transforming assessment is a very serious challenge for each teacher, student and university in general. Therefore, universities first develop research-based transforming assessment strategies, in which one of the key directions is the teacher continuing education.

Quality theory and empirical studies show that teacher professionalism and unified teaching philosophy of the university definitely are the key factors of education quality and its continuous development. In symmetric analogy, learning is considered a prerequisite for teaching improvement, therefore on an institutional level we should provide continuing education or professional development opportunities for academic staff. To maintain and enhance capacity of academic staff, initiate educational improvements and bring their experience and expertise into practice, RSU has established the Centre for Educational Growth. The aim of this centre is to provide support for the study quality development by providing regular study process analysis and by organising continuing education opportunities for academic staff in compliance with RSU vision. The teacher continuing education is based on quality culture principles and is one of the tools, which allows to develop transforming assessment strategies in the university, by ensuring a formative approach to institutional quality assessment and on an individual level providing support for the integration of formative assessment in the study process.

In addition, within the framework of teacher continuing education, which includes thematic activities, the university management and academic staff suggested to focus on assessment of study courses by developing a formative assessment system and integrating it in practice.

RSU pays extra attention to strengthen experience-based understanding on formative assessment as learning process at the university. It can be achieved by providing students, academic staff and any other administrative units with feedback on their performance and by including colleagues in analysing and defining further goals so that they are aware of the growth of quality culture at all levels and it can become an integral part of teaching and learning, and systematically embedded in curriculum practices.

Formative assessment principles at RSU: Study results. Clear and measurable study results in study programmes, courses and

their units (lectures, seminars). They include a clear and measurable description of necessary knowledge, skills and attitudes.

Inclusive and learning promoting environment. Promoting environment where there is no place for fear of making mistakes either for teachers or students. Ensuring availability of a wide array of learning resource.

Feedback for reflexion. Feedback is either formative or summative, and it helps to



63

ISSN 1822-7864

move forward. Fixed assessment information (descriptive and analytical in the case of formative assessment) is necessary to provide an understanding of what needs to be done next. In the case of summative assessment, it is done on a 10 point scale with clearly defined criteria. Formative assessment feedback is frequent, after each important question of the content so that students can move forward in mastering the content, and the teacher can follow the progress of each student. When the teacher asks his students “Do you understand?” – it does not give the teacher or the students a clear answer about specific knowledge and how it should be improved in the future. In the same way, teacher’s comments: “Good job, well done!” is not sufficient feedback, because it does not explain what from the completed is assessed as high evaluation, nor does it provide the student with direct ideas on how to plan their improvement. Not all descriptive conclusions provide qualitative feedback. The key task of feedback is to give frequent and clear information on when, how and on what level study results were accomplished and what should be done to improve.

Formative assessment not only ensures feedback to students, but also requires active feedback from them (Petty, 2014). To ensure that the feedback works both ways as part of the learning process, RSU provides it both on an individual and an institutional level:

Once per semester as part of performance management system the quality of teacher’s performance is analysed. At first, each teacher evaluates his performance considering self-promoted goals and ones defined by the head of department; the filled in self-assessment form is electronically submitted to the head of department who afterwards invites the teacher to review his work quality and potential development. Then the head of department submits the filled in form with comments to the dean of faculty; and at the end on the basis of the performance assessment forms the next period is planned and communicated to all teachers by the head of department in the department meeting.

Another solution for experience exchange and assessment is classroom visits, in which academic staff of the same department attends lectures and practical classes held by their colleagues and fills in an educationally structured observation protocol, and discusses the performance of their colleagues.

At the end of each course students fill in a centralised electronic survey. Students have to review the content of the course, course materials, teacher’s performance, organisation of the general study process as well as the environment and answer any specific question included either by the course teacher or the department. Senior year students also have to submit study programme evaluation surveys, which afterwards are analysed by the director of study programme to evaluate the programme’s strengths and weaknesses, opportunities and threats, and to provide students with feedback on planned improvement.

The most frequent feedback on an individual level as part of the study process is student knowledge and skill summative tests and information acquired as part of formative assessment, which often is descriptive. Each assessment type serves its own goals. The goal of summative assessment is to evaluate the performance and provide feedback on it, whereas the goal of formative assessment is to acquire information which afterwards can be used to improve one’s performance (academic staff, students).

RSU study process is considered as an interaction based on teacher-student relationship and study content in which academic staff and students are equal partners. Assessment is an important aspect of the institution’s activities both in planning and managing the work of the institution generally and also by applying individually to each teacher and student. Acquired results help to form new decisions, e.g., strategies and methods that should be used to improve teacher’s motivation, study result acquisition, necessities of well performing students and those who lag behind with the mastering of courses. How to promote the individual development of




64

ISSN 1822-7864

each teacher, how to develop student-driven learning process, and how development promoting feedback can be acquired in this process? These questions are equally important both for RSU and other universities in Europe and the world.

Methodology of Research

To reach the aim of research the mixed methodology was acknowledged to be the most appropriate as it provides the possibility to cover methods of qualitative and quantitative data acquisition and analysis in which the quantitative empirical data supports the research of wider field and the identification of tendencies while the qualitative methods – deepened data interpretation.

Sample Selection

To understand the everyday practices of formative assessment in the educational process at RSU and its impact on student achievements, the research among the 1st year dentistry students studying Human Molecular Biology in 2015 and 2016 has been carried out.

The selection is made of 53 students: all students who initiated studies in the Faculty of Dentistry in this academic year. An experimental student group, consisting of 22 students as well as a control group of 31 students were created. The experimental group consists of students studying this course in Latvian, while the control group consists of those who study this course in English. In the study participated all 53 students and as well as 12 independent representatives of academic staff. In the definite case the key factor was the willingness of teacher to participate in the implementation of formative assessment in the study process: lectures and practical classes.

Instrument and Procedures

The following methods were used in the research: survey, focus group discussions, structured educational observation. The research was carried out at several stages:

The head of study course developed formative assessment tasks for each lecture and practical class in compliance with the course content.

To create a varied and content appropriate formative assessment tasks a task classification matrix was created. In the creation of these tasks the emphasis was put on results to be achieved and task classification by ascertaining level (elementary algorithm task and problem-task); task classification by response type (short answer, structured essay); tasks classification by execution type (written, oral, electronic, voting system). Students assessed tasks created by expressing their opinions in a survey and in a structured educational assessment focus group discussion where academic staff was also present.

Throughout the study course both in lectures and in classes, the teacher of course offered only students of experimental group to do these tasks.

12 independent representatives of academic staff participated in lectures and classes by observing the implementation of the formative assessment method and recording feedback in the protocols drafted specifically for this purpose, studying the following criteria:

1) Student performance during the lecture or practical class which approves their involvement in the formative assessment substantially (e.g., students instead of being passive receivers become active co-operators – ask questions, justify answers, analyse mistakes etc.);

2) Teacher performance that approves meaningful comprehension and usage of formative assessment in the study process (e.g., does not neglect students’ questions;



65

ISSN 1822-7864

encourages to reason the answers; provides the immediate feedback; encourages students to (re)think and analyse etc.);

3) Meaningful usage of the information and communications technologies (e.g., to receive the feedback – voting controls; to explain – interactive whiteboard etc.).

Cooperation among teacher and students in lectures or practical classes, and the use of information technologies. After data acquisition a structured educational observation protocol – qualitative analysis was filled in.

At the end of the study course students of experimental group are asked to fill in a questionnaire by the head of study course. The questionnaire was made with the purpose to find out students’ thoughts on efficiency of formative assessment tasks both in lectures and practical classes. Therefore, only open questions were defined. After aggregation of the surveying and visiting protocol results, the head of study course organised three focus group discussions with the students of experimental group participating in the course and two teachers to get detailed feedback on formative assessment experience. The number of focus groups corresponds the number of students’ groups (22 students during the practical classes are divided into smaller groups of 7-8 students per each). In total one focus group discussion took place in each group of students. After data acquisition the transcription of three focus group discussions was carried out.

Data Analysis

Questionnaire results were processed with MS Excel by grouping them according to unifying and distinguishing aspects. And at the end of the course the results of summative assessment of experimental and control group students were processed with MS Excel and SPSS-Statistical Package for Social Science as well.

The summative assessment results of the control and experimental group were analysed according to their correspondence to normal distribution (Kolmogorov-Smirnov test) and tests of Mann-Whitney and Spearman Correlation were applied to clarify the existence of statistically significant differences.

The data acquired within the transcription of focus group discussions and structured educational observation protocols were processed with MS Excel, identifying thematic units corresponding to the previously defined analysis criteria in both data sets.

Results of Research

The focus group discussion of the representatives of academic staff showed that various formative assessment methods exist in other courses as well, but frequently there are challenges with the feedback. For example, sometimes tests filled by students during the class are submitted and corrected only in the next class. In this case feedback is given late, because the next class will contain other emphasis in terms of the content, and often students no longer remember the content of previous class test. This approach is to be used when:

Students answer essay-type questions and the evaluation of their answers takes more time for teacher, therefore immediate feedback is impossible;

The evaluation is given as part of descriptive comments from which the students gain a clear understanding of what needs to be improved.

Within the study process academic staff also uses mutual evaluation of students and self-evaluation by organising various group projects and by analysing situations, playing role plays or by participating in discussions. Voting was found to be an effective and quick tool of feedback. It brings several benefits:

Immediate feedback for teacher and students; It is possible to identify mistakes of each student and understood the reason of

making mistake;




66

ISSN 1822-7864

Teacher realizes the speed of learning.The results of student questionnaire and focus group discussion allow us to conclude

that: 82 % of students feel involved in lectures, which is expressed by them saying: “...

working like this (i.e. purposefully organised formative assessment) helps to control myself; ...I can quickly understand whether I understood everything, …I immediately understand my mistakes...”

27 % of students master contents of the lecture demonstrating: “...other times theory seems dry and you don’t understand how to apply it, but if you have a task, you immediately know why you’re studying that...”

90 % of students are not passive listeners during lectures: “…it keeps me on my feet all the time...”; “… I don’t feel bored...”; “...You have to start thinking...”; “...Usually my attention holds for 45 minutes, but now I’m involved for all 90 minutes...”

45 % of students in these lectures train their attention and memory: “...when you attend the lectures, you understand everything and then it is easier to remember things in classes...”

Students have also listed some preferable improvements: “...give us more and more difficult tasks; include tasks on e-studies as homework; differentiate tasks more precisely, starting from the easiest and moving to the more complex...”

The feedback received in the evaluation protocols and focus group discussions of academic staff are as follows:

“....in the lecture we constantly receive feedback on student understanding...”, “...teacher explanation was very nicely and easy to understand. Talking speed is good. Taking care of all students equally and individually...”; “....constant student involvement by asking questions and encouraging to finish sentences...”; “....regular and diverse tasks in the lectures...”; “....the tasks are effective, because multiple choice tasks include answers that are “tricky”...;

“...students participated actively and looked motivated. They focused on subject to gain a deeper understanding...”; “…students are ready to comment and explain the answers given...”; “...they comment themselves, why they made the mistake...”; “....unclear questions are discussed...”; “...the opportunity to get an answer from each student...”

“...smart board and boards in general as well as voting controls create a maximally visible formative assessment example use...”; “…feedback is immediate and personal…”; “…technologies are used concentrating on the goal…”; “...question diversity…”; “…the voting system is very good to see students’ understanding levels and weak points...”; “…opportunity to avoid incorrect answers even before the colloquium...”

In general, both teacher and students suggest to develop infrastructure to support changes, which includes institutional assessment regulations and philosophy, and the use of technologies to enhance assessment practice, improve feedback and streamline assessment management (e-feedback).

At first, analyzing the summative assessment of control and experimental group their correspondence to normal distribution (Kolmogorov-Smirnov test) has to be detected; it is concluded that data do not correspond to normal distribution, therefore, methods of non-parametric statistics have been used. It is defined that summative assessment test is test which covers tasks corresponding to all levels of cognitive performance (according to Bloom Taxonomy) and the results of which can be expressed in 10 point scale. In the 2nd semester of this study course students had 5 summative assessment tests (4 colloquiums and one exam).

To clarify if in the summative assessments of students there exist statistically significant differences, tests of Mann-Whitney and Spearman Correlation were applied.




68

ISSN 1822-7864

How to change the assessment policy and practice in the institution, how to promote each teacher’s individual development, how should the student further his learning process, and how can we acquire development promoting feedback in this process? These questions are equally important for RSU as well as other universities in Europe and in the world.

Although quality culture understanding philosophy in the context of higher education has been described widely, in real life an ever rising decrease of identifiable natural bonds or even separation of subcultures (management, academic and administrative staff, students, subject specific communities) are seen, which prevent all stakeholder interaction in order to improve their shared commitment to educational quality within the organisation. Therefore, the task of empirical studies is still to enlighten the necessity for a methodological framework of quality culture aspects and further detailed study in order to promote learning and teaching quality innovations in the practice of higher education institutions, that would be based on a mutual ‘psychological contract’ (Rowley, 1996).

One of the further fields of research would be formative approaches as a prerequisite for in-depth research of quality culture creation and development, its integration principles both in theoretical and practical aspects and adaption of methods on an individual and institutional level. Implementation and activation of the formative approach and assessment in the educational and administrative process at the university considerably helps unified forces to nurture a people-focused and student-centred community mind-set. In the change of thinking and beliefs, the leadership can play an important role, as it can initiate and promote progress of the process in stakeholder communication, which helps to disseminate mutual ideas and provide with regular support in daily work.

Conclusions

Theoretical literature and analysis of empirical data show that quality culture is a socially constructed phenomenon and a complex profile integrated in the culture of an organisation, which should be viewed both individually and contextually in quality implementation, provision and evaluation, taking into account the ‘hard’ and ‘soft’ aspects of synergies.

The work that has been started at RSU continues by encouraging more and more targeted academic and administrative actions in formative assessment, by not only measuring the end result, but also interacting with the development of the organisation culture, including new values and understanding about the direction of future development and anchoring it in rituals, symbols, actions and creating a unified formative approach to quality culture as key motivator of actions.

RSU study process is considered an interaction based on teacher-student relations and study content in which academic staff and students are equal partners. Assessment is an important aspect of RSU activities both in planning and managing the work of the institution generally and also by applying individually to each teacher and student. Acquired results help to form new decisions, e.g., what strategies and methods should be used for the improvement of teacher motivation, study result acquisition, what more is needed for well performing students, and those who are lagging behind with the mastering of courses. Upon data analysis, it is possible to conclude that formative assessment has a positive impact on summative assessment results. This is evidently expressed in students’ surveys and in the focus group discussion: “I realised my mistakes right away; I repeated the material; better not to know in the class than in the colloquium; I felt good after lessons, because I understood everything; lessons had an atmosphere, in which I was not afraid to ask in order to understand.”

Formative assessment influences not only the summative student achievement, but also improves the learning environment, which is an important indicator of quality culture. The student centeredness, partnership and quality are the most essential key factors promoting individual growth of each stakeholder and the institution as a whole.



69

ISSN 1822-7864

References

Ball, S., Bew, C., & Bloxham, S. (2012). A Marked improvement: Transforming assessment in higher education. A Higher Education Academy. Retrieved from: https://www.heacademy.ac.uk/enhancement/frameworks/framework-transforming-assessment-higher-education#sthash.1ff7SMbJ.dpuf.

Bendermacher, G., Egbrink oude, Wolfhagen, I., & Dolmans, D. (2016). Unravelling quality culture in higher education: A Realist review. The International Journal of Higher Education Research, 1-22. doi:10.1007/s10734-015-9979-2.

Bollaert, L., Brus, S., Curvale, B., Harvey, L., Helle, E., Jensen, H. et al. (2007). Embedding quality culture in higher education. A Selection of papers from the 1st European forum for quality assurance. Brussels: European University Association.

Bukley, A., Soilemetzidis, I., & Hillman, N. (2015). The 2015 student academic expierence survey. York: The Higher Education Academy.

Clarck, B. (1972). The organizational saga in higher education. Administrative Science, 17(2), 178-184.Davies, J., Douglas, A., & Douglas, J. (2007). The effect of academic culture on the implementation of the

EFQM excellence model in UK universities. Quality Assurance in Education, 15 (4), 382-401. doi:10. 1108/09684880710829965.

Department for Business, Innovation and Skills. (2016). Impact assessment higher education and research bill. London: Crown copyright.

Dimovska, S., Skadborg, M., & Szabo, M. (2016). PASCL report. Rīga: Rīga Stradiņš University. Retrieved from: http://www.rsu.lv/eng/about-rsu/news-and-events/university-news/1016-rsu-recognised-as-a-student-centred-university.

Donlagic, S., & Fazlic, S. (2015). Quality assessment in higher education using the SERVQUAL model. Management, 20 (1), 39-57.

Earl, L., & Timperley, H. (2015). Evaluative thinking for successful educational innovation. OECD education working papers. 122. Paris: OECD Publishing. doi:10.1787/5jrxtk1jtdwf-en.

Ehlers, U. (2007). Quality literacy - competences for quality development in education and E-Learning. Educational Technology & Society, 10 (2), 96-108.

Ehlers, U. (2009). Understanding quality culture. Quality Assurance in Education, 17 (4), 343-363.Ehlers, U., & Schneckenberg, D. (2010). Discovering the unknown territory - how move from control

to culture in higher education quality. Changing cultures in higher education. Heidelberg: Springer.

European Commission (2013a). High level group on the modernisation of higher education. Report to the European Commission on improving the quality of teaching and learning in Europe’s higher education institutions. Luxemburg: Publications Office of the European Union.

European Commission. (2013b). Communication from the Commission to the European Parliament, The Council, The European Economic and Social Committee and the Committee of the Regions. European higher education in the world. Retrieved from: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013DC0499&from=LV.

European Network for Quality Assurance in Higher (2009). Standarts and guidelines for quality assurance in the European higher education area (3rd ed.). Helsinki: European Association for Quality Assurance in the Higher Education.

European University Association. (2006). Quality culture in European universities: A Bottom-up approach. Report on the three rounds of the quality culture project 2002–2006. Brussels: EUA.

European University Association (2010). Examining quality culture part I: Quality assurance processes in higher education institutions. Brussels: EAU.

European University Association (2012). Examining quality culture part III: From self-reflection to enhancement. Brussels: EUA.

Europena University Association (2007). Embedding quality culture in higher education. A Selection of papers from the 1st European Forum for Quality Assurance. Brussels: EUA.

Gardiner, L. (1997). Redesigning higher education: Producing dramatic gains in student learning. ASHE-ERIC higher education report. Washington DC: Association for the Study of Higher Education.

Gibbs, G. (2010). Dimensions of quality. York: The Higher Education Academy.




70

ISSN 1822-7864

Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18 (1), 9-33.

Harvey, L., & Stensaker, B. (2008). Quality culture: Understandings, boundaries and linkages. European Journal of Education, 43 (4), 427-442. doi:10.1111/j.1465-3435.2008.00367.x.

Heraud, J. (2016). A New approach of innovation: From the knowledge economy to the theory of creativity applied to territorial development. Journal of the Knowledge Economy, 1-17. doi:10.1007/s13132-016-0393-5.

Hoidn, S., & Karkkainen, K. (2014). Promoting skills for innovation in higher education: A Literature review on the effectiveness of problem-based learning and of teaching behaviors, OECD education working papers. Paris: OECD Publishing. doi:10.1787/5k3tsj67l226-en.

Kasetwar, R. (2008). Quality in higher education. University News, 46 (20), 6-12.Kearney, S., Perkins, T., & Kennedy-Clark, S. (2016). Using self- and peer-assessments for summative

purposes: Analysing the relative validity of the AASL (Authentic Assessment for Sustainable Learning) model. Assessment & Evaluation in Higher Education, 41 (6), 840-853.

Kettunen, J. (2014). The stakeholder map in higher education. International Proceedings of Economics Development and Research, V, 7.

Kokare, M. (2011). Mācīšanās organizācijā kā pedagoģiskā procesa perspektīva. [Organizational learning as a perspective of pedagogical process]. Promocijas darbs skolas pedagoģijā. Rīga: Latvijas Universitāte.

Kools, M., & Stoll, L. (2016). What makes a school a learning organisation? OECD education working paper. 137. Paris: OECD Publishing. doi:10.1787/5jlwm62b3bvh-en.

Loughborough University (2016). Times higher education student expierence survey. Times Higher Education. Retrieved from https://www.timeshighereducation.com/student/news/student-experience-survey-2016-results#survey-answer.

Maassen, P. (1996). The concept of culture. Tertiary Education and Management, 1 (2), 153-159.Maric, I. (2013). Stakeholder analysis of higher education institutions. Journal Interdisciplinary

Description of Complex Systems, 11 (2), 217-226.Morley, L. (2003). Quality and power in higher education. Berkshire. Philadelphia: Open University

Press.Neves, J., & Hillman, N. (2016). The 2016 student academic expierence survey. York: The Higher

Education Academy.Ouchi, W. (1981). Theory Z: How American bussiness can meet the Japanese challenge. Addison-Wesley:

Reading MA.Pettigrew, A. (1979). On studying organisational cultures. Administrative Science Quarterly, 24 (4), 570-

581.Petty, G. (2014). Teaching today: A Practical guide (5th ed.). Oxford: OUP.Rowley, J. (1996). Motivation and academic staff in higher education. Quality Assurance in Education, 4

(3), 11–16. doi:10.1108/09684889610125814.Schein, E. (1985). Organizational culture and leadership. A Dynamic view . Sanfrancisco: Jossey-Bass.Shepard, L. (2000). The role of assessment in a learning culture. Educational Research, 29 (7), 4-14.Sursock, A. (2011). Examining quality culture II: Processes and tools – participation, ownership and

bureaucracy. Brussels: EUA.Sursock, A. (2015). Trends 2015: Learning and teaching in European Universities. EUA publications

2015. Brussels: EUA.The Business Innovation and Skills Committee. (2016). The teaching excellence framework: Assessing

quality in higher education. Third report of session 2015-16. London: The Stationery Office Limited.

Thomas, L., Jones, R., & Ottoway, J. (2015). Effective practice in the design of directed independent learning opportunities. York: The Higher Education Academy.

Vettori, O. (2012). Examining quality culture part III: From self-reflection to enhancement. Brussels: EUA.

Williams, R. (1983). Keywords: A Wocabulary of culture and society. Oxford: Oxfor University Press.



71

ISSN 1822-7864

Endnotes

1 http://ec.europa.eu/eurosta2 National Reports regarding the Bologna Process implementationhttp://www.ehea.info/article-details.aspx?ArticleId=863 The regulations of the Cabinet of Ministers n 68. http://www.likumi.lv/ta/id/280100-

darbibas-programmas-izaugsme-un-nodarbinatiba-8-3-6-specifiska-atbalsta-merka-ieviest-izglitibas-kvalitates-monitoringa

4 Information from the website of the Ministry of Education and Science http://www.izm.gov.lv/lv/aktualitates/1804-latvijas-augstskolas-efektivizes-resursu-parvaldi

5 The Centre for Educational Growth, The internationalisation study at the Rīga Stradiņš University, 2014/2015 study year.

6 The Quality Culture Project 2002-2006, European University Association. http://www.eua.be/activities-services/projects/past-projects/quality-assurance-and-transaparency/quality-culture-project.aspx

7 www.rsu.lv

Received: November 11, 2016 Accepted: February 15, 2017

Tatjana Koķe Dr. habil. paed., Professor, Vice Rector for Education, Rīga Stradiņš Univer-sity, 16 Dzirciema Street, Rīga, LV-1007, Latvia. E-mail: [email protected] Website: http://www.rsu.lv/eng/

Nora Jansone-Ratinika Dr. paed., Assistant Professor, Director of the Centre for Educational Growth, Rīga Stradiņš University, 16 Dzirciema Street, Rīga, LV-1007, Latvia. E-mail: [email protected] Website: http://www.rsu.lv/eng/

Rudīte Koka Dr. paed., Assistant Professor, Rīga Stradiņš University, Department of Biol-ogy and Microbiology, 16 Dzirciema Street, Rīga, LV-1007, Latvia. E-mail: [email protected] Website: http://www.rsu.lv/eng/


Documents

PROBLEMS OF EDUCATION IN THE 21 st CENTURY Vol. 75, No. 1 ...oaji.net/pdf.html?n=2017/457-1488299643.pdf · Tatjana Koķe, Nora Jansone-Ratinika, Rudīte Koka Rīga Stradiņš University,