R e v i e w s

E V A L U A T I N G T H E EVALUATORS (Studies in Teaching and Learn ing) : Maurice Holt b Hodder and Stoughton, London (pp 187). 1981, ISBN 0 340 27245 7 £3 .45

To one like me who has invested more than two decades of his life time to projects in the field of evaluation, the title of the book was very st imulating and made me read it from cover to cover.

The first thing that came to my mind on reading the book was that a careful and systematic distinction deserved to be made in the t reatment og the book in regard to the concepts of pupil evaluation, curr iculum evaluation, project evaluation and school supervision. These are distinctly separate specialist areas and inspite of the overlapping do deserve separate treatment. The contents of the book left it only as wishful thinking. Fur thermore , the caution in calling some chapters 'notes ' is likely to stave off a great deal of criticism, which the contents of the book would have otherwise earned from experts in different fields of education.

One of the charges made against evaluation is that it is not value free. In fact it cannot and ought not be value free. It is this element of value judgement, which really distinguishes 'Evaluat ion ' f rom the so called objective 'Measurement ' . Evaluat ion has but to be a continuous process and the closeness and intimacy between evaluator and the evaluand is the basic foundat ion which enables it to perform the valuable function of diagnosis and subsequent remedial and enr ichment instruction. The purpose of pupil evaluation in this context is not just to measure achievement but to improve it. While an over-emphasis on evaluation (as an element of curr iculum) would be very undesirable, it would be equally undesirable to underestimate the role of evaluation as a powerful instrument for improving teaching and learning. Fur thermore , though it would be too much for any educationist to claim that evaluation possesses the potential to improve everything, it would at the same time be reasonable to accept that it can improve school programmes.

This the reviewer can testify f rom his personal experience. Way back in 1965 when the students appearing at the Higher Secondary Examinat ion in the Indian State of Rajasthan were put to an improved system of evaluation and were asked about its impact, they made a number of observations among which were that the improved system of examinations would make them read the whole course (and not just to make selective study); make them study throughout the school sessions (as compared to making eleventh hour preparat ions) ; make them thoroughly unders tand the concepts (as compared to memorising and reproducting them) and devalue the cheap examinat ion guides like %lathematics Without Tears' , 'English Within a

Week' and so on. These statements are enough to substantiate the potential of evaluation as an instrument of change.

The personal e lement is an impor tant ingredient of evaluation. In fact, a teacher could be justified in arguing that testing is as much her responsibility as it is her right and that testing her pupils is her personal and private business and that any external evaluation is an interference in her personal and private affairs. Any situations like these, where the teacher assesses the levels of achievement and degrees of proficiency of her own pupils, there are chances of disagreement between different assessors and also sometimes a need to make choices. Even so this cannot and shall not make evaluation a political achivity. I t will remain an academic activity f rom beginning to end.

It is also difficult to accept the proposition that the pr ime concern of the teacher is 'curr iculum change'. In fact her main role rests in curr iculum implementat ion. Despite this statement, it would be wrong to ignore that in an ideal situation, the teacher could also be an impor tant col laborator in curr iculum development, review and renewal. Others, besides teachers, who participate in this process are the educational philosophers, educational sociologists, educational economists, educational psychologists, educational administrators, the pedagogues and the educational evaluators. Each one has a distinct role to play in the collaborative venture that curriculum development and implementat ion is.

Again like teaching, testing in the cognitive, affective and psycho-motor domains has to assume different forms. The techniques, the tools and the procedures will all differ in different situations. It would be wrong to evaluate a psycho-motor skill with the same technique and tool as cognitive achievement in a certain area. Thus, if we have to test whether a person is a safe driver or not, we cannot just depend on his score on an achievement test on the rules of the road. We also need to know the extent to which he is able to apply them in actual practice. This will need evaluation based on observation, not on written answers.

The chapter entitled 'Self-evaluation in Schools' ought really to have been given the title 'Evaluat ion by Schools'.

As could be inferred from what has been already said a diagnosis of the disease is just not enough. To be of value, the diagnostic conclusions have to be supplemented by corrective measures. Such corrective measures are not possible without diagnosis and diagnosis is not possible without evaluation.

With these observations, it could be concluded that inspire of the differences with the view-points presented by the author, the reviewer has great respect for his courage in putting down some of his personal beliefs and experiences. Differences in

2 0 1

people working in different situations are natural. It is not possible for the reviewer to agree more with the author in his conclusion that real improvement would emanate from the development and implementation of a good curriculum. But he would not like to undermine the role of evaluation as an effective means of educational transformation.

H. S. Srivastava

Sample Unit Tests in Geography, H. S. Srivastava and K. Seshan, (National Council of Educational Research and Training, New Delhi, 1980), 172pp.

Sample Unit Tests in Geography consists of a set of twenty tests designed for formative evaluation purposes, assessing the curriculum of India's Classes IX and X. The tests were devised by the Examination Reform Unit of the National Council of Educational Research and Training at New Delhi. The tests reflect an attenuated Bloomian scheme of objectives, ranging ,from 'knowledge' to ~application', each major category broken down into sub-categories. These sub-categories will cause some confusion to those brought up in the Bloom idiom. This is not to suggest it should be slavishly followed. But it is necessary to adopt a logical subsumptive process. This does not seem present, for example, in an 'application' category, in which the sub-category descriptors include 'selects relevant facts', 'analyses', 'predicts', 'judges adequacy', 'proposes a plan', 'infers', 'establishes relationship', and so on. In addition to the cognitive objectives, the tests aim to develop geographical skills and assist in the development of 'positive attitudes towards peoples and environments'.

Each of the twenty sections contains a blueprint or specification, in the form of a matrix, in which the proportion of marks to be allotted to the sub- topics in the content, to the different abilities to be tested, and the assessment instrument to be used (here essays (very few), short answer questions, very short answer questions, and objective questions), is pre-specified. Most of the objective items are of the multiple-choice simple completion type. Each test includes a scoring key for the multiple choice items and a marking scheme for the rest, together with expected outline answers (usually very bare), estimated times needed for answers, and expected difficulty levels. No evidence is provided, however, of statistical analysis, to elicit the difficulty and discrimination levels of items. Questions of different types, while separately clustered, are not explicitly signalled, going against normal British practice, certainly in objective tests. Unfortunately, too many of the multiple choice items lend credence to what seems to me a generally misguided view that such items best test low level abilities and traditional content.

The actual questions are varied in quality. A number of the multiple-choice items are relatively sophisticated, genuinely assessing higher level abilities including, for example, simulations of industrial

location. A few short-answer questions aim to explore values and attitudes, but the "expected answers' are too closed and condensed for a guess to be hazarded as to whether they are likely to succeed in this purpose. There is a suggestion of lack of discrimination in matching the most appropriate forms of question to particular objectives, one of the high-level skills in this area.

A major problem is the nature of the content which is being tested, reflecting largely though not wholly a pretty traditional view of geography. The main content headings do not promise a lively approach: 'temperature'; 'wind systems'; 'rainfall ': 'climatic and natural regions'; 'soils'; 'mineral wealth': 'fuels'; and 'international trade', to take just a selection. A range of maps, graphs and sketch diagrams, as well as verbal material, provides the basis of the questions, but there are no photographs.

A positive feature of exercises of this type is that the planning is not only necessarily careful, but also is made explicit. The skills and the values embodied in the production, as well as its weaknesses, are exposed to the light of day, and laid open to evaluation. The Examination Reform Unit's work is clearly worthy of further development which, to bear full fruit, should be paralleled and linked with a Curriculum Reform Unit. A 'new curriculum' is mentioned in the Preface, but there appears little evidence of it here. Certainly the present unit's work seems in advance of the curriculum it is testing.

W. E. Marsden

Indicators of Education Systems James N. Johnstone. (1981) International Institute for Educational Planning. London: Kogan Page; Paris: UNESCO

In this long (279 pages plus appendices) and comprehensive book, James Johnstone offers what must now rank as a definitive work for those who believe that 'proper planning and researching of education systems must be based on sound quantiative methods' (p. 279). The book offers, in the author's words, 'an outline of some of the main concepts for defining, developing and using indicators of education systems . . . which politicians, planners and researchers might consider using in their examination of education systems'. In an initial chapter on 'The Concept of an.Indicator of an Education System'. Johnstone suggests that indicators have four principal uses: the statement and development of policies in a more definite and coherent way; the monitoring of change in systems; representing education system characteristics in research studies; and the facilitation of the formation of a valid and reliable classification of education systems. Subsequent chapters deal at a more or less sophisticated statistical level with the nature of education systems data, the theoretical and empirical identification of indicators for national and regional education systems, and how to present and use indicators in educational planning. research and classification.

202