An Accountability Information SystemAuthor(s): Richard M. DurstineSource: The Phi Delta Kappan, Vol. 52, No. 4, Accountability (Dec., 1970), pp. 236-239Published by: Phi Delta Kappa InternationalStable URL: http://www.jstor.org/stable/20424479 .

Accessed: 28/06/2014 15:42

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Phi Delta Kappa International is collaborating with JSTOR to digitize, preserve and extend access to The PhiDelta Kappan.

http://www.jstor.org

This content downloaded from 91.238.114.151 on Sat, 28 Jun 2014 15:42:14 PMAll use subject to JSTOR Terms and Conditions

Lovely Old Boxes Are For Lovely Old Times

By Sarah Cirese

Once upon a time an organism (it doesn't much matter

whether you see it as a baby or a bug) lived in a lovely old box. It was fed a diverse menu ranging from steak to garbage, but primari ly peanut butter. It gained both because of and in spite of its diet. Over the past decade it grew to such dimensions and proportions that it found the box confining. It

was a long time before those who paid the grocery bill began to no tice this growth. In 1964 the organ ism had mumbled "Out!" Very few

heard what it said in their concem for silencing it. Someone studied the situation. It continued to be fed and to grow but it began to have an occasional upset tummy. Now and then it poked an appendage out tentatively but received no real help. A tiny corner of the box was enlarged but the shape and size were not significantly changed. Eventually an arm burst out and, not by accident, hit a window with a shattering blow.

"What is this monster?" "Who sired it?" "What do we do with it?"

"Shove it back in." "Lop its arms off." "Deplore it." "Starve it."

Recently the organism was hand ed a piece of grit it just could not digest. Its reaction has not been to vomit, to have a tantrum, or to explode. Instead it came up with a rational and logical suggestion to

change the shape of its box - to make the box fit the needs of the organism it houses and to open up the sides to promote interchange

with its environment. "But it won't be a box any

more." Whatever the shape, those who

favor this change hold that it will be a far more appropriate home than the old box. Lovely old boxes are for lovely old times.

-Reprinted with permission from The Research Reporter

of the Center for Research and Development in Higher

Education, University of California, Berkeley.

an accountability

in format on systemr

q by richard m. durstine

T he body of this paper is in four parts. First are some comments on background and viewpoint,

to further define the role of careful handling of information when dealing with accountability measures. Second, a structure for dealing with accounta bility information is outlined. This outline is brief and tentative in order not to be tied to a specific situation. Third, the benefits and requirements that might be associated with a system,, for accountability information are re viewed. And fourth, a proposal (again tentative) is made for the beginning of such an information system, centering about the creation and early activities of an "Office of Accountability Infor

mation." In a final brief section, the major points made in this paper are brought together in summary.

If the idea of accountability is to be used in planning, negotiation, and de cision, the framework for its use must be designed with care. In the present discussion, the process of dealing with accountability is viewed as requiring four steps: 1) measurement of inputs to the educational process (e.g., per sonnel, equipment, facilities, funds); 2) measurement of direct effects (e.g., type, frequency, and intensity of edu cational activities), especially in rela

RICHARD M. DURSTINE is lectuer er on education, Harvard University.

tion to inputs; 3) measurement of long-term and collective effects (e.g., on the student's future progress in school and on his success in life, or on groups of students, or on schools as a whole); 4) synthesis of the above into an informational base for decision

making. The above four steps taken together

form a big job, even an impossible one.

But it is within a framework such as this that any attempt to deal with accountability must be viewed. Any real activities will, of course, fifll only part of the whole need. But recogni tion of the partial nature of these steps can help us to see better which needs they fill and which they do not.

Also clear is the need for very careful design, development, testing, and evaluation - over a considerable period of time - of any system of

accountability measures if it is to be useful in the service of education and

not just a bit of administrative window

dressing. This paper is not directly about

measures of accountability, but rather about putting them to work in a useful

way. Attention here will be on collect ing, keeping, processing, and using information about accountability mea

sures. This leaves the choice of mea

sures to others and concentrates on

the activities needed to monitor and to

apply them. Thus this paper relates to

236 PHI DELTA KAPPAN

This content downloaded from 91.238.114.151 on Sat, 28 Jun 2014 15:42:14 PMAll use subject to JSTOR Terms and Conditions

only a part of the job, albeit a part that is essential if those measures are to serve their intended purpose.

Politics and technology. It is impor tant to remember that objective ac countability measures are tools toward the end of more appropriate operation of the educational system. The basic atmosphere in which they will be used is a subjective and political one. Its realities must at all times govern the application of any accountability system that may be developed. In this paper, the development of means to deal with information about accounta bility measures is treated as a technical task, in the service of those who must attend to both the political and the technical issues involved. The sugges tions made here are advanced only insofar as they can be useful to real operations.

The need for goodwill. Any organi zation that is serious about the devel opment and use of accountability mea sures must recognize that the job will take time and patience, will be some

what tricky if the results are to be both technically and realistically sound, and will require a large reser voir of good intentions on the part of its developers and users. If those mini

mum conditions cannot be met, there can be no more than superficial appli cation of these measures.

Where to begin. Finally, it is impor tant to begin with activities that have

high likelihood of success. The recom mendations that follow are meant to contain more than enough activities for a start. What is done first should be chosen to contribute best to prompt and continuing success of accountabili ty measures in the improvement of education.

A Structure for Dealing with Accountability Information

At this point, a brief look at the term "objective measures of accounta bility" seems appropriate.

Objective. I take objective to mean

explicitly stated and measurable in a

way that can be defined independently of the person making the observation. If two persons make the same "objec tive" measurement of the same thing, using the same rules of measurement, they should get (more or less) the

same result. The importance of having objective measures is that they will be similar enough for comparison among them to make sense. The measures themselves must be designed to be free of (or at least to account for) the feelings and opinions of the observer

who makes the measurement. This does not necessarily exclude feelings and opinions from the definition of the measure, only from individual in stances of its use. The choice and definition of measures can be full of feelings and of values. The purpose of "objectivity" is to put the results of

measurement on an explicit and equal basis.

Measures of accountability. In the following pages, measures of accounta bility are conceptualized as follows:

When something passes from one party to another, there may be a need to

measure the amount that passes. The parties may, for example, be individu als or organizations (schools, unions, or boards of education); the thing that passes may be money, experience, a person's time. Often this transfer takes place in both directions, as in the exchange of work for money. "Mea sures of Accountability" will refer here to statements about this transfer from party to party. There are two reasons for keeping track of such statements: 1) so that the amount delivered can be compared with that agreed upon, and 2) so that the amount delivered can be compared

with that received by other recipients of the same thing.

With regard to measures of account ability for a public school system, a

variety of parties are involved. They include, among others, children, teach ers, parents, unions, schools, and the board of education. The things passing from party to party are also many. To create measures of accountability that

work in such a situation requires that both the measures and the framework for dealing with them be very carefully designed.

This paper does not deal with the

measures themselves, but with means of keeping track of them and with

converting them to a form that can be used for planning, negotiation, and management. These means must be workable in application to a variety of measures. And they must be flexible in their ability to make those measures

(and information derived from them) available to all who are concerned with the operation of the educational sys tem. To that end, this paper recom

mends design and creation of a system for accountability information.

Categories for Classification

Though it cannot be known at the outset just what measures any specific system for accountability information

will need to handle, the design of such a system causes us to anticipate certain features of the measures. This is done here in terms of: 1) the categories or classes of things to which the measures

must relate; and 2) the actions that

will be taken with the measures, once they have been collected.

Categories for keeping track of in

formation about accountability mea sures will be considered first. Though detailed investigation of this point is

needed, the categories might be ex

pected to fall along the following dimensions:

1. Administrative component of the educational system - e.g., individ ual student, teacher (or other person nel), department, program, school, or district.

2. Level and type of education offered - e.g., primary, secondary, technical, or special.

3. Type of people served - e.g., by geographical, racial, or economic dis tinction.

4. Type of service - e.g., normal program, enrichment, or guidance.

5. Measure of accountability being used - e.g., money, time spent, kind of personnel, kind of child reached, intensity of service, or test scores.

6. Time - to which the measure relates.

The above list is suggestive only, and can be no more than that at this time. Categories such as these require careful definition to be flexible with out being cumbersome, since they set

the framework within which the sys tem for accountability information

will operate. It is important that, as far as possible, this framework be made up of parts that are comparable to one another across the whole system, so that measures can be collected, com bined, and related to one another as needed. Unless information is collect

December, 1970 237

This content downloaded from 91.238.114.151 on Sat, 28 Jun 2014 15:42:14 PMAll use subject to JSTOR Terms and Conditions

ed with this capacity, it will be hard to use it in ways that are not expressly provided for beforehand.

Uses for Accountability Measures

Whatever measures are collected, they will be treated in two steps. First is the immediate, mainly routine and

mechanical, refinement from raw data to a form in which they give some indication of how the educational system or its parts are performing. Second is use of these refined mea sures to evaluate performance and to

make provisions for its improvement. Both kinds of treatment are included in the following list. Like the earlier list of categories, this one is suggestive rather than complete.

1. Initial gathering of the measure of accountability - the original collec tion of raw information.

2. Aggregation into totals and sub totals - according to the dimensions of the earlier list; giving, for example, level of the measure by school, by program, by district, by type of school, or by population served. Such aggregation will be needed in a variety of forms if the accountability mea sures are to receive full use. If they are properly collected and labeled in the first place, this will be a straightfor

ward (though sometimes lengthy) job. 3. Comparisons among the mea

sures for various parts of the system -

for instance, among schools, among districts, or among programs. This will give a first indication of the distribu tion of the benefits of education,

which is useful to assure equality and to focus the activities needed to modi fy inequalities.

4. Projections into the future -

estimates of how the measures will

change and the conditions on which this change will depend.

5. Allocation of resources among activities, subject to limitations on total effort. Given the cost of provid ing various levels of service, the extent of that service and its preferred distri bution among schools, districts, and levels can be determined.

6. Management control -the vari ous aspects of keeping the operations of the system in balance, in conform ance with plans, to maintain services as decided upon.

7. Policy-level decisions - includ

ing negotiations, planning, and evalua tion of performance generally.

Many of the items listed require extensive attention, sometimes well beyond the mere application of ac countability measures.

Measuring the Performance of a System

In this section the standards to be met by a system for accountability information will be indicated briefly, including the expected benefits from use of such a system and some of the requirements that should be kept in

mind during its design. Benefits. The primary aim of these

measures may vary. In urban districts, it may be the alleviation of education al retardation. This should be the priority issue in that context when deciding upon and using whatever

measures are created. In other cases, the motivating issue may be different; any such difference will affect the design of the accountability measures.

Another potential operational bene fit from such an information system is its availability as a source of informa tion for other efforts aimed at plan ning and management of the educa tional system. These roles should be regarded as secondary, however, to the job of measuring, balancing, and im proving the delivery of educational service.

Requirements. Finally, there are requirements that any such informa tion system should meet. These are the

"accountability measures" for its own

performance. It is desirable that: 1) withdrawal of information be easy and

inexpensive; 2) presentation of infor

mation be in a form that is clear and easy to understand and interpret; 3) the information delivered be of the kind and detail needed, whether at the individual, the school, the program, the department, the district, or the system level; 4) the information be broadly accessible to those who can use it productively; 5) users of the information be encouraged to under stand its nature and its limitations, so that they expect neither more nor less than it offers; 6) there be compatabili ty from level to level of the form and content of the information, so that comparisons within and among levels are facilitated; and 7) ongoing revision of the information system be provided for as essential to its effective per formance and growth.

Proposal for Development of a System

The general requirements and activ ities outlined above for a system for accountability information call for careful and lengthy development of that system. Some specific steps in that direction are suggested below.

Office of Accountability Informa tion. One approach to implementation is to create an "Office of Accountabili ty Information," to be responsible for storing and processing any measures of accountability that are developed and applied to the schools, their personnel, and their clients. This office would in

particular be charged with keeping, analyzing, reviewing, projecting, and reporting various accountability mea sures and information derived from them. It might further play a role in

New York educationi leaders at accountability conference. Photo: courtesy of UFT.

238 PHI DELTA KAPPAN

This content downloaded from 91.238.114.151 on Sat, 28 Jun 2014 15:42:14 PMAll use subject to JSTOR Terms and Conditions

the testing and verification of these measures, including their reliability, acceptability to their users, their cost, and their value as a basis for decision.

It is left open at this point whether such an office would have a direct hand in the choice and development of the individual measures of accountabil ity, or in the first-hand collection of information. That remains outside the present discussion, which is addressed to the role of such an organization as a processor and distributor of this infor

mation after it has been collected. Some suggested activities. To begin

fulfilling the total task, activities such as the following might be undertaken.

1. Conduct a preliminary study of

existing information on accountabili ty, and of other background materials.

There is likely to be a considerable "inheritance" of structure and infor mation from other earlier activities that, if not directly addressed to ac countability measures, are at least re lated to their use. Design of the proposed system might well be linked with this inheritance, so as to be compatible with it.

2. Select a pilot project in the use of accountability measures, such proj ect to incorporate the most common characteristics expected to be encoun tered in full operation of the eventual system for accountability information.

3. Develop a structural scheme for classifying accountability measures and other measures based on them.

4. Choose and agree on initial mea sures to use in connection with the pilot project.

5. Gather and report upon these initial measures for the pilot project, including:

a) collection of the information; b) aggregation, comparison, and

other forms of processing; c) presentation of this pilot infor

mation to those who will use it; d) its trial use (at least in simulated

form) in evaluation of the educa tional process.

6. Revise the design of the informa tion system on the basis of the above

experience, until a reasonably accept able level of performance is achieved.

7. Make a report on progress to those responsible for the full range of activities in development of accounta bility measures.

8. Establish on a regular basis an

Office of Accountability Information, with its form and responsibilities based on experience with the pilot system.

9. Carry out a final test, followed by subsequent regular operation of this office.

10. Continue with ongoing evalua tion, revision, and development.

It is not feasible to go into more detail without considerable additional knowledge about the situation, the available resources, and the limitations on action that hold in the specific situations to which the measures will be applied. It is, for example, impossi ble to say much at this point about the schedule for, or cost of, developing such a system or office. Clearer defini tion of these issues must come from a slow, careful start, to be developed into a larger effort if success seems likely. In any event, this will not be a quick or inexpensive job, nor one that will be fully realized in only one or two steps. What is suggested here is a

way to start, nothing more.

Evaluation

Implicit and important here is the notion that the accountability system itself must be tested and evaluated; i.e., that it must itself be subject to some measures of accountability.

The first step, therefore, should in most cases be the appointment of a pilot team for treatment of accounta bility information, with appropriate staff, resources, and consultation. This team would be charged with the first development, testing, and evaluation of a system for accountability infor

mation, and would make recommenda tions toward its establishment on an operational basis.

Whatever is the exact form eventu ally chosen for organization and devel opment of measures of accountability and their use, the following general requirements seem minimal:

1. Access to relevant information by the pilot team should be assured; specifically, there must be official commitment to and support of its

activities. 2. Adequate staff and facilities

must be provided to give a fair chance to the creation of objective measures of accountability and to their aggrega tion into a usable tool for decision.

In short, if those responsible for

operation of the school system are serious about their interest in account ability measures, and if they are pre pared to put some substantial support and effort into them, there is a chance of success. But it will not be an easy or uncomplicated job. If it is not to be given a fair trial, it ought not be attempted at all.

Summary of Major Points

Let me now summarize the major suggestions and assertions of this pa per. Briefly, I have stated that:

1. A working system of accounta bility measures should be supported by a system for accountability infor

mation, embodied in an Office of Accountability Information.

2. The first step in this direction should be establishment of a pilot team to undertake developmental ac tivities in this direction.

3. The system created through this pilot activity should be subjected to thorough and continuing development, testing, and evaluation.

4. The first activities chosen for the proposed system for accountabili ty information should be aimed at areas where prompt (if limited) useful ness to the improvement of education al operations is likely; a more com

plete set of activities for the system should be developed from this base.

5. Any information system thus developed should be oriented toward service to its users; its operations and products therefore should be readily accessible and clearly defined.

6. The sum total of the activities suggested here is extensive. Adequate time, personnel, and other support are essential, especially in the exploratory pilot stages.

7. Though the job will be complex, a massive information network is not

contemplated. Nor is the extensive use of data processing, though some will be necessary. A small, but vigorous and effective, beginning is to be

sought. 8. The time, personnel, and cost

needed to carry out this proposal cannot be accurately stated in a gener al review like this one, but must await a full and careful study, once these suggestions are accepted in principle.

U

December, 1970 239

This content downloaded from 91.238.114.151 on Sat, 28 Jun 2014 15:42:14 PMAll use subject to JSTOR Terms and Conditions