Publish Perish

8/2/2019 Publish Perish

1/6

Abstract- Every year, the number of papers submitted to the

Power Engineering Society increases. This growth, which is in

part the result of the pressure to publish, is straining the

publication process, overloading the pool of reviewers, and

creating the impression that the quality of publications is

diminishing. This paper summarizes the informal discussions

which have taken place on the Power Globe Listserve and shows

the level of concern among members of the power engineering

community. The paper covers aspects of the peer review process

in PES, including present IEEE/PES publication policies, ethical

aspects of the review process, technologies for aiding the review

process, topics related to value of the technical content, graphics

and peer review, and more. The several contributions make

constructive suggestions which will hopefully improve the PES

peer review process and the quality of the publications.

Index Terms: Publications, peer review, technical papers, power

engineering society.

I. INTRODUCTION

The topic of the increasing number of papers (and theperception of decreasing quality) was discussed on thePowerGlobe Listserve. First, it was proposed that the "publishor perish" environment of the academic world tends to causemore papers to be submitted for publication. This tendency,plus the societal survivalist competition we all face haveencouraged a culture of a so called soft plagiarism. Weborrow from our own work, adding little new, and we mayeven build on the work of others without giving credit. Theseforms of plagiarism are not always easily detected due to thesophistication of the performers. These symptoms are addedto our natural tendency to seek relevance and prominence

among our peers and are usually disguised as the so called"search for originality." This type of originality we seek,encouraged by the university system, and which is regard as asign of wealth, is in reality a confession of immaturity.

One or two good paper resulting from a graduate degreeused to make any advisor very happy. Nowadays this smallnumber is unimaginable and students are forced to make any

P. F. Ribeiro is with Calvin College, Engineering Department, Grand

Rapids, MI 49546, A (e-mail: [email protected]).

un-reflected idea publishable and impressive, and thanks toadvanced word processing, produce papers by the bundles.

These developments are the background and source formuch of the rubbish we produce in the name of scholarshipand scientific / technical papers. This author does not excludehimself from contributing to this chaotic situation.

But the problem becomes even more complex anddisturbing when a soft form of plagiarism becomes part of theacademic / research culture, when individuals are searchingfor any new ideas from wherever source they can get themand consciously or un-consciously neglecting or notacknowledging the possible sources or information. Softplagiarists become academic alchemists who must turn (basemetal into gold) other people's idea into their own. The sadthing, however, is that the system is not sensible enough todetect it and apparent success is possible.

It is important to recognize that the number of paperspublished by an individual has very little relationship withtheir significance and contribution to the real academic,

research and professional world. We must avoid replacing thetrue search for originality by a soft form of plagiarism, and actmore consistently and ethically. A literature review [1-8]shows that the concern is well established. Recommendationsfor addressing different aspects have been made by severaltechnical societies.

II. THE ISSUES FROM DIFFERENT PERCEPTIONS

The initial post summarized in the Introduction generated anumber of excellent replies summarized below. (Due to spacelimitation only a fraction of the public replies aresummarized.)

A. The Academic Environment

The first debater affirmed the seriousness of the issue andstated that

The present situation is due to a number of factors.Proliferation of number of journals seeking papers to publishaggravates mediocrity and soft plagiarism. Insistence on thenumber of publications for getting promotion and researchgrants causes a rat race leading to the present situation.

There was a time when the title of Senior lecturer wascoveted and honored as a position as a full professor. In manyinstances there was only one professorship available. If

Publish or Perish: An Evaluation of theQuality, Quantity, Ethics and Review Process

of IEEE/PES Publications:

A Summary of the Power Globe Discussion

Paulo F. Ribeiro, Fellow, IEEE

2008 IEEE.


2/6

financial benefits are decoupled from the title, some remedy ispossible. Consideration for promotion and tenure is supposedto include factors such as teaching excellence, publications,funding, service, and collegiality.

But the sad fact is that funding and publications decide thefinal outcome in many instances. Academic communityexperienced violence and murder in some instances. Thesubject has been a blot on the academic community for the

last few decades and 'everyone talks about it but nothing isdone about it' [9]

B. A Historical Viewpoint

Another contributor reflected on the historical developmentsleading to the present reality:

. . . before we get too involved in complaining about it, letus look at how we got into this situation. I think there is atendency to view the more remote past of our profession as asort of golden age, and our intellectual ancestors asengineering superheroes: few in number, but capable of greataccomplishments. Was it really like that, I wonder, or are our

views clouded by the historical separation?Well, of course, there were some very great men backthen, a century and a half ago. Their names are known to allof us: they are used for many of the units we use. A goodmany such luminaries, from several countries, have left theirnames to posterity this way. With perhaps a hint of envy, weacknowledge that few of the papers that we see publishedtoday have the potential to be as important as the product ofthose giants on whose shoulders we stand.

Many of the people who made these early and greatcontributions were working in universities: Kelvin, forexample, was at Glasgow University. Maxwell was at KingsCollege, London. It wasnt that the scientists of the time were

well-educated, independently wealthy people working alone.The situation was, in fact, not enormously different than it istoday.

There are a couple of identifiable reasons for the changes.First, whether in academia or not, the researchers which werein natural philosophy (or physics) of 150 years ago faced aworld where very little was known and understood aboutelectricity.

Thus, one reason for the importance of the contributionsof the mid-nineteenth century was that, by definition, they laidthe groundwork for all that has followed. Second, thecompetition was not numerous. Because there were sorelatively few people involved, the papers seem special. But

then times began to change. In 1878, William Ayrton, whohad studied under Kelvin, returned to England from a spell inJapan, where he had established the first electricalengineering course in the world, at the Imperial College ofEngineering in Tokyo. Back in London, at the City and GuildsInstitute, he started to teach classes to electricians, or (as wewould call them), electrical engineers. Along with a colleague,W.C. Haycraft, he wrote a paper showing how electricalcalorimetry could accomplish in a matter of a few minuteswhat physicists had been occupied with for the previous fewyears, determining the value of what we nowadays call Joules

constant. The paper was A Students Simple Apparatus forDetermining the Mechanical Equivalent of Heat, read beforethe Royal Society in 1894. This was shocking stuff, and theworld of physics went ballistic. This paper marks the momentwhen electrical engineering split away from physics.

To make matters worse, Ayrton was teaching electricalengineering at night-school, because the students had to earn aliving during the day.

In other words, by the 1890's, the business of electricalengineering was getting democratized. As the generationshave passed since then, the number of electrical engineers hasincreased, and the amount of terra incognita remaining forexploration has diminished. Until we find ourselves in thepresent circumstance. . .

It is my contention that the proliferation of papers thatresults was inevitable -- we must learn to live with it. So whatdo we do about it now? Well, about all we can do is toconcentrate on the quality of the papers we publish. [10]

C. Are We Just Moaning?

A more optimistic view of the situation and was taken by

another commentator that observed:Ancient Romans used to say O tempora! O mores! whichis a moaning What bad times, what bad customs. Oldergeneration has always had a tendency to moan about thecurrent situation and hark back to good old times. While itis unquestionable that the "publish or perish pressure hascaused an inflation of journals and papers of sometimes notvery good quality, I also believe that current PhD studentswork much harder than my generation did. It used to be thecase that PhD was treated as a goal in itself and publicationswere not strictly required. We should applaud the currentgeneration of PhD students for striving to publish theirresearch, not deride them. After all, a definition of a PhD

work (at least my definition) is that the research must be of apublishable quality. If a PhD does not result in a journalpublication (or at least a couple of good conference papers),it's either because it was so poor that it should not have beenawarded in the first place, or because the PhD student was toolazy to do it (or more likely their supervisor too lazy to pressfor it). If a research is not published, who will know about itand therefore what good was it for?

So well done, current PhD students; work hard andpublish a lot!

As for making sure that journal papers are indeed trulyoriginal, it's the task for journal editors and reviewers.Obviously, the standards must be maintained. And I believe

that the standards are maintained, at least in the top 2-3journals. Of course there seems to be a proliferation ofsecond-rate journals but I don't have time to read them so Ican't comment. [11]

D. The Review Process and the Metrics

Others looked into the dynamics of the review process andcommented:

. . . that the reviewers are to blame, but so too are thesocieties for not using technology to find papers of significantsimilarity for special review.


3/6

All reviews are accomplished by a small group of people,those willing to review. As industry leaves the process, as it istoo busy keeping a job, it will get worse.

Since no one wishes to pay for universities, especially theUS, we are in for hard times. Hard times lead to insidergroups, protectionism, and inexplicit favors in exchange. Weall know what we will do to survive, if left to that low level ofachievement.

Given the exponential growth of information, I wouldsuggest that the present paper publication process, based onthe paper technology of the last century, should be replaced.[12]Another contributor said:

. . . Who has the time anymore to spend a half a day ormore per paper on a review? Thats what it takes to do aquality review.

One way out for faculty is to ask graduate students to dothem. Its a great educational opportunity if done right butagain, it takes time.

And time is in increasingly short supply. As a corollaryand in my opinion, many people who review papers are not as

competent or as conscientious as in the past and this is notalways because of time constraints. If you are educated in asystem where sloppy writing practices and poor understandingof engineering design practices (e.g., poor use of English, useof simulation as a substitute for mathematical proof, etc.) aretolerated, thats probably what you will bring to the tablewhen reviewing the works of others. Editors have to takeresponsibility for selecting good reviewers not just the oneswho will take on the job. And the educational system has totake responsibility for training good reviewers.

But thats not my main concern. There is another problemthat engineers in academia have. That is, engineers inacademia are judged against metrics developed to judge

scientists, not engineers. In his IEEE Proceedings article (6)William Wulf, President of the National Academy ofEngineering said about engineers in academia,

Unlike engineering, every other creative field on campus

expects their faculty to practice/perform. Even if you do not

buy the argument that engineering is creative in the same way

as art or music, you can look to the professions of medicine

and law, which also expect their faculties to practice the field

they teach. Can you imagine a medical school where the

faculty was prohibited from practicing medicine? Yet, this not

so in engineering. Engineering faculty are, for the most part,

judged by the same criteria as science faculty and the

practice of engineering is not one of those criteria. The

faculty reward system recognizes teaching, research, andservice, but not delivering a marketable product or process,

or designing an enduring piece of the nations infrastructure.

The criteria for promotion and tenure make it hard to hire

and reward people with such experience, even though it

would be valuable for students. In many cases, even taking a

sabbatical in industry may be risky for faculty because it does

not contribute to the usual resume-building activities of an

academic.

I agree. Perhaps changing the performance metric forpromotion in academia to something relevant to what

engineers do (or should be doing) would solve part of thepublishing problem for engineering. As Charlie Gross [14]and others have said, faculty are smart people. Give themrules and they will play to win unfortunately sometimesethics get stretched in the process. [13]

E. The Pressure versus the Ethics

The financial and professional pressure and the consequentstretching the ethics are the more difficult issues to resolve asone commentator noticed:

I think the prime mover behind much of the pressure topublish may be money. At [some schools], to be a successfulengineering professor, the target number at present is $300kin annual extramural funding. Untenured professors who failto meet this goal are in real danger of being terminated whichcan end their academic career.

Young professors entering the field for the most part, arebrilliant, hard-working, ethical, and dedicated. But they arealso realists and survivors. You define the game and they willfigure out how to win at it. Acquiring funding requiresobtaining contracts. Getting contracts requires publications.

Unfortunately, quantity as well as quality counts. Faced withpublish or perish alternatives, they respond rationally. Theypublish whatever and wherever they can. Can we really blamethem? After all, they didn't create the system; they're justtrying to survive in it. It has been said that 95% of the IEEETransactions could be obliterated, and the profession wouldn'tbe affected at all. It's just that we're not sure which 95%.Perhaps the criticism is unfair; on the other hand, maybe theestimate is too low. We've all been disappointed to discover,that having dissected a journal paper, that there wasessentially nothing there. On a positive note, virtually all ofour colleagues individuals I've had the privilege of reallygetting know have been hard-working, ethical, dedicated, and

trying to make the system better.The quality of journal publications can only be improvedwhen we find some way to reduce the extreme pressure onfaculty and graduate students to publish volumes of papers.Until then, given the ever-increasing number of graduatestudents worldwide, the ever-increasing expectations forpromotion and tenure, an ever-increasing number of papersare inevitable. [14]

F. Responsibilities and Recommendations

Another commentator talked about responsibilities, listedcommon problems and made a number of detailedsuggestions:

The quality of our technical journals is the responsibilityof the editors who have the last word in what gets published.They have a tough job though because in addition tomaintaining the technical integrity of their journals, they mustalso be sensitive to the desires and needs of the technicalcommunity. If for example, the technical community isn'tclear on what it considers to be a significant contribution, thenthe editors won't have a clear mandate for setting the bar foracceptance.

As a reviewer, I am frequently disappointed by the papersI am asked to review. Here are common problems:


4/6

1. The paper is incomplete and a knowledgeable readersimply cannot use or reproduce the results

2. The paper is simply a report on an experiment or acalculation which was of interest to the author but isof no particular interest to anyone else

3. The paper is a rehash of existing, readily availableknowledge

4. The paper is technically unsound5.

The paper is poorly written to the point that a readercannot understand it

In addition to being reasonably well written, here areexamples of things that I like to see in a paper:

1. It presents a novel approach to a new or existingproblem

2. It presents experimental results that can be of use toothers in the technical community

3. It presents a new experimental or measurementtechnique

4. It presents a new application of existing technology5. It presents new theoretical approaches or extends

existing ones

6. It presents a totally new technologyI would like to suggest that all of us can contribute toimproving the quality of journals. Perhaps guidelines such asthe following might be helpful:

Senior faculty, managers, etc:I. Don't encourage publication simply for the

sake of numbers.II. Evaluate publications on their technical merit.

Certainly, the quality of publications reflectsnot only on the authors but also on theinstitutions which they represent.

Authors:I. Don't submit a paper for publication unless

you have something new to say that will beof significant interest to others in ourtechnical community.

II. Don't attempt to publish work that isincomplete.

III. Make sure that what you do write is wellwritten and clearly presented so that yourreader will be able to fully follow what youhave to say.

Editors:I. Raise the bar!

II. Clearly define the standards for acceptance ofpapers in your publication and then enforce

them. This may require some interaction withreviewers to help them understand thecriterion.

Reviewers:I. Make sure that papers meet the minimum

criterion for publication before you acceptthem.

II. Ask yourself if there are readers who will beinterested in and benefit from the paper inquestion and whether there the technicalcommunity will benefit if the paper is

published and preserved for readers yet tocome.

III. Although I don't agree that there is any pointin including reviewers' names at the time ofpublication, it would probably be a good ideaif each reviewer asked him/her self if theywould tell their colleagues that theyrecommended a specific paper for publication

and that they really should read it as soon as itis published.Readers:

I. Let the editors know if you are concerned thatthe quality of publications in any given

journal is below your expectations.II. Participate in the process by volunteering to

review papers. [15]

G. The Transcendence of the Issue

Finally, another commentator shows that the problemtranscends the academic world and the need to be selectivewhen using technical publications:

As a non-academic working engineer with 45 yearsexperience in the electricity industry I would like to heartilyendorse [15] remarks, in particular his checklist of things tolook for in a paper. In the course of my daily work I am onthe continual lookout for ways of doing things better . . . andseize upon any new formula or methodology and create aspreadsheet model from it that will inform and improve ourwork practices.

Over the last week for example I have downloaded 589papers from the recent CIRED conference in Vienna, some ofwhich are rubbish but others of which are real gems and Ihave circulated as widely as I can. Without the benefit ofyears of practical experience in the industry though it must be

difficult for researchers and perhaps even reviewers to knowwhat will be useful and what will be not.I must say I really appreciate this plethora of papers that

creates a vibrant marketplace of ideas for me to shop in butthere must be a downside in the wasted effort creating stuffthat is condemned to sit on the shelf forever gathering dust.We used to be told at university that we would only use 10%of what we were taught; but no-one knew which 10%.Currently I think I'm batting in the low twenties and considermyself exceptionally privileged. [16]

III. CONCLUSIONS

This paper has summarized the informal discussions which

took place on the power globe listserv during the month ofSeptember 2007. The debaters, in response to an initialposting, made many constructive comments and suggestionswhich will hopefully help the IEEE Power EngineeringSociety to cope with the increasing quantity of papers as itadjusts the review process and to maintain the high qualityand ethics required for the proper functioning of the powerengineering community. A panel session was organized forthe PES General Meeting in Pittsburgh for July 2008(Appendix 1).


5/6

As a final observation the author would like to refer to aprevious posting on Power Globe (and transcribed in theAppendix 2) which encourages reviewers to be firm, butcareful and civilized with their words.

IV. APPENDIX 1ADVICE TO REVIEWERS (*)

1 Avoid unqualified statements. It is not useful writingmerely that the paper is of low quality or irrelevant or a

case of bad engineering. Be specific and tell the failures andlet the author(s) see by themselves where they should work toimprove, correct or even accept the rejection.2 Avoid the use of offending words. The vocabulary ofendearment, complaint, and abuse, provides almost the onlyspecimens of words that are purely emotional, words fromwhich all imaginative or conceptual content has vanished, sothat they have no function at all but to express or stimulateemotion, or both.3 Be on guard when criticizing. If we honestly believe apaper to be very bad and we can hardly help but express ourextreme dislike, remember that the function of the reviewer isto get out of the way and let logic speak; not to discharge

hatred, but to expose the grounds for it; not to vilify faults butto diagnose and exhibit them. Unfortunately, to express ourhatred and to revenge is easier. Hence there is a tendency toselect pejorative epithets with a view not to their accuracy butto their power of hurting. The best protection against this is toremind ourselves again and again what the proper function ofpejorative words is. The ultimate, simplest and most abstract,is the word bad itself. The only good purpose for everdeparting from that monosyllable when we condemn anythingis to be more specific, to answer the question, Bad in whatway? Pejorative words are rightly used only when they dothis.4 Finally, be fair and kind. We must get it firmly fixed in

our minds that the very occasions on which we should mostlike to write a slashing paper review are precisely those onwhich we had much better hold our tongues. The very desireis a danger signal. When a colleague whom we admire ingeneral, writing about a topic we know, producesdisappointing work, we may proceed with tolerable safety. Weknow what we had hoped for. We see, and would haverelished, what he was trying to do. By that light we maypossibly diagnose where the paper has gone wrong and makeconstructive suggestions. But when an author we do not likeor whose knowledge of the subject we question is attempting(unsuccessfully - or worse still, successfully, according to theeditorial rules) exactly the sort of thing we know will not bevery useful (beautiful and consistent analysis, but withoutmuch physical or practical content) then, if we are wise, weshall be very careful with our words. The strength of ourdislike is itself a probable symptom that all is not well within;that the raw place in our psychology has been touched, or elsethat some personal or partisan motive may be secretly atwork. If we were simply exercising judgment we should becalmer; less anxious to speak. And if we do speak, we shallalmost certainly make fools of ourselves.(*) Adapted from CS Lewis's writings on literary criticism[17].

V. APPENDIX 2PANEL SESSION LINE UP FOR THE PESGM08

Publish or Perish: An Evaluation of the Quality, Quantity,Ethics and Review Process of IEEE/PES PublicationsAbstract: Every year, the number of papers submitted to thePower Engineering Society increases. This growth, largely theresult of pressure to publish, is straining the publicationprocess, overloading the pool of reviewers, and creating theimpression that the quality of papers is diminishing. This

panel plans to discuss many aspects of the peer reviewprocess in PES, including present IEEE/PES publicationpolicies, ethical aspects of the review process, technologiesfor aiding the review process, topics related to value of thetechnical content, graphics and peer review, and more.Throughout, suggestions will be made for improving the PESpeer review process.1 A Short Summary of the PowerGlobe Discussion: Paulo

Ribeiro2 Publication Policies: Atef Morched3 Dealing Ethically with the Publish or Perish Pressure:

Charles Gross4 Advanced Technology for Assisting the Review Process:

Thomas Baldwin5 Peer Review and Graphics: Harold Kirkham6 Questioning the Metrics for Performance Evaluation:

Mariesa Crow

VI. REFERENCES

[1] Dimitris Kalles, Improving Professional Conduct in Publishing, IEEE

Computer Society, October 2005 (Vol. 38, No. 10) pp. 116, 114-115.

[2] Bruce Bower, Peer Review Under Fire, Science News, June 22, 1991, Vol

139 pp 394-395.

[3] Jon Turney, End of the peer show: Who decides who should do what

research? Most scientists agree that peer review is a deeply flawed method.

But other solutions might be worse From New Scientist Print Edition, 22

September 1990.

[5] James Hendler, Avoiding Rejection, A Letter from the Editor, IEEEIntelligent Systems, IEEE Computer Society, Sept.-Oct. 2005, Volume: 20,

Issue: 5, pp. 2- 4

[6] William A. Wulf, How Shall We Satisfy the Long-Term Educational Needs

of Engineers?, Proceedings of the IEEE, Volume 88, Number 4, April 2000,

pp. 593-596.

[7] Markel, M.; Sanders, S.P.; Turning a conference paper into a Transactions

article, Professional Communication Conference, 1994. IPCC '94

Proceedings. 'Scaling New Heights in Technical Communication'.,

International, 28 September.-1 October. 1994 pp. 426 - 429

[8] Kashihara, Akihiro; Kamoshita, Yasuhiro; From Knowledge Publishing to

Peer Review, Advanced Learning Technologies, 2007. ICALT 2007.

Seventh IEEE International Conference on 18-20 July 2007 Page(s):459 -

463

[9] Arun Sekar, Power Globe Posting, Sept. 2007.

[10] Harold Kirkham, Power Globe Posting, Sept. 2007.

[11] Janusz Bialek, Power Globe Posting, Sept. 2007.[12] Gerald Sheble, Power Globe Posting, Sept. 2007.

[13] Robert Thomas, Power Globe Posting, Sept. 2007.

[14] Charles Gross, Power Globe Posting, Sept. 2007.

[15] Steve Umans, Power Globe Posting, Sept. 2007.

[16] Leith Elder, Power Globe Posting, Sept. 2007.

[17] CS Lewis, Studies in Words, Cambridge University, 1960.

VII. BIOGRAPHY

Paulo F. Ribeiro (M1978, SM1988, F2003) received a BS in Electrical

Engineering from the Federal University of Pernambuco, Brazil, completed the

Electric Power Systems Engineering Course with Power Technologies, Inc.

(PTI), and received his Ph.D. from the University of Manchester, Manchester


6/6

England. Presently, he is a Professor of Electrical Engineering at Calvin College,

Grand Rapids, Michigan. Dr. Ribeiro is a Fellow of the IEEE.

Documents

Publish Perish