1

IS8004 – Seminar 12

Presenting and Reviewing Qualitative Research

2

Presenting and Reviewing

These also may seem to be quite different topics, but actually they are closely linked

In the previous class, we looked at planning and writing Getting these right is critical to good

research Presenting relates to how you

present the research at a conference Reviewing is how others evaluate

what you present – written or spoken

3

Presenting

Presentation of a written paper means that you have to ensure it is well written, well argued, well justified and pleasant to read

You have to write for the audience – and meet their expectations (more or less)

You have to use language that they understand – not too much jargon

If you present well, then reviewers can at least understand your message

4

Presenting at Conferences

This is both harder and easier You have much less time – typically

10-20 minutes; perhaps an hour in a seminar.

So you have to miss out much of the detailed content – and focus on what’s really interesting – for the audience

You also have to answer questions – which may be quite difficult

5

Presenting 1

Although much work went into the planning of the study, you may only assign 1 slide to this!

You may cite a few key literature sources, and mention the methods

What should excite the audience is the results – especially if they are novel or radical in some way

This can be usefully discsussed at more length

Interesting future research opportunities can be discussed

6

Presenting Layout (10 mins)

1. Title, authors, affiliations 2. Introduction and Background 3. Literature 5. Method 6. Discussion of results 9. Future research No conclusion or references. They

are in the full paper.

7

Dealing with Questions If you know the answer, that’s great.

But often you don’t. So, either you make something up (risky).

Or you deflect the question, like a good politician!

You could say something that is related. Or, you say “that’s a great question” and ask

the questioner what s/he thinks! Or ask for input from the rest of the audience

Usually there is at least one person with a *lot* to say who will be keen to help out!

It is unlikely that you are the first to be in this state – you can learn from others’ prior experiences.

8

Dealing with Discussants

A discussant is a person who has the task of reading and commenting on all 2, 3, 4 papers in a session at a conference and saying something intelligent about all of them.

It is not an easy task – and discussants sometimes try to present their own research

They also ask you questions. Instead of replying directly, you can

try to develop a “conversation” with the discussant

9

Time

Conference time is limited, so you must not overrun your time allocation

If you do, you may be cut off with many slides not yet presented

The audience then focuses on the next paper and forgets you.

You must budget for time very carefully.

I suggest a maximum of 1 slide per minute

10

Presenting Qualitative Research

Qualitative research is more about words and observations, so you need to use that kind of evidence in the presentation

It is nice to cite what people said as evidence to support your findings

But make the cites short and precise 2-3 lines at most, ideally less.

You could include a diagram, photograph or other media to make a point

Examples from your own observations are nice You can easily expand on them if the audience is

interested They can be the entry to a conversation – with audience

or discussant

11

Reviewing

After you submit, then you have to wait Some reviewers are fast, others slow Most reviewers are critical and some are

really nasty Editors have the job of managing the

review process You can’t appeal to a reviewer (usually

blind) but you can appeal to an editor You are an author – and will also be a

reviewer You see both sides of the situation When you are the reviewer, try to be

constructive and helpful

12

Reviewing & Reviewers

How do reviewers think and work? Evaluation criteria Rejection criteria

Reviewing is common. What are the advantages and

disadvantages? Is there potential for bias or

favouritism? Developing competence in

responding to reviews So as to increase your chance of being

accepted

13

Attributes of Good Reviewers

Competent and Constructive Reasonable, Unbiased and Open

Minded Ethical in their behaviour

Conflicts of interest Respect for your creativity and

copyright Not too critical, nor too lenient Persuasive in their

arguments/comments Diligent and timely

14

Attributes of Good Reviews

The paper should be summarised To show that the reviewer understood the

paper The strengths and weaknesses of the

paper should be identified All advice for improving the paper

should be actionable If there are weaknesses, precisely what

should the author do about them? Which references should the author read? Respect for the fact that it is the author’s,

not the reviewer’s, paper.

15

Evaluation Criteria for Papers

Ethics Did the researcher act ethically? Did the researcher act to protect the

interests of data subjects (individuals or organisations)?

Research methods Are the research methods used

appropriate given the nature of the research problem?

Are the data collection and analysis methods appropriate?

16

Evaluation Criteria for Papers

Are the references correct, and up to date?

Is the presentation clear, concise and grammatically correct?

Are the concepts and arguments well organised, structured and defensible?

Are the findings/contributions appropriately positioned with respect to the existing literature in this area?

17

Rejection Criteria The article is uninteresting and no one

would want to read it The problem researched is trivial,

irrelevant or not a problem at all The article is so poorly constructed that

a completely new start is required Or the arguments are so

weakly/subjectively developed as to be meaningless

There are serious ethical concerns about the way the research was conducted E.g. plagiarism, use of deception,

illegal/unethical practices, failure to protect research subjects’ privacy

18

IT&People Criteria Criteria for papers are not bound to a particular methodology, but rather to: 1. is the topic relevant? 2. is the treatment new or newsmaking? 3. is the lit review adequate to support the framework presented? 4. Is the framework clear? 5. Is the evidence, discussion, description clear and convincing and does it

match the framework? 6. Does the paper apply standards appropriate to method (if quantitative

then defensible, if descriptive then thorough enough to reflect academic responsibility)

7. Is the paper well-written, well organized If no to any of the above, what is needed? However if the paper is so bad it

tires you to even think about it, then just reject. It is OK to reject, we have a lot of submissions. If promising theme but not well developed, make some general statements for revise and resubmit; if pretty good, then thorough critique to help authors.

19

How to Respond to Reviewers?

Make sure that you address everything that they ask for Even if you choose to dispute their view

Provide a detailed presentation of your changes in a two-column format Left column – Reviewer Comments Right column – Author revisions Revision notes may be 10-20 pages

long! Even longer than the paper itself.

20

What if…

The reviewer totally disagrees with your choice of method – and rejects for this reason alone?

You can’t change the method now! So you may have to go to the editor

(AE, SE, E) and ask for a different reviewer

Sometimes it is the editor who says this, so you learn not to send this kind of article to this journal again

21

Remember!

Most journals accept 8-15% of submissions And therefore reject about 85-92%

Conferences may accept 30, 50 or higher %

It is easier to reject (find fault) than accept (appreciate). Less experienced reviewers and editors tend to reject more – so if you have the chance to nominate a reviewer or editor, don’t pick a PhD student or recent graduate!

22

Decisions

If the editor offers a review and resubmit (R&R) decision, be very happy! If you can get past the first round of review

and stay in the process, that is 50% of the way to acceptance

Because 50% of papers don’t make it through. The R&R may be major, but you must do

as much as you can and explain your changes carefully. Don’t fight with the reviewers – you will lose,

even if you are right! To get 2, 3 even 4 R&Rs happens. Don’t

give up!

23

Journals

Each journal has its own culture, its own values

You need to learn about this – it will save you much time

MISQ, ISR, ISJ, CAIS, JAIS, JMIS, I&M, DSS

24

References

Davison, R.M., Vreede, G.J. de and Briggs, R.O. (2005) On Peer Review Standards for the Information Systems Literature, Communications of the AIS, 16, 49, 967-980.

Davison, R.M. (2003) Discussants and the Quality of Interaction at Conferences, Communications of the AIS, 11, 7, 128-136.