146
Research Fundamentals Dick Heinhuis

Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Embed Size (px)

Citation preview

Page 1: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Research Fundamentals

Dick Heinhuis

Page 2: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Programme

May 8th:- Influences on Research- Reading an article

May 15th: - Research question(s)- Conceptual mapping- Ethics and research- How to write

C 2

Page 3: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course goals

Being able to formulate an appropriate research question

Being able to criticise scientific literature Being able to write a research paper (?)

Page 4: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Let’s get started

Page 5: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Bryman, (2012), author of books on research methods since 1988

Research is full of false starts, blind alleys, mistakes, and enforced changes to research plans. But these are rarely written about in the final research

publications.

Books (no matter how well written) can only deal with generalities, so individual pieces of research may seem difficult to reconcile with the broader

picture.

There are plenty of things that can go wrong with a research plan so being flexible and being willing to persevere in adverse circumstances are desirable

traits in a social researcher.Nonetheless, the methodological principles and techniques outlined in this

book provide a road map for the research journey.

Page 6: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Research process

Literature review Concepts and theories Research questions Sampling Data collection Data analysis Writing up the research

Page 7: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The nature of IS researchIS research does not ‘exist in a bubble’ in a world apart from

the social sciences -it is integrated with them;

The variety and diversity of business and management studies makes it hard to find agreement on how IS research claims should be evaluated;

Perhaps it can only be seen as an ‘applied’ field because it is concerned with solving management problems;

Academic researchers add to theory “supported by bits of practice” and management consultants contribute to practice “backed by bits of theory”. (Gummesson, 2000)

Page 8: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

But it is all about ….

The quality of research

Page 9: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Criteria for research

Reliability – are measures consistent?

Replication/replicability – is study repeatable?

Validity – are conclusions well-founded?

Page 10: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Validity

Measurement (or construct) validity – do measures reflect concepts?

Internal validity – are causal relations between variables real?

External validity – can results be generalized beyond the research setting?

Ecological validity – are findings applicable to everyday life?

Page 11: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Influences on research(Bryman, 2011, 2012)

Page 12: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

1 Theory and research

Theory as a framework

Theory and data collection

Page 13: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Theory as a framework

Grand theoriesThese theories seek to explain much of human behavior, but are often considered outdated and incomplete in the face of modern research. Psychologists and researchers often use grand theories as a basis for exploration, but consider smaller theories and recent research as well.

Middle range theories

Page 14: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Theory and data collection

Which comes first, theory or research?

- should we examine a problem and try to work out how it’s caused and how it might be solved (inductive reasoning),

- or should we take a generally accepted theory of how things work and find evidence for it in the problem we examine (deductive reasoning)?

Page 15: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Deduction

Page 16: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual
Page 17: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

2 Epistemological considerations What is (or should be) considered acceptable

knowledge?

Can the social world be studied ‘scientifically’?

Is it appropriate to apply the methods of the natural sciences to social science research?

Page 18: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Epistemological considerations (how to study “the world”) Positivism Interpretivism Critical research

Page 19: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Positivism

Knowledge is based on the senses Apply the methods of natural sciences to all

research Knowledge is arrived at through gathering of facts Purpose of theory is to generate hypotheses Science must be value free (objective) A difference between normative and scientific

statements

Page 20: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Positivism

Roots in sociology (Saint Simon, Durkheim)

Page 21: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Interpretivism

A different approach is needed for social sciences (compared to natural sciences)

The objects are different: human action has a meaning for human beings

Social sciences need to understand the subjective meaning

Human behavior is complex

Page 22: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Interpretivism

Subject matter of the social sciences (people) demands non-positivist methods

Verstehen: interpretative understanding of social action (Weber 1947)

Attempts to see world from the actor’s perspective: subjective reality

Influenced by Symbolic Interactionism

Page 23: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Example: Symbolic Interactionism

Erving Goffman (1958), a prominent social theorist in this tradition, discusses roles dramaturgically, using an analogy to the theater, with human social behavior seen as more or less well scripted and with humans as role-taking actors. Role-taking is a key mechanism of interaction, for it permits us to take the other's perspective, to see what our actions might mean to the other actors with whom we interact. At other times, interactionists emphasize the improvisational quality of roles, with human social behavior seen as poorly scripted and with humans as role-making improvisers.

Page 24: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Critical theory

Sciences have an emancipatory role to play Academic research in not objective and neutral Emphasis on power relations

Page 25: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Critical theory

Assume (power) conflicts Roots in Marxism

Page 26: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Example

Security issues in an organization. The research focuses on the reasons of using or not using security features of a system by the employees.

Page 27: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Positivist approach

Search for theories that explain the use of IS systems, e.g. TAM, DeLone&McLean, TTF

Select suitable theory and use constructs from this theory

Pilot research Quantitative approach

Page 28: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual
Page 29: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Interpretivist approach

Meaning of security and IS for employees The role of colleagues Search for theories in sociology and psychology

that explain the role of security and work

Page 30: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Approach

Interviewing of managers and employees

Page 31: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Critical approach

Work implies power relations that have to be analysed

Organizations are institutions to maintain power relations

Security is related to these power relations What is the purpose of the security features and

what will it do to power relations

Page 32: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Approach

For instance action research in which employees are made aware of this power relation and the ways in which they can change it

Page 33: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

3 Ontological considerations (social constructions or reality)

Are social entities objective Do they have a reality external to social actors Or perceptions and actions of social actors

Page 34: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Ontology

Objectivism Constructionism

Page 35: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Objectivism

Objectivism is an ontological position that asserts that social phenomena and their meanings have an existence that is independent of social actors. It implies that social phenomena and the categories that we use in everyday discourse have an existence that is independent or separate from actors.

Page 36: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Constructivism

Social phenomena and their meanings are continually being generated by social actors and are produced through social interaction. Social phenomena and categories are in a constant state of revision. Researchers' own accounts of the social world are constructions.

Page 37: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Example

Safety and security of IS systems

Page 38: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Objectivism: the organization

Safety and security have a reality that exists without the individuals

Page 39: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Constructionism: Everyday interaction determines safety and

security

Page 40: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

4 Values personal beliefs or the feelings of researcher

all ‘preconceptions must be eradicated’ (Durkheim 1938)

affect every stage of research process

Page 41: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

5 Practical considerations

May influence or determine choices on:

research strategy

design

method

resources and costs

May be influenced or determined by:

nature of the topic

people being investigated

political acceptability

Page 42: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Practical considerations

time

cost/funding available

how much prior literature exists (theory testing or

theory building?)

topic (deviant activities/sensitive issues may be more

suited to qualitative research)

research is a compromise between the ideal and the

feasible

Page 43: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The politics of IS research

Effect of peer review Effect of publish requirements Effect of positivist tradition Effect of editor’s policy

Page 44: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reflection

Evaluate your own research paper according to these five influences and the effects on reliability, replicability and validity

Page 45: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Part 2

Literature review

Page 46: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The literature review

Essentially, we must read the existing literature to find out:

what is already known about the topic,

what concepts and theories have been applied to it,

which research methods have been applied to it,

what controversies exist about the topic and/or how it

has been studied,

what clashes of evidence exist, if any,

who the key contributors are.

Page 47: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

A literature review can be...

a list of relevant theories; a search for relevant information; a survey of writing on the subject; a way of learning about the subject; a help in finding dissertation topics and research

methods; an important component of a dissertation or

report.Adapted from Bruce (1994)

Page 48: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

So, you write a literature review because.....

you need to know what is already known about your research area;

you can learn from the mistakes of others; you can learn about different theoretical and methodological

approaches to your research area; it may help you to develop an analytic framework; it may lead you to consider including variables in your research

of which you might not have thought; it may suggest further research questions;it is required!

Page 49: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Getting the most from your reading

o Take lots of notes (including references);o Ask yourself questions about what you read (its value to

your topic, methods used, conclusions);o Find research questions (perhaps because what you read

indicates contention);o Identify sources to which you will want to return later on;o Be selective (read for relevance);o Don’t stop reading!!

Page 50: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How to search the literature: 1

Read books and articles you know, or are recommended by your supervisor;

Keep notes based on your reading; Note the keywords used; Note other literature which might be interesting

later; Generate your own keywords.

Page 51: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How to search the literature: 2

Search your institution’s library; Search online (using an appropriate database); Examine titles and abstracts for relevance; Retrieve selected items, read, take notes; Check regularly for new material.

Page 52: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Bryman: Social Research Methods, 4th edition

One way of searching the literature

Figure 5.1page 119

Page 53: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

A word (or two) of warning about the Internet

Search engines find sites but do not evaluate them Dictionaries and encyclopaedias are fine for basic definitions,

but little else Many Internet sites serve a commercial purpose, so don’t be

taken in by propaganda Rely on academic sites (usually.ac or .edu), government sites

(.gov), not-for-profit institutions (.org), in that order. Download and save all material found on the Internet. Sites are

dynamic and you may not find the data a second time. Your institution may also require proof when defending your thesis.

Page 54: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Putting your reading to work

Through......o putting material together which is not usually related

(synthesized coherence);o building up separate items into a consensus

(progressive coherence);o showing lots of related contributions but no overall

agreement (non-coherence);o revealing a gap in the literature;o developing a new perspective;o showing the existing literature to be simply wrong.

(Golden-Biddle and Locke, 1997:43).

Page 55: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Two main approaches to a literature review: 1

1. Systematic review: a) Very extensive search of the specific research field; b) Grouping of sources into categories:

* very relevant, based on good research,* very relevant, weak research,* low relevance, or too general;

c) Giving a step-by-step report on the search method used, decisions taken and derived conclusions. (Tranfield et

al, 2003:209)

Page 56: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Two main approaches to a literature review: 2

2. Narrative Review:

a) Limited search for the most interesting contributions;

b) Concentration on connections between sources located and research questions;

c) Revision of research questions in the light of readings.

Page 57: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Systematic or narrative review?

Systematic: Positivist; “Quantity” focus; Theory informs the search; Synthesis is the research

goal.

Narrative: Intrepretivist; “Quality” focus; Theory emerges from the

search; Identifying the discourse

is the research goal.

Page 58: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Plagiarism: Crime and Punishment

Plagiarism means pretending that we, ourselves, wrote what others actually wrote;

Plagiarism might be accidental:

- not using quotation marks for direct quotes might

be careless rather than deceitful;

- or, forgetting to cite a source in the text; Plagiarism might not be seen for what it is:

- recycling our own material from previously submitted work;

- not referencing ourselves as the author of our own older work.

Page 59: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Questions to ask when doing a literature review: 1

Is your reading list up-to-date? Are there any new areas of interest? What have you read recently? How much time do you allocate to reading? What have you learned from your reading?

Page 60: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Questions to ask when doing a literature review: 2

Has this changed your understanding of your subject area?

Has your reading influenced your research design? Has it given you ideas about your hypotheses and

research tools? Have you been taking notes from your reading?

Adapted from Bruce, 1994

Page 61: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Key Points

Writing a literature review is a means of reviewing the main ideas and research relating to your chosen area of interest;

A competent literature review confirms you as someone who is competent in the subject area;

A great deal of the work of writing a literature review is based upon reading the work of other researchers in your subject area; key skills can be acquired to help you get the most from your reading;

Systematic review is a method that is gaining in popularity in business research as a way of enhancing the reliability of literature searching and review;

Narrative review is a more traditional approach which has advantages of flexibility that can make it more appropriate for inductive research and qualitative research designs.

Page 62: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How to read and analyse scientific articles

Background of an article General structure of an article Important “extra’s” of an article

Page 63: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Background of an article

Journal

Author(s)

Background of the author(s)

Page 64: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reading an article

Structure of an article:

- Introduction (literature review)- Methods- Results- Discussion

A. McNeal, Hampshire College: How to read a scientific research paper- a four-step guide

Page 65: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Introduction (literature review)

What is the overall purpose of the research How does the research fit into the context of its

field (new, settling a controversy) Relevance of the literature review

A. McNeal, Hampshire College

Page 66: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Methods

What research What kind of experiment Comparison other research Quality of the research: sample population Research methods

A. McNeal, Hampshire College

Page 67: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Results

What are the most important findings Does this becomes clear from the data in the

article Other patterns

A. McNeal, Hampshire College

Page 68: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Discussion

Do you agree with the conclusions Are there other factors influencing the results

A. McNeal, Hampshire College

Page 69: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How to read a scientific paper

How are they organized How to prepare to read a paper Difficulties How to understand and evaluate

J.W. Little and R. Parker, University of Arizona, How to read a scientific paper, 2009

Page 70: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Organization of the paper

Summary Introduction Materials and methods Results Discussion

J.W. Little and R. Parker, University of Arizona, 2009

Page 71: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Evaluating a paper

What questions does the paper address What are the main conclusions of the paper What evidence supports those conclusions Do the data support the conclusions What is the quality of the evidence Why are the conclusions important

J.W. Little and R. Parker, University of Arizona, 2009

Page 72: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Quality of evidence

Understand the methods used in the experiments/research

Limitations of the methods (sample)

J.W. Little and R. Parker, University of Arizona, 2009

Page 73: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

A guide to reading and analysing academic articles Consider the article as a whole Determine the overall purpose,structure, design Read the article Evaluation of the article

A. Graham, Yokon College, 2009

Page 74: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The article as a whole

Who is writing What are the author’s qualifications What audience What is the article about What sources

A. Graham, Yokon College, 2009

Page 75: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The overall purpose, structure

What is the author’s main point What evidence is used What limits What is the author’s point of view

A. Graham, Yokon College, 2009

Page 76: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Evaluation of the article

What was left unfinished Did it make the case What are the implications of the article Is the organization of the article clear

Page 77: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reading an article: part 1

Author(s): background, university, career Journal: quality, peer review Subject/purpose of the article/research (reading

the introduction) Screen remaining part of the article Read conclusions References

Page 78: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reading an article: part 2

Decide relevance of the article- No use- Worth citing (example of research)- Worth citing (sideline)- Key article- Use references

Page 79: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reading an article: part 3 (based on purpose) Screen/read for citing

Read article

Page 80: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Read an article

Research question Previous research (conclusions) Relevance theoretical part The model The research method(fieldwork, experiment,

sample) Analysis method (statistics) Conclusions

Page 81: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Relevant sites

http://home.aisnet.org/ http://www.ieee.org/portal/site

Page 82: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Next week

Read articles Wade and Davis Review 3 research questions (presentation)

Page 83: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Part 3: Research Question

Dick Heinhuis

Page 84: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

84

Research Question

How to come up with a suitable research question?

Page 85: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual
Page 86: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

86Course Research Methods Lecture 2

Page 87: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

87Course Research Methods Lecture 2

Page 88: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

88Course Research Methods Lecture 2

Page 89: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

89Course Research Methods Lecture 2

Page 90: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

90Course Research Methods Lecture 2

Page 91: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

91Course Research Methods Lecture 2

Page 92: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

92Course Research Methods Lecture 2

Page 93: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

93Course Research Methods Lecture 2

Page 94: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

94Course Research Methods Lecture 2

Page 95: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

95

Research Question

How to come up with a suitable research question?

Bryman, 2011

Maxwell, 2013

Page 96: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 296

Relevance of Research Questions

Guide literature search Guide decisions about research design Guide decisions about data collection Guide decisions about analysis of data Guide writing up Avoids taking unnecessary paths

Page 97: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 297

Sources of resource questions

Personal interest/experience Theory The research literature Puzzles New developments (organizational/technical) Organizational problems

Page 98: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2 98

Page 99: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 299

Criteria for evaluating research questions

Questions should be clear Questions should be researchable Questions should connect with established theory

and research (?) Questions should be linked to each other Questions should have potential for making a

contribution to knowledge Questions should be neither too broad nor too narrow

Page 100: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Research question Dissertation Dick Heinhuis

Which factors explain consumer channel choice in an ICT enabled multichannel configuration, therewith finding an explanation for the trial, adoption and choice of a new channel?

Course Research Methods Lecture 2 100

Page 101: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Subquestions

Which theories can be used to find the factors that explain the trial, adoption and choice of an ICT enabled channel by customers in a multichannel configuration?

Is it possible to arrive at a model based on these theories that explains the use of ICT enabled channels?

Can this model be confirmed empirically?Course Research Methods Lecture 2 101

Page 102: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2 102

Page 103: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Make your own choices Choose a topic that interests you Ask yourself whether you can answer the research question Read a lot, read thoroughly and appropriately. Identify your strengths, weaknesses, interests, and personal development

opportunities. Don’t pre-commit to one idea, approach, research design, or data source. Use opportunities to talk to others in your own field and other fields about

your proposed research. Research something that is likely to be interesting to others: either

practitioners or researchers. Start writing early. Analysis takes much longer than you think, but this is

where ‘added value’ can be gained. Remember that this is not your life work or a bid for a Nobel Prize. Listen to your supervisor, but make your own choices.

103Course Research Methods Lecture 2

Page 104: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2 104

Six honest serving men,(They taught me all I knew);Their names are What and Why and When,And How and Where and Who

(Kipling)

(

Page 105: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Descriptive and explanatory research

Descriptive: what, when, where and who Explanatory: why and how

Has consequences for research design and research strategy

Course Research Methods Lecture 2 105

Page 106: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2 106

As a general rule of thumb, if a quick Google search can answer a research question, it’s likely not very effective (George Mason University) 

Page 107: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Reflection

Evaluate your own research question

Page 108: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Ethical Issues

Dick Heinhuis & Ronald Kleijn

Page 109: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Ethical issues

How should we treat the people on whom we conduct research?

Are there activities in which we should – or should not – engage in our relations with those people?

Page 122

109Course Research Methods Lecture 2

Page 110: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Based on Diener and Chandall (1978) we can say that behaving ethically means:

• No harm should come to research participants,

• They should agree to participate and know what the research is about,

• Their privacy should not be invaded,

• They should not be lied to or cheated.

What is “ethical” research?

Page 128

110Course Research Methods Lecture 2

Page 111: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Intermezzo

What about the use of students?

Course Research Methods Lecture 2 111

Page 112: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Must be important

Lazar, J. et al., (2010), Research Methods in Human-Computer Interaction:

pp. 391 – 392

(about Online research!)

Course Research Methods Lecture 2 112

Page 113: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How could you harm research participants?

Physically By damaging their development or self-esteem

By causing stress By hurting their career prospects or employment

opportunities By breaking confidentiality By revealing their identity

Page 128

113Course Research Methods Lecture 2

Page 114: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2114

Milgram’s research (1963)

Milgram was concerned with the processes whereby a person can be induced to cause extreme harm to another by virtue of being ordered to do so. To investigate this issue further, he devised a laboratory experiment. Volunteers were recruited to act out the role of teachers who punished learners (who were accomplices of the experimenter) by submitting them to electric shocks when they gave incorrect answers to questions.

Page 115: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2115

Zimbardo et al. (1973)

Experiments conducted by Zimbardo and his graduate students from the Department of Psychology at Stanford University, California, involved creating a mock prison, in order to examine the roles played by prisoners and guards. Twenty-one male participants were selected from a group of seventy-five who responded to an advertisement in a local newspaper. Individuals were selected on the basis that they were mature, emotionally stable, middle class, well educated, and had no criminal record. Each was paid $15 per day to participate in the study. A coin was flipped in order to decide if the participant was to play the role of prisoner or guard.

Page 116: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Course Research Methods Lecture 2116

French television program

Page 117: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Tuskegee Syphilis Study (1932 – 1972)

400 infected men compared with 200 non infected men

400 men were never treated with penicillin In 1972 only 74 participants were still alive; at

least 28, but possibly more than 100, died of a direct result of the complications of syphilis

Course Research Methods Lecture 2 117

Page 118: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Research participants must know what they are and what the research process is

implementing this principle `is easier said than done‘ (Homan, 1991:73).

It is extremely difficult to present prospective participants with absolutely all the information that might be required to make an informed decision about their involvement.

In ethnographic research, the researcher is likely to come into contact with a wide spectrum of people, and ensuring that absolutely everyone has the opportunity for informed consent is not practicable, because it would be extremely disruptive in everyday contexts. Page 133

118Course Research Methods Lecture 2

Page 119: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Privacy is very much linked to the notion of informed consent

The research participant does not abrogate the right to privacy entirely by providing informed consent

Covert methods are usually deemed to be violations of the privacy principle

The issue of privacy is invariably linked to issues of anonymity and confidentiality in the research process

Invasion of privacy

Page 136

119Course Research Methods Lecture 2

Page 120: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Lies, damned lies and research

Deception usually means we represent our research as something other than it is, so that participants will respond more naturally.

This means it is quite a widespread practice. So deception means not telling the whole truth, while not

actually telling a lie. We must be vigilant in keeping deception to a minimum

and, when it is necessary to the research, mitigating its degree and effects as much as possible.

Apart from moral objections to deception, research participants and funders can become wary of being fooled, or tricked into providing data.

Page 137

120Course Research Methods Lecture 2

Page 121: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

So why should there be a problem?

Unfortunately, a lot of writers about ethics in business differ about what is and is not ethically acceptable.

The same issues seem to be always with us – they don’t seem to get resolved.

Certain research methods have a bad name because they are identified with a few extreme cases.

But the real problem is that the potential to behave unethically in research is constant and it does not just depend on particular situations or methods

Page 123

121Course Research Methods Lecture 2

Page 122: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Various ethical stances are possible

Universalism absolute rules about un/acceptable conduct

Situation ethics case-by-case assessment

the end justifies the means? Ethical transgression is pervasive

virtually all research involves some ethically questionable

practices

Key concept 5.2Page 124

122Course Research Methods Lecture 2

Page 123: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Doing the right thing….. You can try to do the best you can by making yourself familiar

with guidelines set by respectable institutions: The Academy of Management: tries to “..treat…research

subjects, and clients with respect, dignity, fairness, and caring.” The Market Research Society: the first rule for researchers is that

they “shall ensure that participation in their activities is based on voluntary informed consent.”

The Social Research Association: includes a guideline that “Social researchers must strive to protect subjects from undue harm arising as a consequence of their participation in research. This requires that subjects’ participation should be voluntary and as fully informed as possible”.

Page 122

123Course Research Methods Lecture 2

Page 124: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

What does your own institution require?

Most colleges and universities have developed their own guidelines for ethical research. Some of the questions they might pose are as follows:

1. Is the study funded?2. Is the research compromised by the source of funding?3. Are there potential conflicts of interest in the financial or organisational arrangements?4. Will confidentiality be maintained appropriately at all stages of enquiry? 5. Will human rights and dignities be actively respected? 6. Will highly personal, intimate, or other private or confidential information be sought? 7. Will there be any harm, discomfort, physical, or psychological risks?8. Will participants be involved whose ability to give informed voluntary consent may be limited?9. Will the study involve obtaining or processing personal data relating to living individuals?

Tips and skills, page 127

124Course Research Methods Lecture 2

Page 125: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Legal considerations

The 1998 Data Protection Act states that personal data must be:

obtained only for one or more specified and lawful purpose and not further processed in any manner incompatible with that purpose or those purposes,

adequate, relevant and not excessive in relation to the purpose or purposes for which they are processed,

not kept longer than necessary, (for example)

Copyright is an intellectual property right, extending to interviews – in which the interviewee keeps the copyright to the spoken words. Permission is needed from the interviewee in order to share this data.

Page 140

125Course Research Methods Lecture 2

Page 126: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

The difficulties of ethical decision-making: a summary

The boundary between ethical and unethical practices is not precise

The potential for deception / lack of informed consent pervades most social research

There is insufficient guidance on marginal areas of research (but be familiar with the existing guidelines)

Internet-based research provides new ethical dilemmas, for which we are still debating solutions.

Page 143

126Course Research Methods Lecture 2

Page 127: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Issues in writing up social research

Many people find writing up research more difficult than carrying it out You must convince your readers about the

credibility of the knowledge claims you make

Good writing is to do with developing your style so that it is persuasive and convincing

Page 684

Page 128: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Tips for writing up your research

Start early!Be persuasive.Get feedback.

Avoid sexist, racist, and disablist language.

Structure your writing:Title page – acknowledgements – list of contents – abstract – introduction - literature review - research methods – results – discussion – conclusion – appendices - references. Pages 685-691

Page 129: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Typical ways of constructing an argument

Figure 29.1Page 688

Page 130: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Lessons to be learned from the article Strong opening statements grab the reader’s attention

and act as signposts to what the article is about The rationale of the research is spelled out clearly Research questions are spelled out very specifically The research methods, nature of data, measurement of

concepts, sampling and data analysis are clearly and explicitly summarized

The presentation of findings is oriented specifically to the research questions

The conclusion spells out the implications of the findings for the research questions and the theories examined earlier in the paper

Pages 694, 695

Page 131: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Lessons to be learned from the article Strong opening sentences attract our attention and give a

clear indication of the nature and content of the article. The rationale of the research is clearly identified. Research questions are specified but are somewhat more

open-ended than in other publications. The research methods are outlined and an indication is

given of the approach to analysis. The presentation of main themes is geared to the broad

research questions that motivated the researcher’s interest. The discussion and conclusion elucidate in a more specific

way the significance of the results for the research questions.

Page 699

Page 132: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Type your footer here 132

Page 133: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Key points

Page 707

Page 134: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Checklist for writing up your research: 1.

Have you clearly specified your research questions?

Have you clearly indicated how the literature you have read relates to your research questions?

Is your discussion of the literature critical and organized so that it is not just a summary of what you have read?

Have you clearly outlined your research design and your research methods?

Page 135: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Checklist for writing up your research: 2.

Have you presented your data in a manner that relates to your research questions?

Does your discussion of your findings show how they relate to your research questions?

And on how they shed light on the literature that you presented?

Are the interpretations of the data that you offer fully supported with tables, figures, or segments from transcripts?

Page 136: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Checklist for writing up your research: 3.

Does your writing avoid sexist, racist, and disablist language?

Have you included all appendices that you need (interview schedule, communications with research participants)?

Have you checked that your list of references includes all the items referred to in your text?

And that it follows precisely the style that your institution requires?

Page 137: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Checklist for writing up your research: 4.

Have you ensured that your institution’s requirements for submitting projects are fully met in terms of word length (neither too long nor too short) and whether an abstract and table of contents are required?

Have you ensured that you do not quote excessively when presenting the literature?

Have you fully acknowledged the work of others so that you cannot be accused of plagiarism?

Pages 706, 707

Page 138: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Use the 3W method of Heinhuis

Type your footer here 138

Page 139: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Write, write, write

Type your footer here 139

Page 140: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

After the project

Getting published in the Top Journals

Type your footer here 140

Page 141: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Straub (Editor-in-Chief) MIS Quarterly, 2009

How high is this hurdle? The stark reality is that a few hundreds of people publish a very small number of top journal articles over half a career while very few persons publish dozens upon dozens, with the middle part of the distribution being extremely sparse (Chua et al. 2003). Known as a power distribution, this description of IS publishing behavior has been found repeatedly in scientometric studies (Athey and Plotnicki 2000; Dennis et al. 2006).Type your footer here 141

Page 142: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Type your footer here 142

The simplistic and often-offered explanation for this is that, even assuming the quality is present, the top journals do not have the space to publish all of these articles. Therefore, the gatekeepers restrict the pipeline to correspond to the space limitations. While there is some evidence that this has been true in the past (Dennis et al. 2006), is it still the case? Since the Dennis et al. article appeared in 2006 with its clarion call for top IS journals to increase their ability to print more articles and help to level the playing field against other business disciplines (Kozar et al. 2006),

Information Systems Research has explicitly increased its pages and MIS Quarterly has not only published an extra issue since 2006, but also now has the ability to print many more papers than it has traditionally published. Electronic journals like Journal of the AIS, of course, have the luxury of not having such space limitations. Thus, in 2009, this explanation may be overblown.

Page 143: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Type your footer here 143

Page 144: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

How about social psychology? Stapel published “fraud determined”papers in:

Personality and Social Psychology Bulletin Journal of Personality and Social Psychology Journal of Experimental Social Psychology British Journal of Social Psychology Motivation and Emotion European Journal of Social Psychology Self and Identity Journal of Environmental Psychology Social Influence Basic and Applied Social Psychology Psychological Science Cognitive Methods in Social Psychology Science Social Cognition Journal of Consumer Research

Type your footer here 144

Page 145: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Why the Power Distribution

I believe that this underlying problem is that many do not fully understand the reasons top journals accept papers in the first place. This was the point that Soon Ang, Roberto Evaristo, and I made in 1994, one that had been articulated by numerous management researchers to that time and afterward. To raise the general level of appreciation of these factors in the profession, it is useful to know why papers are accepted at the top journals, a set of criteria that seems to be fairly consistent across the business disciplines at least.

Type your footer here 145

Page 146: Research Fundamentals Dick Heinhuis. Programme May 8 th : - Influences on Research - Reading an article May 15 th : - Research question(s) - Conceptual

Type your footer here 146