Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
IS Research Development an International Perspective
Nicholas C. Romano, Jr.
SDA BocconiInformation Systems Division
Università Commerciale Luigi [email protected]
Spears School of BusinessManagement Science and Information Systems
Oklahoma State University – [email protected]
Università degli Studi di VeronaFacoltà di Economia
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Talk Outline1. Personal Introduction
2. Business and IS Research
3. Research Defined and Ways of Knowing
4. Scientific Method and Theory
5. Research Process
6. Research Paradigms
7. Theory Construction
8. IS Research Methods
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
1. Personal Introduction
• Nicholas C. Romano, Jr., Ph.D.– Associate Professor, MSIS Spears School OSU– Ph.D. from University of Arizona in MIS (1998)– Family Man, Husband and Father
• Wife Rosalina, 2 Daughters Isabella (8) Gabriela (6) 1 Son Nico (3)• Dog Osa
– AIS Council Member (Americas Representative – 2007-2009)• Very Active in AIS and the AMCIS and HICSS Conferences
– 20 years’ work experience, much of it in group support for teamwork and projects
– Former IBMer (Dad, Brother, Sister, Brother in law, Great Uncle, Second cousin, others I am sure)
– Occasional, but Terrible Golfer ( FORE!!!) – want to do it more..
3
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is Business Research?
• the “systematic and objective process of gathering, recording and analyzing data for aid in making business decisions” (Zikmund, Business Research Methods, 2002, p. 6)
• Systematic and Objective Distinguish Business Research
• Important tool for managers and decision-makers in corporate and non-corporate organizations
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
When is Business Research Used?
Business research methods used in situations of uncertainty, when decision-makers face two or more courses of action and seek to select the best possible alternative under the circumstances
Aims to improve the quality of decision-making which, in turn, benefits the organization and helps ensure its continuity and efficiency
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Typical Users of Business Research Methods
• Businesses and Corporations
• Public-Sector Agencies
• Consulting Firms
• Research Institutes
• Non-Governmental Organizations
• Non-Profit Organizations
• Independent Researchers and Consultants
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
General Business Conditions andCorporate Research
• Short- & Long-Range Forecasting, • Business and Industry Trends• Global Environments • Inflation and Pricing • Plant and Warehouse Location • Acquisitions
Financial and Accounting Research
• Forecasts of financial interest rate trends, • Stock,bond and commodity value
predictions • capital formation alternatives• mergers and acquisitions • risk-return trade-offs • portfolio analysis • impact of taxes • research on financial institutions • expected rate of return • capital asset pricing models • credit risk • cost analysis
Fields Where Business Research is Often Used – (1)
Management and OrganizationalBehaviour Research
• Total Quality Management• Morale and Job Satisfaction• Leadership Style• Employee Productivity• Organizational Effectiveness• Structural ssues• Absenteeism and turnover• Organizational Climate
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Information Systems Research
• Knowledge and information needs assessment• Computer information system use and
evaluation• Technical suppot satisfaction• Database analysis• Data mining• Enterprise resource planning systems• Customer relationship management systems
Corporate Responsibility Research
• Ecological Impact• Legal Constraints on advertising and
promotion• Sex, age and racial discrimination / worker
equity• Social values and ethics
Sales and Marketing Research
• Market Potentials• Market Share• Market segmentation• Market characteristics• Sales Analysis• Establishment of sales quotas• Distribution channels• New product concepts• Test markets• Advertising research• Buyer behaviour• Customer satisfaction• Website visitation rates
Fields Where Business Research is Often Used – (2)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Value of Business Research for Managers – (1)
Uncertainty Reduction and improved decision-making quality with several consequent advantages (e.g. strategic, operational) and benefits for Firms
Business Research Methods can be employed in 4 stages:
(1)Identification of problems and/or opportunitiesUseful for strategy planning, analysis of internal and external organizational environment
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Value of Business Research for Managers – (2)
(2) Diagnosis and Assessment of problems and/or opportunitiesGain insight into underlying reasons and causes for the situation. If there is a problem, it asks what happened and why? If there is an opportunity, it seeks to explore, clarify and refine the nature of the opportunity and, in the case of multiple opportunities, seeks to set priorities
(3) Selection and Implementation of Courses of ActionAfter alternative courses of action have been determined, selection of the best possible course.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Value of Business Research for Managers – (3)
An important consideration is the quality of forecasting which is an essential tool of research
(4) Evaluation of Courses of Action
Business Research Methods are used after a course of action has been implemented in order to determine whether activities have been properly implemented and have accomplished what they intended to do
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Value of Business Research for Managers – (4)
Evaluation Research
Formal objective measurement and evaluation of the extent which an activity, project or program has achieved its goal, and the factors which influence performance (e.g. audits).
Formal objective measurement and evaluation of the extent to which on-going activities, projects or programs are meeting their goals (performance-monitoring research)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Yes
Yes
Yes
Yes
NO
NO
NO
NO
When Should Business Research be Undertaken?
Is sufficient timeavailable?
Is information inadequate?
High importance of decision?
Research benefits greater than costs?
Do Undertake Business Research
Do notundertake
Business Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Value and Costs of Undertaking Business Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Building Blocks
Measurements of phenomena (e.g. sales statistics of a department store)
DATA
Determination of relationship amongst data with a view to facilitating understanding of the phenomena,
their relationships and decision-making(e.g. past and predicted future sales trends)
INFORMATION
KNOWLEDGE
Blend of information, experience and in-sights that provides a framework that can be thoughtfully evaluated
when assessing new information or evaluating relevant situations
WISDOMBlend of Knowledge information, experience and in-sights that
provides a framework that can be thoughtfully evaluated when assessing new information or evaluating
relevant situations
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Introduction
Definition
Ways of Knowing
The Scientific Method
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Origin of the Word “Research”From French word “recherche”
to travel through or survey.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is Research? “If research is to make the contribution to practice
that is now possible, we must start with an adequate concept of the nature of research.”
“Research is an unusually stubborn and persisting effort to think straight which involves the gathering and the intelligent use of relevant data”
Hamlin, H. M. (1966) What is Research? American Vocational Journal, September 14-16. See: http://www.cals.ncsu.edu/agexed/aee578/hamlin.html
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Pendulum of Scholarship in Business Management Schools
Professional Learning
Community
ManagementConsulting
Disciplinary Science
Social System of Practice
-Practitioners-Managers-Businesses-Trade Associations-Management Societies
Social Systemof Science
-Scientists-Graduate Schools-Research Institutes-Scholarly Societies
Adapted From Van de Ven, Engaged Scholarship
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Elements of Research Not Frequently Discussed
• Research is not a linear process• It is just written up like it is. One study leads to others
• Research is a social process• Not because research is social but because results
must enter into a social “learned” society (be read and cited)
• Research value (impact) more a question of importance than volume• But volume is a wonderful, simple measure of productivity• But, one good published idea is worth more than 100 articles
• How do you know value? CITES!!!Citation Tools [isi web of science; Google Scholar, Citeseer, SSRN, Libra, Publish or Perish]
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Little Discussed Elements (II)• Research is for posterity
– i.e., it has a different time scale than consulting• Refereed archival journals versus the Internet
• Research builds upon the past– …by tearing it down (theory building), – or by supporting it (replication studies; theory extension)
• Research not published is virtually worthless• The importance is more to be read than to read!
• Research demands special form of writing and language
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Some Myths About Research
• Purpose of research is to “Prove” or “Confirm” a theory
• Research findings are presented as “Complete” and “Conclusive” answers
• Research Scientists come to “Consensus” or “Agreements” on how things work (i.e. Global Warming; Pluto a planet)
• There is a hierarchy of research methodologies that places true “experimental” research at the top.
• NONE OF THESE ARE TRUE!
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Key Terms
Philosophy Epistemology
The Love of Knowledge
Distinguishing True (Real) Knowledge fromFalse (Pseudo) Knowledge
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Different Ways of “Knowing”• Authority
– Because someone you respected told you so
• Tenacity– Because it has withstood the test of time
• Serendipity– discovery by accident
• Logic / Reason– Because you figured it out with your mind
• Science (Research)– We’ll get to that shortly…
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Authority
• How do we know that the Earth is flat?
• Right, but how do I know?
• But how do you know?
• Because Claudius Ptolemy said so.
• Because The Pope said so. (Pope Paul V)
• Because I’m in charge and I amputting you (Galileo)in prison!
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Galileo Galilei (15641642) First to use telescope to study skyDiscovered Solar spots and Jupiter’s satellites (Galilean moons)Believed Earth moves around Sun
In 1632 he was convicted of heresy.In 1992 it was officially stated by the Pope that Galileo was right.
(360 years later)Authority is SLOW to change
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Tenacity
• Grandpa, how do I know that I should drink 8 cups of water per day?
• But how did he know?
• But how did he know?
• But how did THEY KNOW?!
• Because that’s what my father did.
• Because that’s what his father did.
• Because that’s what his father did!
•Well you’re alive, aren’t you?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Serendipity
Columbus is the archetype ofsurprising discoveries In 1492 Columbus sailed theocean blue In quest of a passage throughThe Indies and the orient too He discovered America,Serendipitous through andthrough.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
SerendipityIsaac Newton's famed apple falling from a tree, led to his musings about the nature of gravitation.
“In the year 1666 he retired again from Cambridge to his mother in Lincolnshire. Whilst he was pensively meandering in a garden it came into his thought that the power of gravity (which brought an apple from a tree to the ground) was not limited to a certain distance from earth, but that this power must extend much further than was usually thought. Why not as high as the Moon said he to himself & if so, that must influence her motion & perhaps retain her in her orbit, whereupon he fell a calculating what would be the effect of that supposition.“John Conduitt, Newton's assistant at the royal mint
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Serendipity
The Post-it note was invented in 1968 by Dr Spencer Silver, a 3M scientist who stumbled upon a glue (Acrylate-copolymer microspheres [adhesive formula] ) that was not sticky enough.
• In 1968, Silver developed a high-quality but "low-tack" adhesive, made of tiny, indestructible acrylic spheres that would stick only where they were tangent to a given surface, rather than flat up against it. As a result, the adhesive's grip was strong enough to hold papers together, but weak enough to allow the papers to be pulled apart again without being torn. More importantly, the adhesive could be used again and again.
• Silver wanted to market the adhesive as a spray, or as a surface for bulletin boards on which temporary notices could be easily posted and then removed. Over the next five years, Silver tried to interest his colleagues at 3M, informally and in presentations. A marketable form of the product proved elusive however, until Arthur Fry attended one of Silver's seminars.
• Fry sang in his church choir. He was frustrated the paper bookmarks he used to mark the songs in his hymnal would not stay put. In a moment of insight, Fry realized that Silver's reusable adhesive would provide precisely what he needed.
• Fry wrote up his idea for a reusable bookmark and presented it to his supervisors. Initially, management was skeptical, but the staff could not get enough of the samples Fry was passing around. Soon, 3M gave the invention its full support. It took another five years to perfect and design machines to manufacture the product, but in 1980, Post-it® Notes were introduced nationwide. Within two years, the product became a necessity in the office, schools, labs, libraries, and even in homes.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Logic and Reasoning• Understanding phenomena by analyzing with
our minds what we observe with our senses.
• Syllogism– A logical argument consisting of two premises and a
conclusion.– Example: Persons who smoke cigarettes have a high
rate of lung cancer. Persons who do not smoke cigarettes have a low rate of lung cancer. Therefore, smoking cigarettes causes lung cancer.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
– Persons who smoke cigarettes have a high rate of lung cancer and yellow teeth.
– Persons who do not smoke cigarettes have a low rate of lung cancer and yellow teeth.
– Therefore, smoking cigarettes causes lung cancer and yellow teeth.
Syllogism
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Well that sounds pretty good, I’ll just use logic and reasoning for my research!
• Problems with limiting our knowledge to what we can discover with logic and reason:– Subjectivity (Bias)
• We do not observe “the whole picture”• We have no external “check” on our logical thought
processes– Example: I observe that all stars follow a regular pattern
of motion in the sky in relation to the Earth. Therefore the Earth is stationary and at the center of the universe.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Why not just rely on pure observation?
What one observes: • May not be Quantifiable• May Change over time• May not be Reality• Can be based on Misinformation or Bias
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Why not just rely on pure observation?Count the Black Dots….How many do you see?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Why not just rely on pure observation?Are the Horizontal lines parallel or do they slope?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Actual building in Melbourne, Australia
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
How many legs does this elephant have?Why not just rely on pure observation?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Are the two boys the same or different?Why not just rely on pure observation?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Count the menWhy not just rely on pure observation?
Count the men AgainCount the men one last time
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Is the wine glass on or off the tray?Is the water glass standing or laying down
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Are the three purple Shapes Squares?Are their sides parallel?
Are they moving or still?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Ames RoomPeople seem to change size as they move around
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
How the Ames Room works
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di VeronaNicholas C. Romano, Jr.
Oklahoma State UniversityDoctoral Seminar – MSIS 6333 Wednesday August 21st, 2009
Actual Position of Person A
ApparentPosition of Person A
Actual and ApparentPosition of Person A
ApparentRoom shape Viewing Hole
How the Ames Room works
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Why not just rely on pure observation?
• Subjectivity (Bias)– “group A is nicer than group B”
• Recall (forgetfulness – selective memory)– What did you say to me last week about topic X?
• Interpretations or conclusions that lack convincing support– “most kids don’t care what their parents say”
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Even reason, when applied with bias,leads to irrationality and incorrect conclusions.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Logical Fallacies• Fallacies occur when we reach wrong conclusions
based on real observations or facts.• Examples:
– Cum hoc ergo propter hoc (with this, therefore because of this) – Attributing causality based on correlation
– Converse accident – Generalizing to a group based on an individual (or a small set of individuals)
– Accident – Specifying to an individual based on a group– Post hoc ergo propter hoc (after this, therefore because of this) –
Attributing causality based on temporality– See a list of 40 Fallacies at:
http://abyss.uoregon.edu/~js/glossary/fallacies.html
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
We all know that Penguins cannot fly.
Logic, one more thing that Penguins are not good at.
Penguins are Blank and White.
Old TV shows are Blank and White.
Therefore some Penguins are Old TV Shows.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
So I should just accept that ignorance is bliss and that I don’t know anything?
• Maybe if you want to sit on your porch all day doing nothing, but …….
– Authority isn’t Always Inaccurate• We need to rely on knowledge our parents, teachers, government tells us
– Tradition is Important• We all need a starting point and roots
– Logic and Reason are Powerful Tools• Our brains are like supercomputers
• We couldn’t survive without Thinking
• Each way of knowing can only lead us so far
• We need a method to correct for weaknesses of other approaches
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Scientific Method(or methods)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
How one discoversKnowledge
Extent to whichknowledge changesthrough time
Extent to whichFuture changes inKnowledge areexpected by user
From ancient texts orrevelations of inspiredindividuals.
Little.
None.
Unchangeable except byreinterpretation byauthorities, or by newinspired revelations, or bydivergence of mavericks.
Outrageousstereotype of user
Bible-thumpingfundamentalistor robe-draped monk; fondof Sunday-morning radio.
Crystal-hugging wearerOf tie-dyed T-shirts;listens to new-agemusic.
From personal insight,Or insight of others
May be considerable.
Can be expected, to thedegree that the userExpects personalDevelopment
As user changes or asUser encounters ideasof others
Geek with pocket protectorAnd calculator; watchesDiscovery Channel often.
Considerable.
Considerable.
By new observations orexperiments, and/or byreinterpretation of existing data.
From evidence generated byobservation of nature or byexperimentation.
Science and other kinds of knowledge
How knowledgechanges throughtime
Religious Knowledge Artistic/Mystic Knowledge Scientific Knowledge
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science and other kinds of knowledge (continued)Religious Knowledge Artistic/Mystic
Knowledge Scientific Knowledge
Certaintyof the user
High, given sufficient faith;Can be complete. High
Dependent on quality andExtent of evidence; should
never be complete.
AssumptionsAncient texts or Inspired
revelation havemeaning to modern or
future conditions.
personal feelingsAnd insights reflect
nature.
Nature has discernible,predictable, and
explainablepatterns of behavior.
Where users putTheir Faith
In the supernatural beingsThat they worship or in theauthorities who interpret
Texts and events.
In their ownperceptions.
In the honesty of peoplereporting scientific data (theincomes of whom dependon generation of that data),and in the human ability to
Understand nature.
Sources of Contradiction
Between different religions;between different texts
and/or authorities withinone religion; within
individual texts (as in thetwo accounts of human
origin in the JudeoChristian Genesis).
Between users, whoEach draw on their own
Personal Insights
Across time, asunderstanding changes;between fields, which
use different approachesand materials; and between
individuals, who usedifferent approaches and
materials.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Religion and Science
Science is based on skepticism and experimentReligion is based on faith
HoweverMany scientists are religious Also many leaders of religion have been great scientists
(Mendel – father of experimental Genetics - Monk)
Science and Religion are simply different parts of our livesScience cannot disprove the idea of GodReligion cannot prove that Science is wrong
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Scientific Inquiry as a way of Knowing
• Science is a disciplined, systematic way to understand the nature of the universe.
• Science uses empirical data to test falsifiable theories via a deductive method.
• WHAT THE HECK DOES THAT MEAN???
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science = order, explanation, rational methods, logic
• The main purpose of science is to trace, within the chaos and flux of phenomena, a consistent structure with order and meaning.
• This is called the philosophy of rationalism, rational as in conforming with reason.
• And the purpose of scientific understanding is to coordinate our experiences and bring them into a logical system.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science is also a Dialogue between Humankind and Nature.
• Science is far from a perfect instrument of knowledge, but it provides something that other philosophies fail to, concrete results.
• Science is a “candle in the dark'' to illuminate irrational beliefs or superstitions
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Six General Goals of Science
1. Organize & categorize things (typologies and ontologies)
2. Explain Past Events3. Predict Future Events4. Control Future Events5. Provide a Sense of Understanding6. Generalize Results
Adapted from: Reynolds, P. D. (1971). A primer in theoryconstruction. Indianapolis, The Bobbs-Merrill Company.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
More Specific Goals of Science
• Create Causal Models for phenomena of interest (Theory)
• Test the usefulness of our models (Experiments and other methods)
• Use those models to increase the likelihood people will survive and thrive. (Applications)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science – a DefinitionScience, ... organized systematic enterprise that gathers knowledge about the world and condenses the knowledge into testable laws and principles.
Diagnostic features of science that distinguish it from pseudoscience are:
1. Repeatability: The same phenomenon is sought again, preferably by Independent investigation, and the interpretation given to it is confirmed or discarded by means of novel analysis and experimentation.
2. Economy: Scientists attempt to abstract the information into the form that is both simplest and aesthetically most pleasing the combination Called Elegance while yielding the largest amount of information with the least amount of effort.
3. Mensuration: If some thing can be properly measured, using Universally accepted scales, generalizations about it are rendered unambiguous.
4. Heuristics: The best science stimulates further discovery, often in unpredictable New directions; and the new knowledge provides an additional test of the original principles that led to its discovery.
5. Consilience: The explanations of different phenomena most likely to survive are those that can be connected and proved consistent with one another.
Edward O. Wilson (1998) American Scientist, 86(1) Jan/Feb P.6.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process
1. Pick a research topic.
2. Formulate an appropriate research question related to that topic.
How do you do this?
Pick an outcome you think is interesting and ask, “What do think caused that outcome? and Why?”
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process
3. Refine the research question by hypothesizing relationship(s) between the variables
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process
4. Operationalize the variables
The conversion of abstract concept into concrete terms.
Measurement -- how do we know anything happened?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process5. Select an appropriate research technique
Examples: 1. Experiments2. Quasi-experiments3. Surveys4. Interviews5. Unobtrusive Data Collection6. Content Analysis7. Case Studies8. Action Research9. Design Science10. Simulation
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process6. Collect data – Measure attributes of the real world.
Classifying things that actually happen in the world with your operational scheme and then recording that data.
Things to consider:1. Quantitative vs. Qualitative2. Primary vs. Secondary3. Sample vs. Population4. Selection of Cases5. Validity
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process
7. Analyze Data
Look for systematic differences
STATISTICAL ANALYSIS
CONTENT ANALYSIS
HERMENUETICS
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process8. Interpretation of the results
What did you find?
How do your findings relate to other findings?
What are the theoretical implications, how will this impact other IS research?
What are the Practical implications, how will this impact IS practice?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Definition of the Scope of Lit. Rev.
Review of Selected Literature
Identification of initial findings- “Gap”
Gap Definition
Initial Research Objective
Taking the gap further ..
Development of:
The Value
Matrix The 3rd. Dimension VM
Hard & Soft Value
Footprints
Gap Asses. Tool
Research Questions
Value Matrix Value cube Footprints
Identification of:
Contributions to Knowledge and Theory
Contributions to Practice
Limitations of Framework/ Footprint
Validation of Frameworks
Answer to RQ
Review of: Scientific Paradigm Research strategies Techniques among others
Understanding of research methods
Definition of Phenomenological
Research Approach
Controls Criteria to evaluate results and the whole research
Research tools
8 Case Studies
Counting M.
Feedback : consultants &conferences
Application of R. Methods
Evolution of Frameworks
Acceptability of Frameworks
Development of:
Workshop
Exploratory Research
Initial Literature Review “Value Creation”
RESEARCH
OUT COMES
Why study Value? Answer
Weaknesses of current solutions Unsolved issues
PHASEPre-understanding
Study of Research Methods
Theory Building
Theory Testing
Evaluation of the research
Point of
Departure
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
“Thus, the task is not so much to see what no one has yet seen, but to think what no one has yet thought about that which everybody sees.”Arthur Schopenhauer
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science tests all hypotheses, but some scientists summarily dismiss opposing views:
Science has proven itself to be an infallible tool for unlocking certain areas of knowledge, but it's not logical to conclude from this that all thinking by scientists is infallible.
Science can be used to discover many things, yet some scientists wrongly presume that all things can be discovered through it.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is Science?
Science =
A set of facts and the theories that explain the facts.
Whatever’s being done by institutions carrying on “scientific” activity.
A particular approach, the scientific method
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Data• Data are concrete facts, records or collections of
information we gather about phenomena of interest.– Usually expressed in numerical terms
• Data may express facts of individuals (blood pressure, disease status, response to survey questions) or geopolitical areas (crime rate, death rate, per capita income)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Empiricism
• An empirical (observational) approach to research is one that strives to be objective.
• It expresses concepts in concrete, tangible ways.
– Not fuzzy or abstract ways…
• Empiricism tests relationships between these concepts.
• Empiricism is closely tied to the type of data that you use.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Limitations of Empirical Data
• Our research cannot attempt to make value judgments– Cannot answer questions such as, “What is morally right?”
or “Which drug is better?”– Can analyze people’s opinions about such things
• Some concepts are difficult to measure with numbers– If we wanted to know if “happier people” lived longer,
how could we measure happiness?
• Sometimes we just can’t get the data
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Popular Fictions
• The goal of science is to accumulate facts• Science distorts reality and can’t do justice to
the fullness of human experience.• Scientific knowledge is truth.• Science is concerned primarily with solving
practical and social problems.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
“Science is neither a philosophy nor a belief system. It is a combination of mental operations that has become increasingly the habit of educated peoples, a culture of illuminations hit upon by a fortunate turn of history that yielded the most effective way of learning about the real world ever conceived.”
Edward O. WilsonConsilience: The Unity of Knowledge
So then, what is Science?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Heart of the Matter
Why do we see what we do and not see something else?
Paradigm, Ontology, Epistemology, Axiology
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Paradigm
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Some Fundamental Paradigms
• Positivist Research• Interpretivist Research• Criticalist Research• Design Research• Action Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Taxonomies
It is important to
understand where you fit
in regards to research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Ontology Epistemology Methodology Methods Sources
What’s outthere to know?
What and how can we know about it?
How can we go about acquiring that knowledge? Which precise
procedures canwe use to acquireit?
Which data canwe collect?
The interrelationship between the building blocks of research (Grix 2004: 66)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Study of how we evaluate our investigations.
Strategy or plan of action; Research Design – shapes our choice of methods and links that choice to the research outcomes.
Study of values: what values does an individual or group hold and why?
Study that explores the nature of knowledge: for example, on what does knowledge depend and how can we be certain of what we know?
Study that describes the nature of reality: for example, what is real and what is not,what is fundamental and what is derivative?
Philosophical Assumptions
Ontology
Basic Belief
Epistemology
Methodology
Axiology
Research PerspectivesPositivist Interpretivist Design
Single Reality; Knowable, Probabilistic
Multiple Realities; Socially Constructed
Multiple Contextually Situated alternative World States; Socio-technologically enabled
Objective; Dispassionate. Detached observer of truth
Subjective, i.e. values and knowledge emerge from the researcher-participant interaction
Knowing through making: Objectively constrained construction within a context. Iterative circumscription reveals meaning
Observational; Quantitative; Statistical
Participative; Qualitative; Hermeneutical; Dialectical.
Developmental; Measure artifactual impacts one the composite system
Truth: Universal and Beautiful; Predictive.
Understanding: Situated and Descriptive
Control; Creation; Progress (i.e. improvement) Understanding
CriteriologyInternal validity; construct validity; external validity and reliability.
Credibility: triangulation multiple data sourcesConfirmability
Expected Functionality and performanceUseful and easy to useSolves Problem at hand
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Behavioral vs. Design Science (Hevner, et al. 2004)
Behavioural Science Research (BSR) Design Science Research (DSR)
Origin Natural Science Engineering, Sciences of the Artificial
Paradigm Problem understanding paradigm Problem solving paradigm
Objective
develop and justify theories which explain or predict organizational human phenomena surrounding the analysis, design, implementation, management, and use of information Systems
create innovations that define ideas, practices, technical capabilities, and product through the analysis, design, implementation, management, and use of information systems
Object human-computer-interaction IT artefact design
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
IS Research CycleBehavioral Science Research
DesignScience Research
Understanding,Truth
Design,IS Artifacts
Utility,Usage in Practice
Theory Building,Hypotheses
Build
Evaluate/ApplyTheorize
Justify
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Reassembling the Dimensions• A given research project is a point in multidimensional space.• Some regions of this space are popular:
These often go together as Quantitative research.
These often go together as Qualitative research.
Observational Interventionist
Biophysical Psychosocial
Sample Case(s)
Quantitative Qualitative
Objective Subjective
Neutral Partisan
topic
scope
method
mode
ideology
politics
• This pigeonholing doesn’t apply to the novelty, technology and utility dimensions.
Before AfterTheory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Diagnosing your research paradigmH&H p.73 (A if you agree/D if you disagree)
1. Quantitative data is more scientific than qualitative data2. It is important to state the hypotheses before data
collection3. Surveys are probably the best way to investigate business
issues4. Unless a phenomenon can be measured reliably, it cannot
be investigated5. A good knowledge of statistics is essential for all
approaches to business research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Diagnosing your Research Paradigm H&H p.73
6. Case studies should only be used as a pilot project before the main research is conducted
7. Using participant observation to collect data is of little value in business research
8. Laboratory experiments should be used more widely in business research
9. It is impossible to generate theories during the course of research into business issues
10. Researchers must remain objective and independent from the phenomena they are studying
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Diagnosing your Research ParadigmH&H p.73
To score, count the number of As and Ds:• More As than Ds – Positivist• More Ds than As –
phenomenological/Interpretivist/Subjectivist• Equal – flexible (Post-Positivist)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Science and Scientific Method
Science “the methodological and systematic approach to acquisition of new knowledge” (Geoffrey Marcyzk, David DeMatteo, David Festinger, Essentials of Research
Design and Methodology, John Wiley & Sons, 2005, p. 4)
Scientific method, evolved since 13th century, concerns set of tools, techniques and procedures used by researchers to analyze and understand phenomena and support or discard prior conceptions
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Essence of the Scientific Method
Characteristics of the Scientific Method
ObjectivitySystematic Analysis
Logical Interpretation of Results
Elements of the Scientific Method
Empirical ApproachObservations
QuestionsHypotheses
ExperimentsAnalysis
ConclusionReplication
Basic Research
Applied Research
Scie
ntif
icM
etho
d
Information or Ideas for alternative
Courses of action
General Laws
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
4 Scientific Argument Types
1. Deduction: Conclusion is drawn from a set of propositions (pure logic)
2. Induction: One draws general conclusions from particular facts that appear to serve as evidence
3. Probability: Passes from frequencies within a known domain to conclusions of stated likelihood,
4. Statistical: On the average, a certain percentage of a set of entities will satisfy the stated conditions.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
MathematicsComputer
Simulations
Logical/RationalThought
Laws of Nature
Deductive
4 Scientific Arguments types
Temporal DataSpectral Data
Images
Correlations/Patterns
Rules of Nature
InductiveTemporal Data
Likelihood
Generalities of Nature
ProbabilisticSets of Data
Trends
Predictions of Nature
Statistical
The fact that scientific reasoning is so often successful is a remarkable property of the Universe, the dependability of Nature.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Inductivisim vs. Deductivism
• Exploratory• Starts by observing,
ends with a theory• May be necessary to
uncover relationships when little is known about a phenomena (i.e., AIDS in 1980’s)
• Confirmatory• Starts with a theory,
ends with test results• Is considered the gold
standard for conducting scientific research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Adapted from, Kuhn, Thomas (1961) "The Function of Measurement in Modern Physical Science." in The Essential Tension. (1977) Chicago, IL: University of Chicago Press, pp. 178-224.
Theories
Results
Theory Experiment
1.414 1.418
1.732 1.725
2.236 2.237
ManipulationLogic and Math
(X) f1 (X)
(X) f2 (X)
…………
(X) fn (X)
Quantitative
Confirmation
“Deductive”
Qualitative
Exploration
“Inductive”
Reasonable Agreement
Qualitative vs. Quantitative
Observations
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Deductive Reasoning
Theory
Hypotheses
Observation
Confirmationand finally check to see if
the data confirms (supports) our hypotheses and theory or not
(Is our theory valid or not?)
Deductive reasoning starts with a Given Theory
as the basis for which we develop Hypotheses
and then acquire Specific Data through Observation or Experimentation
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Inductive Reasoning
Observation
Pattern
Tentative Hypothesis
Theory As foundationfor a Theory
Inductive reasoning starts with a Specific Observation
as the basis for which we develop a General Pattern
and Tentative Hypothesis
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Flow of Research:Top to Bottom Approach
Source: http://trochim.human.cornell.edu/kb/strucres.htm
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
• To support these methods, a scientist also uses a large amount of skepticism to search for any fallacies in hypothesis or scientific arguments.
• Note that there is an emphasis on falsification, not verification.
• If a theory passes any test then our confidence in the theory is reinforced, but it is never proven correct in a mathematical sense.
• Thus, a powerful hypothesis is one that is highly vulnerable to falsification and that can be tested in many ways.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Falsification• For a theory to be scientific, its hypotheses must be
falsifiable– The possibility must exist for the data to prove you wrong
• When collecting data one must not collect data simply to support one’s hypothesis– This is essentially what an inductive approach does, as the
hypothesis is based on the data
• Difference between science and philosophy / religion
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Pluto…a planet or not?
In a move that's already generating controversy and will force textbooks to be rewritten, Pluto will now be dubbed a dwarf planet. But it's no longer part of an exclusive club, since there are more than 40 of these dwarfs
A clear majority of researchers voted for the new definition at a meeting of the International Astronomical Union (IAU) in Prague, in the Czech Republic. The IAU decides the official names of all celestial bodies. The tough decision comes after a multiyear search for a scientific definition of the word "planet." The term never had an official meaning before.
What Is a Planet Today? According to the new definition, a full-fledged planet is an object that orbits the sun and is large enough to have become round due to the force of its own gravity. In addition, a planet has to dominate the neighborhood around its orbit. Pluto has been demoted because it does not dominate its neighborhood. Charon, its large "moon," is only about half the size of Pluto, while all the true planets are far larger than their moons.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Tying it all Together
• Use theory to develop research questions• Formulate specific, empirical hypotheses
that are falsifiable• Select variables based on theory that are
empirically measurable• Try to prove your hypothesis wrong
– Test the falsification (null) hypothesis
• State the limitations of your research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory and Research
What is Theory?
“…the language that allows us to move from observation to observation and to make sense of similarities and differences.”
Rudestam & Newton, 2001, p. 10.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory and Research
Relationship between theory and research
Consider the Research Process Wheel as proposed by Rudestam & Newton in Surviving Your Dissertation (2001).
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Deduction
Induction
Nomothetic
Idiographic
Hypothesis Testing
Empirical Generalization
TheoreticalGeneral/Abstract
Specific/Concrete Empirical
VerifyConfirm
EvaluateFalsify
Generalized Wheel of Science
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Information Systems Research & Practice
Theory:ideas
Practice:use of ideas
Leads to
Leads to
after Checkland & Holwell (1998)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
REALLY What is a Theory? (1)
Zikmund (p. 41) has defined a theory as “a coherent set of general
propositions, used as principles of explanation of the amount of the
apparent relationships of certain observed phenomona”
Concepts (or constructs) are the basic building blocks of theory
development. A concept (or construct) is a generalized idea about a
class of objects, attributes, occurrences, or processes that have been
given a name. A concept (or construct) may vary in terms of the
level of abstraction THEY ARE PART OF THEORY
Examples: Productivity, Leadership, Morale, Assets, Inflation
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is a Theory? (2)
A Proposition is a statement concerned with the relationship between concepts. It asserts a universal connection and logical linkage between concepts. Propositions are at a higher level of abstraction than concepts. THEY ARE PART OF THEORY
Example: Smoking is injurious to health
Hypotheses are propositions which are empirically testable. They are usually concerned with the relationships between variables. THEY ARE NOT PART OF A THEORY.
Example: Increasing salary by 10% will double the production
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory Construction
• Baconian Inductivism (Interpretivist)– Start with what we observe (data)– Look for patterns among data– Create theory based on observed patterns– Empirically test predictions of the theory
• Problems– Not all phenomena can be observed– Depends on large number of observations
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory Construction, continued
• Hypothetico – Deductivism (Positivist)– Propose theory
• Either a pre-existing theory or one that logically makes sense
– Generate testable hypothesis– Collect empirical data– Test hypothesis, interpret results
• Problems– As theories become outdated, knowledge derived from this
method may become meaningless
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Abstraction Ladder
Observations of Objects, Events and Occurrences (Reality)
Concepts / Constructs
Propositions
Theory
Leve
ls o
f Abs
trac
tion
Empirical Level
Abstract Level
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory and Research
Theory functions three ways in research:1. Theories prevent our being taken in by flukes.2. Theories make sense of observed patterns in ways
that can suggest other possibilities. 3. Theories can direct research efforts, pointing toward
likely discoveries through empirical observation.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory as Explanation
• Research questions call for explanations
• Answers or Explanations come from theorie
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Vocabulary
Concept:
“a word or a symbol to represent an idea”
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Vocabulary
Theory:
“concepts and their interrelationships”
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Vocabulary
Model:
“imitation of an existing object”
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Logics
{Good, Fair, Poor}
Approximation Denotation
{True, False}(supported or not)
World Model Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Vocabulary
Hypothesis:
“testable statement based on theory”
Prediction about what Patterns we will see in the world if our theory is correct
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Vocabulary
Operational Definition: (variable)
“concept at a level that is testable”
(measurable)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
concept
concept concept
concept
conceptconcept
conceptconcept
concept
THEORY
Hypotheses
Operational Definition(variables)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
TheoryBased on well established facts, testable hypotheses are formed.
The process of testing "leads scientists to accord a special dignity to those hypotheses that accumulate substantial observational or experimental support."
This "special dignity" is denoted by the granting of the title "theory," which, when it "explains a large and diverse body of facts" is considered "robust" and if it "consistently predicts new phenomena that are subsequently observed," it is "reliable."
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What Theory is NOT
1. References are not theory.
2. Data are not theory
3. Lists of Variables or Constructs are Not Theory
4. Diagrams are Not Theory
5. Hypothesis (or predictions) are not theory
6. Theory is not something one "adds" to data, or something that one transforms from weaker to stronger by means of graphics or references, or can be feigned by flashy conceptual performance.
Sutton, R. I. and Staw, B. M. (1995) What theory is not. ASQ 40:371-384.Weick, Karl, E. What Theory is Not, Theorizing Is, ASQ, 1995, 40:385-390.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
TEN MYTHS OF SCIENCE: REEXAMINING WHAT WE THINK WE KNOW
Myth 1: Hypotheses Become Theories Which Become Laws
Myth 2: A Hypothesis is an Educated Guess
Myth 3: A General and Universal Scientific Method Exists
Myth 4: Evidence Accumulated Carefully Will Result in Sure Knowledge
Myth 5: Science and its Methods Provide Absolute Proof
Myth 6: Science Is Procedural More Than Creative
Myth 7: Science and its Methods Can Answer All Questions
Myth 8. Scientists are Particularly Objective
Myth 9: Experiments are the Principle Route to Scientific Knowledge
Myth 10: All Work in Science is Reviewed to Keep the Process Honest.
See: http://amasci.com/miscon/myths10.html
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
“I grant you evolution was a theory to begin with… but it evolved into a fact a long time ago!“
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory
Observations
Develop HypothesisTo Explain Observations
Test Hypothesis
Hypothesis
Test Theory
LawTheory
Fail
FailPass
Pass
Pass Many Tests
Pass Many Tests
Fail
Hypotheses; Theories;Laws
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory
Hypotheses
Phenomenon of Interest?
Small dots representobservations
Large dots representExperimental Results
Scientific Theories are relatively large, general concepts.Scientific Laws are smaller, mathematically precise concepts.
ScientificLaw
ScientificLaw
ScientificLaw
Collections of Data Observations by other scientists
Inductive Reasoning applied(Specific to General) to developGeneral hypotheses
Experiments designed Through deductive reasoning(General to Specific)
Controlled experiments providenew data that is tested statisticallyfor significance and falsification
Results of statistical tests on new dataAdd evidence to support, modify or falsifythe theory (or more rarely the scientific law)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory C
ScientificLaw
ScientificLaw
Theory B
Theory A
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory
• A Causal Model of the phenomenon-of-interest• Drives all subsequent Scientific Activity
• Hypotheses• Experimental/Research Design• Measures• Analysis• Conclusions• Interpretations• Limitations
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Anything Missing?
TruthTruth
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Positivist Perspective
Science = Useful
Science <> True
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A
Useful Model
is often
Better Than Truth
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
• No scientific inquiry is ever complete, and no scientific theory is ever “final”
• Nor need it be to be useful– A scientific theory in its current state can be very
useful in the present even though it may later be or improved upon or even superceded
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Connecting outputs to outcomes is a challenge
“I think you should be more explicit here in
Step Two.”My Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Useful Is Better Than True
Useful Is Better Than True
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Name the PhenomenonName the Phenomenon
NickeziteBlock
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Describe the PhenomenonDescribe the Phenomenon
NickeziteBlock
A
B
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Explore the PhenomenonExplore the Phenomenon
BobeziteBlock
A
B
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Explore the PhenomenonA
B
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
NickeziteBlock
A
Describe Phenomenon DynamicsDescribe Phenomenon Dynamics
B
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Useful TheoryA Useful TheoryOne Gear
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Truth(Reality)Truth
(Reality)
Many GearsBelts and pulleys
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The model becomes usefulwhen you want to do something new
The model becomes usefulwhen you want to do something new
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Therefore
For matters of cause-and-effect
A useful model (Theory)
Is/Can Be better
than Truth (Reality)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is a Theory?“A set of interrelated constructs (variables), definitions, and
propositions that presents a systematic view of phenomena by specifying relations among constructs, with the purpose of explaining natural phenomena.”
Kerlinger, F. N. (1979) Behavioral Research: A conceptual Approach. New York, NY, USA: Holt, Rinehart and Winston.
To this add “Theoretical Rationale” “Specifying how and why the constructs and
relational statement are interrelated.”Labovitz and Hagedorn (1971) Introduction to Social Research. New York, NY, USA.
McGraw-Hill.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Good Theory
• Should explain existing data• Explain a range of related observations• Allow statements to be made about the world• Allow predictions about the future• Have meaningful implications
Taken from Davey et al. (2004)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
What is a Theory?
• Causal Model• Internally Consistent • Explains and/or predicts• Proposes mechanisms of causation• Testable
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Structure of a Theory
• Axioms (Assumptions)• Propositions (Causes and Effects)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Substruction• A strategy to help you understand the theory and
methods (operational system) in a research study• Applies to empirical, quantitative research studies• There is no word, Substruction, in the dictionary.
It has an inductive meaning, constructing and a deductive meaning, deconstructing
• Heuristic
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
SubstructionTheory(Theoretical system)
Construct
Concept
Deductive
(Qualitative)
Methods(Operational System)
Measures
Scaling/Data
analysis
(Quantitative)
Inductive
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Substruction: Building Blocks or Statements of Relationships
Construct Pain
Axiom ConstructQuality of Life
ConceptIntensity
Proposition ConceptFunctional status
Measure10 cm scale
Hypothesis Measuremobility scale
Theoretical Model
Measurement Model
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Statements of RelationshipsConstruct:
Postulate:Statement of relationship between a construct and concepts
Pain consists of three concepts
Concepts:IntensityLocationDuration
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Basic Concepts• Hypothesis
– States a relationship between two, or more, concepts and suggests that one has an impact on the other (Grix 2004:42)
– An Hypothesis is a provisional idea whose merit is to be evaluated.
A hypothesis requires more work by the researcher in order to either confirm or disprove it. In the hypothetico-deductive method, a hypothesis should be falsifiable, meaning that it is possible that it be shown false, usually by observation.
Note that, if confirmed, the hypothesis is not necessarily proven, but remains provisional.
(Wikipedia)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Propositions
• Functional Statements of cause-and-effect that must be logically true if the axioms are true
• Examples– P1: Effort toward group goal is a function of
goal congruence– P2. Group Productivity is an inverse function
of distraction
Basic Concepts
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Propositions must be...
• Causal• Composed of Constructs• Without empirical content• Logically derivable from axioms
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Propositions of Direct Causation
Proposition 1: Productivity is a function of effort
Proposition 2: Effort is a function of goal congruence
Proposition 3: Effort is an inverse function of distraction
ProductivityEffort
Distraction
Goal Congruence
+
-
+12
3
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory Explication
• Example: What determines musical Taste?
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Important Terms• Theory• Concept• Variable
– Independent– Dependent– Antecedent– Intervening– Mediating
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Independent Concept (Construct)
Dependent Concept (Construct)
The Simplest Diagram of a Theory
Relationship
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Simple Theory
Peer GroupMusical Taste
Influences
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Adding an Antecedent Construct
Socioeconomic status
Affects
Peer GroupMusical TasteInfluences
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Socioeconomic status
Affects
Peer GroupMusical Taste
Influences
Adding a Mediating Construct
Gender
Impacts
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Adding an Intervening Construct
Influences
Self image
Leads to
Socioeconomic status
Affects
Peer Group Musical Taste
Gender
Impacts
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Peer group Musical taste
A Third Construct Explanation
Parental influence
X
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Deriving Theory and Hypotheses
Derived in part from Bacharach (1989).
Constructs Constructs
Variables
Axioms and Postulates
Variables
LogicalFoundation forPropositions
Propositions
Hypotheses
Theory
Empirical test of Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
TeamAutonomy
culture
IndividualUsage
Axiom 1 (-)
Degree ofFreedom
Independence
Discretion
Usage Intensity
Usage Scope
P1 (-)
P2 (-)
P4 (-)
P3(-)
P5 (-)
P6 (-)
Frequency of use
Duration of use
Percentage of system features used regularly
Proportion of use
“Eight-item scale adapted from Langfred” Score
H1a (-)H1b (-)
H1c (-)
H1d (-)
TheoreticalModel
Measurement Model(Empirical test of Theory)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
SharedCognition
HeedfulInterrelating
Group Awareness P1 (+)
P2 (+) P4 (+)
P3(+/-)
Implicit Coordination Theory (ICT) Causal Process Model
TaskPerformance
(-)
Parallelism ResponseBias
P6(+)
A4(+)
A1(+)
A3(+)
A2(+)
P7(+)
Measurement Model : Independent Variables
False Alarm Rate(Non-Errors identified as errors/Total Non-Errors)
Experimental Control : Dependent Variable
Discriminability(Sensitivity)
Response Criterion
ImplicitCoordination
Hit Rate(Detected Errors/Total Errors)
Theoretical AssumptionsSupported Constructs, Postulates and Axioms
Theoretical ProposalsProposed Constructs and Propositions
SharedInterface
Self-ScribingAbility
H1(+); H5(+)
H2(-); H6(-)
H3(+); H7(+)
H4(+); H8a(+)
SDTGivens
FormalizedGroupMemory
P5 (+)
(-)
166
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Qualities of a Good Theory
• Parsimony ( simple, small )• Explanatory/Predictive• Bounded
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
“If you can't explain it simply, you don't understand it well enough.”
Albert Einstein
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Pragmatic Theory• Usually start with propositions and
work backward to axioms
• Usually start poorly and get better
• Use someone else’s theory whenever you can
• Technology has No Place in your theory(if you tie technology to your theory, what will happen when technology changes?)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Pragmatic Theory
• A good theory will get you to the moon and back safely on the first try
• Good theory will do more to save you from drawing “bone-headed” conclusions than any other discipline of positivism
• Good theory will make you look like a genius
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Explanation in ScienceCAUSAL EXPLANATION1. Common variation
The cause X and effect Y should vary together2. Order X precedes Y3. Third Factors The common variation of X and Y
should not be due to a third factor Z4. Empirical Connection
The connection between X and Y is empirical5. Theory The connection between X and Y
should be deduced from a general theory6. Mechanism
The mechanism that connects X to Y should be known
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Scientific Method (Deductive)
Phenomenon Observe
Record
Analysis(Previous Theory)
Synthesize(Cause -> Effect)
Peculiar? Hypotheses
Construct Theory
Unify/Simplify
Understand Underlying Domain
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
TheoryInterest Idea
Conceptualization
Specify the meaningOf the concepts andVariables to be studied
Choice of Research Method
ExperimentsSurveysField StudyContent AnalysisSecondary data analysisComparativeEvaluationDesign
Population and Sampling
Whom do we want to be able to draw conclusions about?Who will be observed for that purpose
Operationalization
How will we actuallymeasure the variablesunder study?
Observations
Collecting data forAnalysis and interpretation
Data Processing
Transforming the datacollected into a formappropriate to manipulation and analysis
Analysis
Analyzing data andDrawing conclusions
Application
Reporting results andassessing theirimplications
Scientific Method(more detail)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Cycle of Research and Theory-Building
People notice phenomena.
They gather information about the phenomena.
They build a theory which explains and predicts it.
They share their theory with other people.
The theory is strengthened.
People use the theory to write hypotheses.
Are the hypotheses supported?
The theory is weakened.
People conduct studies.
The researchers may suggest modifications.
Yes
No
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Worldview
Theoretical Framework(Conceptual Scheme, Principles, mode of representation, Template)
Empirical Methods Problem
Theoretical Methods
Bodyof Data
TheoreticalModel
Comparison
Possible Actions:• Revise Model• Reassess data• Redefine Problem• Reconsider Empirical Methods• Review Theoretical Methods• Reconstruct Framework• Rethink Worldview
Cycle of Research and Theory-Building Detailed
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
An Experiment without a theory is meaningless
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Phenomena: Large, Odd-Smelling Boxes
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Scientific Instrument: Drill
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Collecting Data Without A Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Collecting Data With a Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Physicist Uses the ‘Elephant’ Theory
+
=Fission!
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Farmer Uses the ‘Elephant’ Theory
Fertilizer!
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
There is nothingmore useful
than A Good Theory
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
So, what is “Science”?
• Theory must be founded on natural laws.• Theory must be falsifiable.• Theory must produce hypotheses that are
corroborated by evidence.• Disconfirmation is “overblown.” Most research
progresses by solving puzzles using the ideas within the hard core of a research program. – Rewards go to those who solve particularly hard puzzles.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Conditions of Science continued
• Predictions of new facts that are then corroborated by evidence is the ideal.
• Scientific revolutions or paradigm shifts are rare. Challenging or amending the hard core is not what science is usually about.
• Changing “how we think about the universe” occurs at several levels, from resolving particularly difficult puzzles to developing a new paradigm.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Lessons for Your Research• Since science is a social enterprise, your work counts
only by how it is received by the scientific community. This reception is partly subjective.
You must argue your case in the face of sometimes conflicting and ambiguous criteria.
• Even though we may agree on the conditions that make a theory “better,” we can still disagree and, therefore, argue over which particular theory best fits those conditions.
• ( and still respect each other’s work and be friends.)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Valid Scientific Arguments• You are solving a genuine and significant
puzzle within the field.• The evidence corroborates your theory and
hypotheses. – This is a question of research design. The better your
research design, the stronger your argument will be.
• Your amendment to the hard core is progressive. Resolving the puzzle uncovers additional implications that are also empirically corroborated.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Scientific Arguments continued• Since multiple theories may exist in the protective
belt or positive heuristic, your theory is more elegant, broader in the range of phenomena its predicts/explains, and supported better by the evidence than its plausible rivals.
• In rare cases, you have sufficiently altered the hard core that you have created a new research program.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
AYER - LOGICAL POSITIVISM• Theories confirmed & areas sewn up• New areas investigated• Less to investigate in each generation• End of Science!
020406080
100120
generations
theo
ries
con
firm
ed
Consequences for Information Growth
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
POPPER - FALSIFICATION• Theories not disproved• All results contingent• Each generation re-investigates results• Exponential growth of science
Consequences for Information Growth
0
20
40
60
80
generations
the
ori
es
co
nfi
rme
d
newretested
Philosophical underpinnings of science drive it forward and predispose it to exponential growth
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Merton’s Norms• Merton’s social norms of scientific conduct
– Universalism: new work is assessed by universal impersonal criteria
– Communality: scientific knowledge should be common property– Disinterestedness: prime concern is the advancement of
knowledge– Organized scepticism: knowledge should be continually subjected
to critical scrutiny
• Reflects stated values rather than actual behaviour: what they do is not what they say. See Watson’s The Double Helix, for example
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
IS Research Methods
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di VeronaNicholas C. Romano, Jr.
Oklahoma State UniversityDoctoral Seminar – MSIS 6333 Wednesday August 21st, 2009
AddressingResearch Questions(explanationor understanding)
PositivismFocus ongeneralizeability and Causal Explanation
InterpretativmFocus onrelativism and understanding
Survey
Experiment
Questionnaire
StructuredObservation
Quantitative Analysis
Case Study Discourse analysis Life History Ethnography
phenomenology
Participant Observation
FocusGroup
Interviewing
Grounded theory
ActionResearch
Hypothesis Testing(Aiming to establish, explain, Predict causal linksBetween key variables
Hermeneutic Inquiry(‘Thick description’And in-depthUnderstanding)
EpistemologicalParadigms
Methodologies/ResearchStrategies
Methods(data collectionAnd analysis)
Methodological field
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
The Research Process Onion
SamplingSecondary dataObservationInterviewsQuestionnaires
ResearchPhilosophy
Positivism
Realis
m
interpretivism
ResearchApproaches
Deductive
Inductive
ResearchStrategies
Experiment Survey
Casestudy
Groun
ded
theo
ryEthnography
Action research TimeHorizons
Cross sectionalLongitudinal
Data CollectionMethods
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Alternative Motivations for Research
• ‘Pure Research'‘because it’s there’contribute to abstract, theoretical understanding
• ‘Applied Research'‘I have hammer, so find a nail’
• Instrumentalist Research‘I see a problem, so find a solution’
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Basic (pure) AppliedPurpose
Context
Methods
• Expand Knowledge
• Academic Setting• Single Researcher• Less Time/Cost Pressure
• Internal Validity• Cause• Single Level of Analysis• Single Method• Experimental• Direct Observations
• Understand Specific Problem
• Real-World Setting• Multiple Researchers• More Time/Cost Pressure
• External Validity• Effect• Multiple Levels of Analysis• Multiple Methods• Quasi-Experimental• Indirect Observations
ResearchType
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Nature of Research Outcomes
• DescriptiveDepiction of a behavior or a domain
• ExplanatorySystemic explanation of how behaviors ariseascription of causes to occurrences in the domain
• PredictiveStatement of: what behavior will arise, and how;what occurrences will arise within the domain;what effect will particular interventions have
• NormativeDeclaration of interventions to a desired outcome
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Three Types of Research
• Descriptive research –finding out (What, Where, When)
• Explanatory research – explaining ; identifying causality; theory/model; prediction (HOW/WHY)
• Evaluative research – evaluation of strategies, policies, programs, practices (Value)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Types of Research Spectrum
To become familiar with phenomena; to gain new insights; to formulate a more specific research problem or research hypothesis.
To portray accurately the incidence, distribution, and characteristics of a group or situation. (Usually not begun with specific hypothesis.)
To investigate relationships between variables. (Begins with specific hypotheses.)
To test hypotheses of causal relationships between variables. (Begins with specific hypotheses.)
Descriptive Research
Survey Research
Independent variables (X) not controlled by investigator
Independent Variables
Correlational/Ex Post Facto
(Explore) (Describe) (Explain - Predict) (Control)
Independent Var. (X) controlled by investigator
Exp./Quasi-Exp.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research generates knowledge in order to:
Action Change Within A System
Pave The Way For Change
Build Broader Understanding
Basic Or Pure Research
Action Research Critical / Radical Ethnography
Applied / EvaluativeResearch Participatory/
EmancipatoryTechnical/ Practical
Emancipate Through Action
Expose And Change The Dominate System
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Approaches
– Two main classes of approaches:• Theory ‘testing’ – apply theory to ‘read’ the
data• Theory ‘emergent’ – look for ‘patterns’,
understanding emerges from the data
(Hirschheim, 2002)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Combining Approaches• Case study/action research
leads to• Research question
leads to• Theory building
leads to• Theory testing with lab. experiments
and• Theory testing with field experiments
leads to• Theory extension and feedback loop to Theory testing
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Approaches
– Mathematical approaches– Approaches studying reality
• Research stressing what is reality – Conceptual-analytical approaches– approaches for empirical studies
» theory-testing approaches» theory-creating approaches
• Research stressing utility of artifacts– artifact-building approaches– artifacts-evaluating approaches(Järvinen & Järvinen, 1999)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Alternative Research Methods
– Constructivist Methods (“Design”)• conceptual development and technical development
– Nomothetic Methods (“Confirmatory”)• field research, surveys, lab experiments … using the
hypothetico-deductive method
– Idiographic Methods (“Exploratory”)• case studies and action research
(Hirschheim, 2002)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Methods
• Non-Empirical Techniques• Scientific Research Techniques• Interpretivist Research Techniques• Research Techniques at the
Scientific/Interpretivist Boundary• Engineering Research Techniques
(Design Fits here as well)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Methods Non-empirical Techniques
The following techniques are detached from real-world data. This is not to say that they are necessarily totally remote or irrelevant, but rather that they are once-removed, depending on synthetic data, or on conceptual thinking about abstractions. The primary techniques are:
• Conceptual research. This is based on opinion and speculation, and comprises philosophical or 'armchair' analysis, and argumentative/dialectic analysis;
• theorem proof. This applies formal methods to mathematical abstractions, in order to demonstrate that, within a tightly defined model, a specific relationship exists among elements of that model;
• simulation. This is the study of a simplified, formal model of a complex environment, in order to perform experimentation not possible in a real-world setting;
• futures research, scenario-building, and game- or role-playing. Individuals interact in order to generate new ideas or gather new insights into relationships among variables. A specific instance that is often applied in the information systems discipline is the delphi technique (Delbecq et al., 1975);
• review of existing literature, or 'meta-analysis'. The literature examined in such research may include the opinions and speculations of theorists, the research methods adopted by empirical researchers, the reports of the outcomes of empirical research, and materials prepared for purposes other than research.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Methods Scientific Research Techniques
The following are common techniques that can be applied by information systems researchers within the scientific tradition:
• Forecasting. This technique involves the application of regression and time-series techniques, in order to extrapolate trends from past data;
• Field experimentation and quasi-experimental designs. Opportunities are sought in the real-world which enable many factors, which would otherwise confound the results, to be isolated, or controlled for (Cook & Campbell 1979);
• Laboratory experimentation. This involves the creation of an artificial environment, in order to isolate and control for potentially confounding variables (Hersen & Barlow 1976, Jarvenpaa et al. 1984, 1985, Jarvenpaa 1988, Benbasat 1990a, 1990b, DeSanctis 1990).
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Methods Interpretivist Research Techniques
The following are techniques which are unequivocally interpretivist in their style: • descriptive/interpretive research. In this techniques, empirical observation is
subjected to limited formal rigour. Controls over the researcher's intuition include self-examination of the researcher's own pre-suppositions and biases, cycles of additional data collection and analysis, and peer review;
• focus group research. This involves the gathering of a group of people, commonly members of the public affected by a technology or application, to discuss a topic. Its purpose is to surface aspects, impacts and implications that are of concern. See Stewart & Shamdasani (1990) and Clarke (1999);
• action research. The researcher plays an active role in the object of study, e.g. by acting as a change-agent in relation to the process being researched. See Clark (1972), Susman & Evered (1978), Mansell (1991), Stringer (1996, 1999), Myers (1997a) and Baskerville & Wood-Harper (1998);
• ethnographic research. This technique applies insights from social and cultural anthropology to the direct observation of behaviour. See Harvey & Myers (1995) and Myers (1997a);
• grounded theory. This is a specific technique that it is claimed enables the disciplined extraction of a theory-based description of behaviour, based on empirical observations. See Glaser & Strauss (1967), Strauss & Corbin (1990) and Myers (1997a).
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research MethodsResearch Techniques at the Scientific/Interpretivist Boundary
Several techniques can be applied within either a scientific or an interpretivist context.
• field study. The object of study is subjected to direct observation by the researcher (Klein & Myers 1999);
• questionnaire-based survey. This involves the collection of written data from interviewees, or the collection of verbal responses to relatively structured questions. See Straub (1989), Kraemer (1991), Kraemer & Dutton (1991), Pinsonneault & Kraemer (1993), and Newsted et al. (1998);
• interview-based survey. This involves the recording of verbal data from interviewees, which arises in relatively unstructured interviews or meetings;
• case study. This involves the collection of considerable detail, from multiple sources, about a particular, contemporary phenomenon within its real-world setting. For guidance on the use of case studies within the scientific tradition, see Yin (1984, 1994), Benbasat et al. (1987) and Lee (1989); and for guidance on their use in an interpretivist manner, see Walsham (1995b) and Myers (1997b);
• secondary research. Rather than producing new data, this technique analyses the contents of existing documents. Commonly, this is data gathered by one or more prior researchers, and it is re-examined in the light of a different theoretical framework from that previously used. The documents may also include materials prepared for purposes other than scientific research.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research MethodsEngineering Research Techniques
Information systems research conducted within the computer science and engineering context uses two categories of research technique:
• construction. This approach involves the conception, design and creation (or 'prototyping') of an information technology artefact and/or technique (most commonly a computer program, but sometimes a physical device or a method). The new technology is designed to intervene in some setting, or to enable some function to be performed, or some aim to be realised. The design is usually based upon a body of theory, and the technology is usually subjected to some form of testing, in order to establish the extent to which it (and, by implication, the class of technologies to which it belongs) achieves its aims;
• destruction. In this case, new information is generated concerning the characteristics of an existing class of technologies. This is typically achieved through testing the technology, or applying it in new ways. The design is usually based upon a body of theory.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Types and MethodsType/ Method Tests,
MeasurementsInterviews Observations Surveys Documents
Experimental P A AQuasi-experimental P A ACausal-comparative P A ACorrelational P A ADescriptive A A P AEvaluation P A A A AEthnographic A P AAction A P ACase study A P A A
P = primary method used; A = additional method that may be used.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Object/Level
of Analysis
TheoremProof
LaboratoryExperiment
FieldExperiment
CaseStudy
SurveyForecasting and Future Research
Simulation and Game/Role
Playing
Subjective/ Argumentative
Descriptive/ Interpretive
ActionResearch
Society
Organization/Group
Individual
Technology
Methodology
TheoryBuilding
TheoryTesting
TheoryExtension
Taxonomy of Research Methods and Appropriate Objects/levels of Analysis
No
No
No
No
No
No
No
NoPossibly (Small Groups)
Yes
Modes for Traditional Empirical Approaches (Observations)
Modes for Newer Approaches (Interpretations)
Yes Yes Yes
YesYes
Yes
Yes
No
PossiblyPossibly Possibly Possibly Possibly
Possibly
Possibly
Possibly
Possibly
Possibly
PossiblyPossibly
PossiblyPossibly
Yes Yes
Yes
Yes Yes
Yes
Yes
Yes Yes
No
Yes
Yes
Possibly
Possibly
Possibly
No
Yes Yes Yes Yes
Yes
Yes
Yes Yes
Yes
Yes
Yes YesYes Yes Yes
YesYes Yes
YesYes
PossiblyPossibly
PossiblyPossiblyPossibly
Possibly
Possibly
No No
NoNo No
(Galliers 1990)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
International IS Research Methods
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Design Science IS Research Framework
• Information Systems (IS) are complex, artificial, and purposefully designed.
• IS are composed of people, structures, technologies, and work systems.
• Two Basic IS Research Paradigms– Behavioral Research – Goal is Knowledge– Design Research – Goal is Utility
Source Al Hevner
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
IS Research Cycle
Adapted from Hevner
DesignScience
Research
BehavioralScience
Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Theory BuildingConceptual frameworksMathematical models
Methods
ObservationCase studies
Survey studiesField studies
SystemsDevelopment
PrototypingProduct developmentTechnology transfer
ExperimentationComputer simulations
Field experimentsLab experiments
Adapted from Nunamaker, Chen and PurdenJMIS (1991) 7(3).
Systems Development in Information Systems Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Arizona SW Engineering Methodology
THEORY
CONCEPT
MODEL
PROTOTYPE
ExperimentationOBSERVATION
FIELD STUDY
Adapted from Nunamaker
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Arizona SW Engineering Research Cycle
Theory, Concept, Model
Prototypes
Field and Lab Research
Product
Adapted from Nunamaker
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Lab & Field Study Objectives
• Develop new Process and Tool Uses
• Develop Metrics for Process and Tool Use
• Evaluate usefulness of new Processes and Tools
• Identify Process and Tool Improvements
• Confirm Lab Results in the Field
• Gain best Practice from Lab and Field UseSource Nunamaker
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Research Road Map Through the Last Research Mile
Identify a Real Problem
1. Proof-of-Concept Prototype
2. Proof-of-Value Prototype
3. Proof of Self-Sustaining Use(Production System)
Travel the Last Mile
Real Problem
1.POC
2.POV
3.POU
Adapted from Nunamaker
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Design Science• Design is a Artifact (Noun)
– Constructs– Models– Methods– Instantiations
• Design is a Process (Verb)– Build– Evaluate
• Design is a Wicked Problem– Unstable Requirements and Constraints– Complex Interactions among Subcomponents of Problem and resulting
Subcomponents of Solution– Inherent Flexibility to Change Artifacts and Processes– Dependence on Human Cognitive Abilities - Creativity– Dependence on Human Social Abilities - Teamwork
Source Al Hevner
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
• Constructs• Models• Methods• Instantiations
Algorithms and Practices that define processes and provide guidance on howto solve problems, that is, how to search thesolution space.
Implemented and Prototype systemsthat show that constructs, models, or methods can be implemented in a working system.They demonstrate feasibility, enabling concrete assessment of an artifact’s suitability to its intended purpose
Vocabulary and Symbols that provide the language in which problems and solutions are defined and communicated.
IS Design theories seek to prescribe effective development practices (methods) and a typeof system solution. (instantiation) for a particular class of user requirements. (models)
Knowledge BaseAbstractions and Representations that use constructs to represent a real world situation-the design problem and its solution space
Foundations• Theories• Frameworks• Instruments• Constructs• Models• Methods• Instantiations
Methodologies• Data Analysis• Techniques• Formalisms• Measures• Validation Criteria
Rigor
Design Science IS Research Framework (Hevner et. al., MISQ, 2004) USA
Develop/Build
IS Research
Justify/Evaluate
Assess Refine
• Theories• Artifacts
• Analytical• Case Study• Experimental• Field Study• Simulation
ApplicableKnowledge
EnvironmentPeople• Roles• Capabilities• Characteristics
Organizations• Strategies• Structure and Culture• Process
Technology• Infrastructure• Applications• Communications Architecture• Development Capabilities
Additions to theKnowledge base
Achieved by appropriately applying existing foundations and methodologies.
BusinessNeeds
RelevanceImplementable, synthesize an existing body of research, [or] stimulate critical thinking. among IS practitioners.
Application in theAppropriate Environment
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Three Cycles of DS Research
Environment Knowledge BaseDesign Science
Build Design Artifacts & Processes
Evaluate
Design Cycle
Application Domain
• People
• Organizational Systems
• Technical Systems
• Problems & Opportunities
Relevance Cycle
• Requirements
• Field Testing
Rigor Cycle
• Grounding
• Additions to KB
Foundations
• Scientific Theories & Methods
• Experience & Expertise
• Meta-Artifacts (Design Products & Design Processes)
Adapted from Hevner
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
SoftwareEngineering
UserInterface
Design
ReferenceModeling
MethodEngineering
….
Consortial Research Method (St Gallen Switzerland)
ResearchOutline: Need
Gap, Goal
ConsortiumAgreement
ResearchPlan
State ofInstantiations
State ofModels &Methods
State ofTheories &Constructs
Domain
Analysis
Design
Evaluation
Diffusion
ScientificPublication
PractitionerPublication
TeachingMaterials
Roll-outPlan
ReviewWorkshop
FunctionTest
Experiment Simulation PilotApplication
Scientific Knowledge• Instantiations• Models• Methods• Theories•Constructs
Practical Knowledge• Business Models• Processes & Structures• Information Systems• Information Technology
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Organisational Forms of Knowledge Creation
Knowledge Creation
Internal External
With ExclusiveExploitation Rights
Without ExclusiveExploitation Rights
With ExclusiveExploitation Rights
Without ExclusiveExploitation Rights
IndustrialResearch
1Collaborative Research
2
With Customers
Or Suppliers
2a With NeutralPartners
2bWith
Competitors
2cContractResearch
3
(adapted from Brockhoff 1999)
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
226
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
A Research Framework for the Organizational laboratory
Prediction
Change
Adapted from Braa and Vidgen =1997)
The Points representIntended Research Outcomes:
Understanding
Points are Ideal Types; That is Weberian abstractionsThat are not attainable in practice.Research Praxis is represented by the constrained space of the triangle. intervention
Reduction Interpretation
The framework is represented by a triangle
Dotted lines inside the triangle represent research dynamics as movementstowards (and away from) the ideal types.
Prediction is the outcome of positivist modes of enquiry; although a ‘good’ theory does indeed have explanatory power, the more significant outcome of positivist theories is the ability to control and predict.
Understanding is the outcome of Interpretivist modes of enquiry; successful Interpretations bring out insider rationalityand promote understanding.
Change is the outcome of interventionist modes of enquiry; successful interventions lead to improvements in the problem situation.One implication for research praxis is that
all three dynamics (reduction, interpretation, and intervention) are, regardless of the research method adopted, co-present, albeit with differing mixes and emphasis.
As the researcher moves towards the prediction point through a process of formalized reduction there should be greater explanatory and predictive power. The traditional approach to explanation and prediction is experimental method.
Movement toward the understanding point through a process of interpretation is associated with greater richness of insight into the role of IS in organizational settings. Understanding is achieved typically through case studies informed by schools of sociological thought such as phenomenology, hermeneutics and ethnography.
Movement toward the change point isachieved through a process of intervention as typified by action research.
For example: interpretivist research methods involve a reducing of the infinite range of factors that might be considered relevant to a particular inquiry, although such a ‘reduction’ is not rationalized through the application of the systematic procedures of positivism.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Qua
si e
xper
imen
t
Field Experiment
Mapping Research Methods into the Framework
Prediction(Reduction)
Understanding(Interpretation)
Change(Intervention)
Adapted from Braa and Vidgen =1997)
Action Research
Soft CaseHard Case
Action case
An experimental design which does not meet the three criteria of multiple treatments (or one treatment and a control group), randomization, and experimental control; but rather attempts to preserve as many of the properties of true experimentation as possible, given the constraints of the research setting.
These are much more common than ‘True’ experiments.
Action research is a way of building theory and descriptions within the context of practice itself. Theories are tested through intervention in the organizational laboratory, that is, through experiments that bear the double burden of testing hypotheses and effecting some desirable change in the situation. Action research comes in many flavours, ranging from formal approaches through to less formalized, more reflective and personal approaches.
an extension of laboratory experiments into an organizational context. There are lots of factors which the researcher cannot control but which could affect the outcomes.
Field experiments aim at controlling a small number of variables which may then be studied intensively. A major advantage is that the experiment is conducted in a real-life setting. A major problem however is the difficulty of finding organizations prepared to be experimented on.
Experimental control is essential and involves taking appropriate steps to eliminate ‘nuisance’ variables, which are factors other than the independent variables that might be responsible for observed changes in the dependent variable.
‘true’ experimental design which meets the criteria of multiple treatments (or one treatment and a control group), randomization, and experimental control;
Empirical inquiry that ‘investigates a contem porary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident’.
Can be used in three modes: explanatory, descriptive, and exploratory .
Applicable where the research question is of a ‘how’ or ‘why’ nature, where control over behavioural events is not needed, and where there is a focus on contemporary events.
Allow reality to be captured in detail and many variables to be analyzed;
from a positivist stance, problems with case studies include the difficulty of generalization, lack of control over variables, and different interpretations by different stakeholders
A hybrid between interpretation and intervention. A trade-off between being an observer who can make interpretations (understanding) and a researcher involved in creating change in practice. when doing case studies researchers contribute to change by questioning events and applying new concepts. On the other hand, full-scale action research projects are often not appropriate due to organizational constraints or the nature of the topic to be inves tigated. Small scale intervention with a deep contextual understanding is one way of balancing this dilemma.
Interpretivist approach is concerned with gaining understanding; generalization is the movement from a concrete situation to the social totality beyond the individual case.
A soft case study based on ethnographic methods can involve a variety of data collection techniques, such as videotape, and data analy sis might involve, for example, techniques from grounded theory
From an interpretive position, the validity of an extrapolation from an individual case or cases depends not on the representativeness of such cases in a statistical sense, but on the plausibility and cogency of the logical reasoning used in describing the results from the cases, and in drawing conclusions from them.
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Adapted from Baskerville and Stage
DIAGNOSINGIdentifying or
Defining a problem
ACTION PLANNINGConsidering alterative
Courses of ActionTo Solve a Problem
ACTION TAKINGSelecting a
Course of Action
EVALUATINGStudying Consequences
Of an Action
SPECIFIC LEARNINGIdentification ofGeneral Findings
Developmentof a Client-
SystemInfrastructure
Cycle Process of Action Research
Nicholas C. Romano, Jr.IS Research an International Perspective
21 Maggio 2010Università degli Studi di Verona
Questions?