MSR- Methodologies for Scientific Research
1. Methods and Techniques for scientific Research A possible vision about Theories on the development and characterization of Scientific Research
http://paginas.fe.up.pt/~eol/PRODEI/mic1718_i.htm
Eugénio Oliveira- 2017/2018
Eugénio Oliveira MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
1. Abstract of my Lectures
Understanding about Science Scientific Methods and Techniques Writing scientific papers Evaluating scientific papers Consulting scientific Data Bases and searching for information ( FEUP’s Library)
2. Abstract of your practical Work
To review and criticize one selected scientific paper All To organize a workshop (DSIE) To apply scientific methods (reasoning, developing, writing) To write your own scientific paper To orally present a scientific paper
Eugénio Oliveira MSR- Methodologies for Scientific Research
1.1 Basic Motivations: Issues on Scientific Methods
Eugénio Oliveira MSR- Methodologies for Scientific Research
A Methodology is the study of the Methods, the philosophical rational or the set of procedures applicable in some area under observation
A Method is the systematization of a process and steps leading to certain objectives
Eugénio Oliveira MSR- Methodologies for Scientific Research
Source: GETA and Graduate School of Electrical and Communications Engineering course: Introduction to Research Methodology, Aarne Mammela, VTT- Helsinki University of Technology,
Eugénio Oliveira MSR- Methodologies for Scientific Research
Source: http://www.experiment-resources.com/
Eugénio Oliveira MSR- Methodologies for Scientific Research
What is your main motivation. What are your current (and future) interests? WRITE THESIS Collect information on the topic and identify an existing problem. Hypothesis Read all about it … proved? chartmethod002.jpg Discuss possibilities for Make your work generalizations Hypothesis shall be Use statistics. complemented by Make Research Questions to be comparisons. answered in the end Answer RQ Computing System?! Real Which Data? world or simulation? Which Metrics? System architecture
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- T.Kuhn
• Basic Characteristics: • Cumulative versus reformulating always from the principles • However: •Thomas Kuhn (Harvard, Berkley, Princeton, MIT until 91. +96) “The Structure of Scientific Revolutions” 1962.
• He points out the role in scientific research of the emergence of NEW PARADIGMS.
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- T.Kuhn
What is your interpretation of the figure? It depends on your mental, apriori perspective….
• He points out the role in scientific research of the emergence of NEW PARADIGMS (new perspectives…).
•A PARADIGM is defined as: Scientific discoveries, which universally recognized during a certain period of time, create a new model for both problems and their respective solutions to be used by a community of practicioners and experimentalists
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research
• JJ Thompson Model “Plum pudding model”
• Bohr model, introduced by Niels Bohr in 1913, depicts the atom as a small, positively charged nucleus surrounded by electrons that travel in circular orbits around the nucleus—similar in structure to the solar system, • Schrodinger Model. The probability function basically describes a cloud-like region where the electron is likely to be found. It can not say with any certainty, where the electron actually is at any point in time, yet can describe where it ought to be.
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- L. Wittgenstein
• Paradigms support theories based on concepts, phenomena and techniques for helping on explaining new facts or information
• Science progresses through successive ruptures depending on the current accepted paradigms.
Paradigms correspond to “different views” over the reality which are influenced by aspects other than scientific ones.
L.Wittgenstein “Tractatus Logico-Philosophicus
EugénioEugénio Oliveira Oliveira- 2014 MSR- Methodologies for Scientific Research Questioning Scientific Research- L. Wittgenstein
• Wittgenstein and the animal??????…
That’s the rabbit Mouth!!
EugénioEugénio Oliveira Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- L.Wittgenstein • Paradigms correspond to “different views” over the reality which are influenced by aspects other than scientific ones.
“ Well, God has arrived. I met him on the 5:15 train. He has a plan to stay in Cambridge permanently”. John Maynard Keynes, after meeting with Wittgenstein at his arrival in Cambridge, in a letter to his wife
“Wittgeinstein’s Poker”, David Edmonds, J. Eidinow Wittgenstein Versus Popper during "Are There Philosophical Problems?" meeting of the Moral Sciences Club
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- A. Einstein & C.
• Scientific interpretation (and perception) is mostly affected by Social as well as Psychological factors.
• Does it lead to Irrationality dominating Science? No
• A good discussion on prevalence of science can be found in: • Kuhn, T.S. “Black-Body Theory and the Quantum Discontinuity”, 1987.
However: Paradigm changes non-cumulative Science (sometimes)
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- A. Einstein & C.
• Examples of important Paradigm changes:
• Solar System Ptolemy's vision changed by Copernicus vision).
• Going from Newtonian Physics to Quantic Physics (Uncertainty Versus Determinism..) and Relativity (Space-Time relationship).
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- K.Popper
• Karl Popper (London S. E., + 94)
• “The Logic of Scientific Discover”, Basic Books, 1959 (1ªEd Ing)
• “Conjecture and Refutations”, Harper & Row, 1968 • Idealistic. He developed a Scientific Method as an Epistomology (a way of “knowing”) independently of the intrinsic value of the subject.
• T. Khun and Feyerabend (1993) criticize this point of view showing how science is guided by value and, therefore being non-idealistic.
• This is a debate on “Consciousness over Existence”
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- K. Popper
• “Critical Rationalism” proposal defends “falsifiability” as a criterion for demarcating (separating) science and non-science
Induction (might be) Baaaad!!! – Falsifiability is gooood!!!
• Popper argues that scientific theory will always be conjectural and provisional.
It is not possible to confirm veracity (the truth) of a theory by means of simply recognizing that the results of a preview already done based on that very same theory, hold.
That Theory should only be seen as a theory not yet contradicted by facts.
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research- K. Popper
A scientific hypothesis must provide a logical possibility to be refuted by a probable true observation statement.
Therefore, according to Popper, falsifiability is a required characteristic for a scientific theory.
Science evolves by shedding its falsified theories.
Popper’s falsification model is similar to the notion of "natural selection".
The best theory keeps surviving and evolving
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research • Verificationism: a sentence should be either verifiable or falsifiable • Critics addressed to Verificationism: • Generalization: specific instances lead to universal assumptions
Observation X, being an instance of Theory T Increases the probability of Theory T being correct The “Black Crow paradox”:
• The “Black Crow paradox”: by Hempel:
“All Crows are Black" is logically equivalent to “Everything which is Non-Black is not a Crow“
x(Cx → Bx) ≡ x(¬Bx → ¬Cx)
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research
• The “Black Crow paradox”: by Hempel: “All Crows are Black" is logically equivalent to “Everything which is Non-Black is not a Crow”. x(Cx → Bx) ≡ x(¬Bx → ¬Cx)
• Thus, IF x(Cx Bx) reassures that x(Cx→Bx), Then IF x(¬Bx ¬Cx) reassures that x(¬Bx→¬Cx)
which, being equivalent to x(Cx→Bx),
That implication would also see its probability of being True increased !!?
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research • Crítics addressed to Verificationism: • Generalization : specific instances lead to universal assumptions • “There are Apples (non-Crows) not Black” Increases the probability that the sentence “All Crows are Black” is True !!?
• If verificationism (generalization) was acceptable, any trivial evidence would support a Theory
“falsifiability” proposed by KP states that keeping just the Existential assertion permit us to know that if another existential proposition that contradicts it, arrises, this falsifies the Theory T !!
Eugénio Oliveira MSR- Methodologies for Scientific Research Questioning Scientific Research
Message
Always make clear all the assumptions you are using under which you believe your THESIS is correct.
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Exploration An exploration Enterprise:
Christopher Columbus – the “Explorer”
Motivation: ambiguous… Problem: a new way to “India” Hypothesis: over the Atlantic and not around South Africa Experiment: Yes he have done it !! Conclusions:: ????? (not the expected ones…)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research
The word “RESEARCH” is composed of two syllabels:
•“RE” a prefix meaning “once again” •“SEARCH” a verb meaning to examine closely, to test, to probe.
“RESEARCH” is a noun meaning a careful, systematic study of knowledge undertaken to establish facts or principles.
N. Anzari
EugénioEugénio Oliveira Oliveira MSRMIC- -Metodologias Methodologies de for Investigação Scientific Research Científica Scientific Research Exploration
research: discover new knowledge
• basic research (no specific application in mind)
• applied research (ideas into operational form)
• development: systematic use of the existing knowledge
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Object • science: organized or systematic body of knowledge • technology: application of scientific knowledge for practical ends in engineering, medicine, agriculture, etc.
Science, Technology and Engineering [Jain97]
• natural sciences and engineering sciences differ in the object of study:
• natural sciences: objects in the nature • engineering sciences : objects (products, services, methods) not found in the nature, using the results of mathematics, natural sciences,…
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific ResearchERS Verification, confirmation, and justification are synonymous terms in philosophy of science. The opposite is falsification or refutation.
R. K. Jain and H. C. Triandis, Management of Research and Development Organizations: Managing the Unmanageable. John Wiley & Sons, 1997.
Requirements for success • You have to be: • analytical and curious, • autonomous and collaborative • critical and tolerant
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific ResearchERS
How does a scientific researcher work? DAILY
• make always notes in a “notebook”, “Logbook” • make summaries about what has been learned. • make plans for the future all the time (outlines, roadmaps) • discuss, ask questions and argue (criticism)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research and Engineering
Learning carefully by induction (bottom up, generalization from examples to models) [Felder88]
• Defending theories by deduction (top down, from models to results)
Break the problem down and then compose and generalize the results (divide and conquer)
R. M. Felder and L. K. Silverman, “Learning and teaching styles in engineering education, ”Engineering Education, pp. 674681, April 1988
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research
1.2 Supporting Theories
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research
• Basic Characteristics
• Research involves Methodologie(s) and Techniques
• A- Methodology includes the study of several methods that are applicable to a class of problems • i.e., the set of processes to conduct each specific Research Project
•B- Techniques refer to the means and specific Tools enabling relevant information acquisition, the respective data analysis, as well as the inferences that can be made to get the results.
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research
• We can say that a Methodology has many dimensions and Research Methods do constitute a part of the research Methodology
•The scope of research Methodology is wider than that of Research Methods
N. Anzari 2012 • Methodology may also be the entire research process stages
• We usually need techniques to implement a Method
EugénioEugénio Oliveira Oliveira- 2014 MSRMIC- -Metodologias Methodologies de for Investigação Scientific Research Científica Scientific Research
PhD Student Examiner
Don’t put yourself in this situation …
EugénioEugénio Oliveira Oliveira- 2014 MSRMIC- -Metodologias Methodologies de for Investigação Scientific Research Científica Research Methods Dimensions
• A- Methodologies • depending on the MOTIVATION:
• ‘Pure Research’: • It contributes to a deep theoretical understanding and for a more abstract formulation of the phenomena. • ‘Instrumentalism-type Research’: • It contributes for making human intervention in Real world environments, more effective. • Two major sub-categories are identified: • ‘Applied Research’ • ‘Problem - Oriented Research’
In the philosophy of science, instrumentalism is the view that a scientific theory is a useful instrument in understanding the world. A concept or theory should be evaluated by how effectively it explains and predicts phenomena, as opposed to how accurately it describes objective reality. Instrumentalism avoids the debate between anti-realism and philosophical or scientific realism. It may be better characterized as non-realism. Instrumentalism shifts the basis of evaluation away from whether or not phenomena observed actually exist, and towards an analysis of whether the results and evaluation fit with observed phenomena.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Methods Dimensions
• ‘Apllied Research’. • Starting from a technology (devices, specific techniques, both) use them for dealing with processes (physic, organizational, social, individual,...) making things differently.
• ‘Problem- Oriented Research’. • Starting with the problem description and then looking for the adequated techniques to solve it.
Who wants to classify, along this dimension, the research (s)he is willing to pursue?
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Methods Dimensions
• Depending on the Theory underlying the Research:
• ‘Descriptive’, of all significative aspects of the domain
• ‘Explicative’, of the behaviour of all the phenomena
• ‘Predictive’ of the future phenomena behaviour
• ‘Prescriptive’, beyond the prediction it should also prescribe and apply norms and processes in identified specific circumstances
Research in Informatics, CS, IS, Electronics and Telecommunications should be of this kind To solve predicted problems in a flexible way !! Or even to deal with some unexpected situations
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Methods Dimensions
• Depending on the tradition in that scientific area or Knowledge Domain:
i) ‘Classic Scientific Research’ which is more quantitative
ii) ‘Interpretative Research’ which is more qualitative
iii) ‘Engineering-oriented Research’
• iii) most appropriated for Informatics, Electronics, Telecommunications ...
• But do not ignore the other approaches!
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
1.3 Positivist (classical) Research method
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
• Traditional/Classic Scientific Research is based on positivist rational thinking:
• From observations we may build up Theories trying to explain what has been observed
• Theories are expressed in Deductive form or using Axioms and Postulates which are then operated through Logics.
Deduction Vs Induction Vs Abduction
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
Do Not Forget :
• Scientific Theories, following K. Popper, are capable of generating Inferences which are, in principle, falsifiable through references to the real world.
• Results coming from the process of testing hypothesis give feedback to the Theory, enabling verification of its trueness or detecting possible “ab-normalities” (T.Kuhn).
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
• Whenever the Observation comes from the Real World it is called empirical and it becomes relevant in two situations:
• during the formation of the theoretical knowledge through the so- called passive observations
• during Hypothesis verification, gathering information in a more active way, although guided by that same theory.
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
• When we do not have a suitable Theory available, we have to pursue ‘exploratory research’ where experiments and interpretation are open, not guided by a Theory but making available Empirical Knowledge that may postulate other Theories.
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classic Methods
• Informatics , IS and Telecommunications are disciplines (scientific areas) without a given stable theory.
• They encompass applied subjects using a partial theory or co-opting a theory from “reference disciplines” such as:
• Mathematics • Information Theory • Physics • “Theory of Organizations”, • “Management Sciences”, • “Computer Science”. (partially…) CShannon DKnuth JVNeumann
Eugénio Oliveira MSR- Methodologies for Scientific Research RECAP Methods and Techniques for guiding Research activities
• Cumulative versus reformulating always from the principles • Emergence of NEW PARADIGMS
• “Critical Rationalism” proposal defends “falsifiability” as a criterion for demarcating science and non-science • Critics addressed to Verificationism A- Methodology B- Techniques refer to the means and specific Tools
• ‘PureX Research’ ‘
•Instrumentalist Research’: • It contributes for making human intervention in Real world environments, more effective. • ‘Applied Research’ • ‘Problem - Oriented Research’
Eugénio Oliveira MSR- Methodologies for Scientific Research •A- Methodology •‘Descriptive’, of all significative aspects of the domain • ‘Explicative’, of the behaviour of all the phenomena • ‘Predictive’ of the future phenomena behaviour • ‘Prescriptive’, apply norms and processes in pre- identified specific circumstances
• Depending on tradition in that scientific area or Knowledge Domain: i) ‘Classical Scientific Research’ which is more quantitative ii) ‘Interpretative Research’ which is more qualitative iii) ‘Engineering-oriented Research’
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classical Methods
3 Theory from Disciplines of Reference
2 Theory Axioms Postulates 7
Deductive Logic 4
Research 6 Project Hypothesis
5 Simulation 1 Real World
1- There exists a Real World that we can perceive. 2- Knowledge is Organized in Theories making inferences possible 3- Applied Disciplines like IS or T rely on Theories of Reference of other Disciplines 4- A Research Project permits testing formal Hypothesis formulated through inferences expressed in the Theory. 5- In Empirical Research, Hypothesis derived from the Theory are directly cheked in the Real World 6- Testing Hypothesis through Simulation 7- Results comming out of the Research give feedback to the Theory
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classical Methods • Summarizing: In traditional (or conventional/classical) Science, first new Hypotheses are extracted from the existent Theory, they are tested and new results are added to that Theory.
• Thus, this implies the pre-existance of:
• A theoretical Body of Knowledge
• An explicit theoretical Framework to guide the research
• Definition of what issues have to be investigated
• What are the explicit Hypotheses (and how could they be refuted (falsifiability))
• Techniques for Applied Research and well defined techniques for testing the Hypothesis.
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classical Methods • Formulating the Problem: • What are you supposed to solve or improve? What is its relevance?
•State of the Art • Literature review, Projects, current Limitations
•What are your Hypothesis and different Research Questions to be answered • In the form of: IF Independent variables THEN Dependent variable (to be measured) •An hypothesis should become a THESIS. •A Dissertation may include several Theses
•Experiments design: • System prototype or Simulation System • Collect Data • Select Metrics
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Classical Methods • Analyze Results • Quantitative versus Qualitative Analyses • Explicitly state the Assumptions • Compare and defend results
•Publish your conclusions • Show you know everything about the others ... • Highlight your contributions • Get feedback
•READY for writing your THESIS report
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Traditional Methods
1.4 Interpretative Research method
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Research Methods
1.4 Interpretative Research method
Is concerned with trying to understand lived experience and with how participants themselves make sense of their own experiences.
Therefore it is concerned with the meanings which those experiences hold for the participants.
IRM is phenomenological in that it wishes to explore an individual’s personal perception or account of an event or state as opposed to attempting to produce an objective record of the event or state itself. http://www.psyc.bbk.ac.uk/ipa/ (Birkbeck- U. London)
Eugénio Oliveira MSR- Methodologies for Scientific Research Interpretative Approach to Scientific Research
• Interpretative Method (or philosophy):
• post-positivist
• criticizes the (traditional) scientific “chimera”
• declares the impossibility of objective observation
• Observations and Interpretations depend on the Observer.
• Addresses critics to both “hard Sciences” and Social Sciences,
• Difficulties for Objectivity:
Eugénio Oliveira MSR- Methodologies for Scientific Research Interpretative Approach to Scientific Research
• Concepts, Factors, relationships that cannot be accessed • Researchers involvement in the research domain
• Results depending on the researcher own perspective: - On selecting and defining the research (sub-) domain - On selecting a pre-existant theory - On setting the issues to be investigated - On setting the precise scope of the research - On selecting and defining the Variables to be instantiated - On measuring those variables possible values
• Different attitudes on all those issues may lead to many different interpretations of the same observed phenomena.
Eugénio Oliveira MSR- Methodologies for Scientific Research Engineering-oriented Research Approach
1.5 Research in Engineering disciplines
Eugénio Oliveira MSR- Methodologies for Scientific Research Engineering-oriented Research Approach
• For this approach to research, technology is very important! (devices, artefacts, tools, practical procedures to be used)
• At least up to a certain extent is the most suitable for research in Informatics, Information Systems, Electronics, Telecommunications.
• Research in Informatics, IS and Telecomm is applied and problem-oriented.
• Implying:
Eugénio Oliveira MSR- Methodologies for Scientific Research Engineering-oriented Research Approach
•Implying:
• application of suitable technology for that specific problem, through
- conceptualization: requirements, models
- specifications: Formal Description
- implementation: prototyping,
- demonstration of (new) technology (real world/sim)
- Evaluation phase
Eugénio Oliveira MSR- Methodologies for Scientific Research Engineering-oriented Research Approach
Observation and Theory [Wohlin99]
• In engineering an hypothesis (also defined in system specifications) is usually an idea of the relationship between the cause and effect (defined in system requirements)
• Theoretical model is always only an approximation of observation in real world
C. Wohlin et al., Experimentation in Software Engineering: An Introduction, Springer, 1999.
Eugénio Oliveira MSR- Methodologies for Scientific Research Engineering-oriented Research Approach
Moreover when finding solutions for a problem:
In engineering we are fundamentally interested in how efficiently the basic resources are used for a needed system’s performance.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
1.6 Techniques associated to Research
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
B- Techniques
I. Non empírical Techniques
II. Scientific Research positivist Techniques
III. Scientific Research Interpretative Techniques
IV. Scientific Research Techniques at the positivist/interpretative boundary
V. Scientific Research Engineering-based Techniques
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques B- Techniques
A predictor variable is the portion of the experiment that is being manipulated to see if it has an effect on the dependent variable
• Experimental research is when a researcher is able to manipulate the predictor variables to identify a cause-and-effect relationship. experimental group, placebo group, Check if the same effects are independent of what you believe are the causes
A laboratory-based experiment gives a high level of control and reliability.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques B- Techniques
• Non-experimental research when a researcher cannot control, manipulate or alter the predictor variable It relies on interpretation, observation or interactions to come to a conclusion.
Typically, this means the non-experimental researcher must rely on correlations, surveys or case studies, and cannot demonstrate a true cause-and-effect relationship.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
I. Non empirical Techniques depend on:
• Artificially created data, or • Conceptual thinking about abstractions
Non-empirical research is when a researcher can make a persuasive (subjective) argument and can prove his/her argument without validating data (qualitative). It doesn’t just rely on statistics or numbers, which are the area of quantitative researchers • Non empirical Techniques include: • conceptual research based on opinion and speculation. ( argumentative and dialectic analysis)
• theorem proving: This applies to mathematical abstractions: • formal methods application, • Induction (in the sense that results are “abstractions”), • mathematical abstractions
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
I. Non empirical Techniques
• Models verification: Model checking is a technique for automatically verifying correctness properties of finite-state systems.
both the model of the system and the specification are formulated in some precise mathematical language: To this end, it is formulated as a task in logic Given a model of a system, exhaustively and automatically check whether this model meets a given specification.
• Simulation: • Mapping a complex environment in a simplified Model.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
• Other non empiric Techniques:
• Using hypothetic future scenarios (“What if” simulation) • Mapping a real scenario into Games • Mapping a real world scenario into Graphs
• Reviewing the State-of-the-art and doing Meta-analysis
Eugénio Oliveira MSR- Methodologies for Scientific Research Using Games for Research RoboSoccer Competition-middle size
Using Games for Research RoboSoccer Competition 3D http://www.youtube.com/watch?feature=player_embedded&v=3N_WSNf_aBI FEUP-LIACC / Eugénio Oliveira Eugénio Oliveira MSR- Methodologies for Scientific Research IA- Futebol Robótico- Sony legged
Março 2007 FEUP-LIACC / Eugénio Oliveira Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Mapping a real world scenario into Graphs
Translate from one domain to another and analyse in the new domain
Eg. To better analyse a signal you apply a Fourier Transform from time to frequency domain
Networks of everything (people, words, transports, companies, machines …)
Represented as Networks of nodes and edges (relationships)
Analyse the network !!
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Graphic’s Network analysis (through GRAPHS)
• Networks are everywhere • • Social, biological, financial, etc • • Complex networks • • Communities reveal properties of networks • • Contagion • • Controversies • • Society!
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Graphic’s Network analysis (through GRAPHS)
• Social networks • Social interactions: Likes, comments, share, cite • e-Commerce • Payments • Digital marketing • Political marketing • etc
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Graphs’ Network analysis
• Centrality Measures • Degree centrality • Edges per node (the more, the more important the node) • Closeness centrality • – How close the node is to every other node • Betweenness centrality • – How many shortest paths go through the edge node
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques Graphs’ Network analysis
• Measuring “importance” – Centrality, prestige, influence (incoming links)
• Diffusion modelling Network data is related to graph properties – Epidemiological • Real world data is big
• Clustering – sub-communities
• Visualization/Privacy/etc.
Eugénio Oliveira MSR- Methodologies for Scientific Research Tools
• Gephi • SocNetV • Netlogo
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
II. Scientific Research positivist Techniques
(applicable to Informatics and other Engineering disciplines): • Empirical Research methods help integrating research and practice • prove relevancy of theory by working in a real world environment
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
II. Scientific Research positivist Techniques
(applicable to Informatics, IS and other Engineering disciplines):
• Techniques for forecasting applying • regression algorithms (Technique for fitting a simple equation to real data points
• time-series based algorithms to extrapolate on historic data ; to extract meaningful statistics from data over time;
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
II. Scientific Research positivist Techniques (applicable to Informatics, IS and other Engineering disciplines):
• Field Experiments : • Possibly isolating and controlling a set of phenomena to be looked at in the real world
• Laboratory Experiments: • Creating artificial setups in which phenomena, factors and variables are isolated and controlled.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
III. Scientific Research Interpretativist Techniques:
• Research approach is Descriptive/Interpretative:
Going from empirical observations to a limited rigour analysis. Need for controlling the researcher’s perspective:
- Critical Self- examination - Revising all the postulates and biases - Varying Observations - Submission to peer reviewing
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
III. Scientific Research Interpretativist Techniques:
• Group Research:
• frequent discussion with a group of people including who may be affected by the application (or the technology) • Use Collaborative Work Tools.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
• Action-based Research:
- Researcher is like an “Agent” changing existent conditions to measure the respective reaction
In http://www.qual.auckland.ac.nz/ Michael Myers Ed. Ass.Inf.Sys. qualitative research is advocated , for example, for IS.
We also can do (but it is not recommended) R&D just through Case-study analysis
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
IV. Scientific Research Techniques at the positivist/interpretativist boundary
Including: - Field Work : Data related to the object of study is directly observed by the Researcher in the original context - Questionary-based : Structured Data collection comming from interviewees
- Case Study: - Isolate and describe a relevant real-world situation, - Gather a collection of data from multiple sources
EugénioEugénio Oliveira Oliveira- 2014 MSR- Methodologies for Scientific Research Research Techniques
• Secondary Research:
• Analysis starting from pre-existent documents (texts, papers, descriptions, memoirs).
• Analysis is done under new perspective and using different techniques or procedures.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques v. Scientific Research Engineering-based Techniques
(Informatics Engineering, IS, Electronic Engineering, Telecomm...)
• Divided into two categories:
1- Constructive Techniques: • Includes Design, Project, Implementation (or prototyping) of a system (ex: Computer programming System, or physical device)
2- Destructive Techniques:
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
1- Constructive Technique: • Includes Design, Project, Implementation (or prototyping) of a system (ex: Computer programming System, or physical device)
- objective: -- explicitly testing an hypothesis or -- to solve a class of problems. - it implies: -- Existence of either a theory or a model for explaining the results (testing the hypothesis)
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Materials and Methods
• in exact sciences materials include definitions, model, and related assumptions
• theoretical model is defined (preferably a mathematical or logical model), including:
boundary conditions (define the environment and scenario) and initial conditions (define the initial values of the parameters of the model) [Losee01]
J. Losee, A Historical Introduction to the Philosophy of Science, 4th ed. Oxford Univ. Press, 2001.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
• methods include: rules of analysis and rules of verification and validation
• rules of analysis by deductive reasoning, statistical analysis simulations (a simplification)
• the theoretical model is verified by comparing the results with reality (measurements with a prototype, known as hypothetical-deductive method)
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
• Results
• results can be analytical (deductive), from simulation, or measurement (in the real world) results
• it is important to present numerical results that verify (or not) your own thesis, In simulation, values of effective parameters must be specified and results will be true just for that value of parameter. But analytical result, give a general description of system for any value of parameters
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Research Techniques
• Results
• use analytical results in simple limited cases to obtain reference values
• the value of the results depend on how well they can be generalized (bottom up approach will help you)
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques Conclusions (“constructive “-based technique report example)
(1)“We have shown that [some] algorithms proved right…”
(2) “Our approach is simpler to implement than previous algorithms and is practically feasible …. On the theoretical side, the greatest deficiency in our results is …”
(3) “In a practical situation the advantages and limitations of our approach are…”
(4) “These results can be extended in a number of directions…”
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
Conclusions (“constructive “-based technique
report example)
1. What is shown by that work and its significance. 2. Limitations and advantages. 3. Applications of the results. 4. Recommendations for further work.
Eugénio Oliveira MSR- Methodologies for Scientific Research Research Techniques
• Destructive Techniques:
• To analyse a system, or a set of methods to gather new information about those technology procedures under analysis
- Recognized information and characteristics are then reused may be in a different way
- Example: Reverse Engineering
Eugénio Oliveira MSR- Methodologies for Scientific Research Phylosophy Underlying Research
• There are philosophies permitting the Classification of Research Methods through Paradigms or basic Epistemologies:
I. Positivist II. Interpretative (post-positivist) III. Critical IV. (Constructivism)
At Harvard Business School they use just the three categories
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Positivist Method:
• Assumes that reality is objectively accessible, it can be measured and described by the observer and measured by the same observer through their instruments and devices. Tests are made making predictions possible over future phenomena • IS research (also Telecomm research?) can be of the positivist kind (for IS, Orlikowski and Baroudi,1995) - Use math/logic formalisms - use quantifiable and measurable variables - Test an Hypothesis and make inferences from what is known.
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Interpretativist Method
• Assumes that accessing to reality is only possible through social constructions like language, consciousness and ontologies (sharing of concepts’ meaning).
• Hermeneutics-based philosophy (based on the human understanding and text interpretation) as well as in phenomenology (describing, understanding and interpretation perceived phenomena)
Proposes to abolish separation between "subject“ and "object", It opposes to XIX century positivist thinking.
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Can Informatics, IS and Telecomm studies influence or be influenced by the context?
• Here, the focus is not as much on the variables, measurements and processes as it is on interpretation of the results of the test, the measurement or implementation
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Research Criticism:
• It is a radical enterprise once it tries to identify what constraints research: • socially, culturally, • Historically, politically, • economically, physically
trying to emancipate science by eliminating causes for ignorance, alienation etc...
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Research Criticism:
Eg: Why is it not possible to do research on X? Eg; Why cannot we implement systems of type Y?
Habermas from Frankfurt School following the path of Adorno, Lukacs, Marcuse
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Junk Science (?) Stories of the Past Decade
Dial “F” for Fear.
Since the 1993 Larry King Live broadcast featuring a man suing a cell phone maker claiming his wife died from a cell phone-induced brain cancer, many cell phone users have worried about phone safety.
But studies failed to identify any risk for many, many years …
Eugénio Oliveira MSR- Methodologies for Scientific Research Philosophy Underlying Research
• Junk Science (?) Stories of the Past Decade Powerline scare unplugged.
Fears that electric and magnetic fields (EMFs) created by power lines and appliances caused cancer started in 1978. Parents worried about power lines over schools.
Power companies worried about burying power lines. The National Academy of Sciences finally unplugged the scare in October 1996, concluding that no evidence showed EMFs presented a health hazard
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
1.7 Quality of Research
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
• Quality depends on the Objectives to reach 1. To increase available (and also general) Knowledge • Objectives and Quality criteria are internal to the discipline • RIGOUR is very critical 2. Developing Computer Systems • Objectives and Criteria are external (regarding R&D) • Relevance can be measured by the impact and is of the most importance
• Relevance Versus Rigour • Not antagonic but taken with different emphasis
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality and directions of Research
• Computer Science (and other Engineering-like areas) tendency:
• Until the 80ies conceptual (not empirical) papers dominated • After the 80 ies Less theoretical papers, more instantiated, less general and Knowledge less cumulative
• Case-studies based research is mainly descriptive concerning simple instances and then less scientific
• Empirical research is now dominated by the descriptive approaches
• Too specific Models have limited capabilities for explaining phenomena as well as little predictive power.
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality and directions of Research
• Informatics and Computer Science Research:
• Services, Platforms and Frameworks for Electronic inter-operability between people as well as Organizations (enterprises, schools, institutions…) inside themselves and interfacing to society (markets)
• Including support to the automation of:
• Logistics, procurement, contracting, trading, Mkt • Reorganization and reconfiguration
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality and directions of Research
• Informatics and Computer Science Research:
• Algorithms and Systems for Information/K management: • Visual (image, graphic…) • Textual (Text Corpora, data in DB) Big Data • Signals (traffic, biologic, environmental….)
• Supporting Sciences and application domains:
• Biotechnology • Graphic Computing • Medicine • Law • Astronomy, Phisics, History, Management….
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
• Requirements for the Research in CS and TC
• Choose an appropriate Research Method:
• Traditional/positivist, “interpretativiste” or Engineering-oriented
• Make explicit a Body of Theory sometimes coming from other adjacent disciplines (ex: Computer Science, Organization Theory, Information Theory, Mathematics, Physics, Statistics…)
• Extend the Theory and Infer predictions to be checked (at least in the case of positivist/traditional research)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods, General
(following Björn Lisper, Datorteknik, Mälardalens högskola Västerås, Sweeden) http://www.idt.mdh.se/kurser/ct3340/ht03/
Theoretical methods:
Create formal models (mathematics, logic)
Define concepts relevant for the Thesis objectives
Prove properties of the concepts
Abstraction, hide details to make the whole more understandable (and to make it possible to prove properties of it)
Proofs of properties by deductive methods
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods, General
Empirical methods:
Perform experiments
Extract appropriate results
Draw conclusions and explain according to a theory
Simulation: Start with a formal model at some "easy-to-understand" level
Make "artificial experiments" in your computer
Collect statistics and draw conclusions
Do not forget to explicitly state the limitations of the simulation
Eugénio Oliveira MSR- Methodologies for Scientific Research Validation and statitical analysis
Statistical tests examine a variety of relationships in data
• "Statistical Significance" (p-value):
statistical significance of a result tells us something about the degree to which the result is "true" (in the sense of being "representative of the population").
Eugénio Oliveira MSR- Methodologies for Scientific Research Validation and statitical analysis
Statistical tests examine a variety of relationships in data
p-value represents a decreasing index of the reliability of a result. The higher the p-value, the less we can believe that the observed relation between variables in the sample is a reliable indicator of the relation between the respective variables in the population
• Specifically, the p-value represents the probability of error that is involved in accepting our observed result as valid, that is, as "representative of the population."
Eugénio Oliveira MSR- Methodologies for Scientific Research Formalization
Formalize as much as possible
• using logic (propositional/ first order/ modal) • using mathematical formalisms • create your own formalism by explaining your Notation
• Example of MASDIMA Thesis and Book
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods
In Physics (exact sciences): Make hypotheses about the surrounding world (theory), observe it (experiment) Relate the result of experiment to the theory
Adjust the theory if it doesn’t predict the reality well enough Theory is used to predict the future (e.g., if a bridge will hold for a certain load, or an asteroid fall down on our heads)
Common pattern in Computing Science: The system is constructed to behave according to some theoretical model Deviations are seen as construction errors rather than deficiencies in the theory (hardware error, bug in OS, : : :)
In both cases: the theory helps us to understand and predict, but in different ways!
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods Theoretical vs. Empirical Methods in Computing Science
Computing Science really has a “spectrum”, from “extreme constructivism” to the use of theory close to the one in physics:
• “Extreme constructivism”: (ideal) programming language design:
Formal semantics for the language, pure construction of model defining the mathematical meaning of each program
Abstraction of details to make the meaning of the language simpler (for instance, assume that data structures can grow arbitrarily big) Implement the language according to the semantics
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods Theoretical vs. Empirical Methods in Computing Science
One can prove formally within the model that a program is correct –valuable!
But the model does not cover all kinds of failures. E.g., hardware errors, or stack overflow (or an asteroid falling down on the computer)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods Theoretical vs. Empirical Methods in Computing Science
Extreme “Theoretical (physics)” approach: performance modeling of sophisticated computer and communication systems
Extremely hard to make analytical calculations Simplified performance models, tested against experiments (e.g., long suites of benchmarks)
Discrepancy leads to a modified theory, as in physics Often use simulation (desire to evaluate systems before building them)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods Theoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Build on some form of formal model for how the algorithm executes (meta-language with formal semantics), and some performance model (how long does a step in the algorithm take, how much memory is needed to store an entity)
Eugénio Oliveira MSR- Methodologies for Scientific Research Scientific Methods Theoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Given that the performance model is correct, one proves mathematically that the algorithm needs certain resources (time, memory) to be carried out.
But the performance model is often very approximate.
Sometimes is possible to refine the performance model, but this can make it impossible to calculate the resource needs of the algorithm
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
• Characteristics of the Research (in CS)
• Always Report the most recent S-o-A. (always moving...) • Prove you made an advancement in Knowledge • Combination of Techniques trying to compensate one’s weaknesses with other one strengths
• Report about the object of study and the nature of the domain • Validate Data • Analysis using statistics • Extract relationships between variables
• Praticable according to the resources • Relevant results (at different levels: community,society...) • Publish • Ambitious
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
• Characterístics of the research (in CS):
• Research may have as an objective: • Building a Theory • Testing a Theory • Extending a Theory
• The practical research objectives may include: • Exploring a specific domain • Describing a domain • Explaining a domain • Forecasting over a domain • Applying the (extended) Theory in a (new) domain • (Testing formalized hipothesis)
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research: DATA
• Other characteristics:
• Concerning Data: • Extensive • Representative • What is Data validity (ex. temporal, precision…) • Granularity: • Simple or agregate • Specific or generic
• Temporal Horizon:
• “Snapshot” of the domain at a specific point in time • “Longitudinal cut section” (múltiple points in time)
• Methods for colecting and analising data; quantification...
Eugénio Oliveira MSR- Methodologies for Scientific Research DATA
• Other characteristics:
• Concerning Data: • Qualitative Vs Quantitative Data Data that approximates or characterizes but does not measure the attributes, characteristics, properties, etc., of a thing or phenomenon. Qualitative data describes whereas quantitative data defines.
Qualitative data: Quantitative data: blue/green color, gold frame picture is 10" by 14" smells old and musty with frame 14" by 18" texture shows brush strokes of oil paint weighs 8.5 pounds peaceful scene of the country surface area of painting is 140 sq. in. masterful brush strokes cost $300
Eugénio Oliveira MSR- Methodologies for Scientific Research DATA Examples of Fuzzy Data: • Tall Persons (Height) • Dangerous Maneuvers (Action Sequences) • Blonde Individuals (Hair color) • Loud Noises (Sound Intensity) • Large Investments (Money) • High Speeds (Speed) • Close Objects (Distance) • Large Numbers (Numbers) • Desirable Actions (Decision or Control Space)
Eugénio Oliveira MSR- Methodologies for Scientific Research DATA
"Fuzzy theory is wrong, wrong, and pernicious. What we need is more logical thinking, not less. The danger of fuzzy logic is that it will encourage the sort of imprecise thinking that has brought us so much trouble. Fuzzy logic is the cocaine of science." -Professor William Kahan UC Berkeley
Eugénio Oliveira MSR- Methodologies for Scientific Research Quality of Research
1.8 Challenges
Eugénio Oliveira MSR- Methodologies for Scientific Research Research in CS and TC Challenges
• Challenges: • There exists shortage of theory. Mainly theories in related disciplines • Phenomena are unstable: • Data may get old quickly, usually a snapshot that may vary - Pay attention to the validity period • Organizations and Markets are always moving
• Significant cultural variations: • Different national/regional cultures imply different speed/rhythm
• When you are doing research (requirements, interaction, questionnaires, demos) you influence the domain
• Pressure for publishing during the research action
Eugénio Oliveira MSR- Methodologies for Scientific Research Research in CS and TC Challenges
• Pressure for technical instantiation:
• Prototyping without a theoretical ground or well defined objective • Economics and business models either not known or consolidate.
• Impact of the automation and rationalization: • Depends on time dynamics concerns • Depends on who is directly affected • Depending on if researchers are also objects of the study
• Adopting a new technology may take long time • Impact on other partners, markets, standards, tools…
Eugénio Oliveira MSR- Methodologies for Scientific Research METRICS
• Define apropriate metrics for all the different stages of the implemented solution
MASDIMA example
Eugénio Oliveira MSR- Methodologies for Scientific Research Research in CS and TC Challenges
• Conclusions on Research actions (in CS and TC)
• Consider tradoff Rigour Versus Relevance • Identify challenges BEFORE starting doing research • Be clear about conventions related with research to be done • Select a significant sub-domain • Define questions to be answered through that research • Select the apropriate Research Method and Techniques (possibly a good combination of some) and justify the choice.
Whenever the selected aproach is instrumentalist, you need to be sure about the external quality, data validation and rigour as well as Relevance
Eugénio Oliveira MSR- Methodologies for Scientific Research Remember
• Follow a “Scientific Thesis project Flow Chart
• Use Falsifiability : Always make clear all the assumptions you are using under which you believe your THESIS is correct.
•Characterize your Research: usually Applied or Problem Oriented. ‘Predictive’ or ‘Prescriptive’,
•In engineering an hypothesis (system specifications) is usually an idea of the relationship between the cause and effect (system requirements). Ususlly Constructive.
•Research in Engineering inherits maily form Positivist research: Empirical and Non-Empirical (simulation)
•Testing formalized hipothesis: Verification Vs Validation
Eugénio Oliveira MSR- Methodologies for Scientific Research Remember
• Qualitative Vs Quantitative Data
• Define appropriate Metrics
• Tradeoff Rigour versus Relevance
Eugénio Oliveira MSR- Methodologies for Scientific Research Other Bibliography
• D. Sternberg, How to Complete and Survive a Doctoral Thesis. St. Martin’s Press, 1981.
Eugénio Oliveira MSR- Methodologies for Scientific Research