Revised 15 September 2021


Peer Review Process

The challenge of the scientific method is to eliminate personal and subjective influences. Methods of science have been developed to eliminate these influences. One method is the peer review process. 


Scientists submit their findings to research journals for publication. Each submission undergoes a peer review by a panel of referees who are experts in the field. Each referee independently reviews the submission and identifies deficiencies and any errors. The referees report back to the editor of the journal who either accepts the submission for publication, invites the submitter to address the issues raised by the referees and re-submit, or rejects the submission outright. A satisfactory re-submission results in publication. 


There are differing degrees of stringency in the peer review process provided by different avenues of publication. The gate keeping to publication is the most severe with high-ranking international journals where the world’s leading scientists are invited to be referees. Gate keeping of the peer review process is less severe with national or local journals. 


Some conferences claim that acceptance of conference papers for publication in the conference proceedings are subject to peer review. These claims are not always upheld. Scientist know which conferences have a rigorous peer review process or not. Many conferences do not have any review process. Scientists submit their research findings to conference organisers to establish priority and to gain feedback from fellow scientists who attend the conferences. Some conference papers are subsequently revised and then submitted for publication in a research journal. More brownie points for promotion are awarded for publications in peer reviewed journals than conferences. The findings of conference papers should be regarded as being tentative and yet to be confirmed by the research community.


Observations

Ideally, all scientific observations should be absolutely objective, but in reality, observations will always be limited and prejudiced in some way due the inherent limitations of our sensory perceptions and our personal subjective biases. Everything we observe depends on our senses and are seen from a particular perspective. Even our choice as to what we observe is a subjective decision based on what we believe to be important and relevant. 


Experiments

Experiments are carried out to enable an investigator to focus on and measure a single or more than one attribute of what is being examined by eliminating all other extraneous influences. Laboratory experiments create an artificial controlled environment which does not fully reflect the real world. Experiments are a convenient way of reducing the complexity of the real world by examining only a part of that world. This approach is called reductionism based on the belief that we can best explain something by breaking it down into its individual parts. The very act of deciding what constitutes a part of the whole is subjective. 


The reductionist approach works to a high degree of success in some cases and not others. The behaviour of an entire system cannot be deduced by examining only the individual behaviour of the parts when the behaviour of the entire system is influenced by the interactions of the parts. A systems dynamics approach is necessary to model and simulate the behaviour of the total system. See Sub-section: Systems Dynamics. 


Reproduction of Results

The results of experiments can be subject to human error and/or defects in the equipment being used. Peer reviewed publications are required to include a data and methodology section in sufficient detail to enable other researchers to repeat the experiment. Experiments which cannot be reproduced are rejected by the science community. Challenges of defective research are normally published in the same research journal and the original researcher has the right of reply. Irretrievably defective publications are retracted by the journal. 


Competing Theories

Francis Bacon (1561-1626) thought it was possible to carry out crucial experiments which would be decisive in establishing which of a number of theories are correct. Pierre Duhem (1861-1916) argued that it is impossible to ever know the extent of all possible theories that can be applied to any set of experimental results. Nonetheless, some experiments or observations do appear to be decisive in determining which theory is superior to another. An example is Arthur Eddington’s observation and confirmation of the deflection of light by the mass of the sun as predicted by Albert Einstein in his 1916 Theory of General Relativity. 


When there are several possible theories which explain the data, the principle of Occam’s Razor is applied where simple explanations are preferred over complicated explanations. For example, astronomers 2,000 years ago assumed that the sun and planets orbited in perfect circles around Earth. Ad hoc subsidiary hypotheses in the form of epicycles were necessary to account for the observed retrograde motion of some planets. The precision of the calculated motions of the planets based on these hypotheses was in reasonable accord with the precision of their astronomical instruments. The current theory of planetary motions where all planets orbit the sun in ellipses is a much simpler theory and provides greater precision of predicted movements which are in accord with our more precise astronomical instruments. 


The Inductive Method

The inductive method of research is based on collecting information and then drawing conclusions which leads to the framing of an hypothesis. A scientific hypothesis is framed in a precise form which predicts an outcome which can be tested by experiments, the result of which either confirms or disproves the hypothesis. 


David Hume (1711-1776), a Scottish philosopher, established that no amount of evidence could ever lead to the claim of absolute certainty. Regardless of how many times evidence confirms a scientific law, that law can be regarded as only attaining a higher degree of probability with each confirmation. Well-established scientific laws are always subject to being incomplete as was the case with Newton’s 250-year-old Laws of Motion and Gravity. 


The classic problem of induction is that no amount of evidence will ever be enough to prove a case. There is always the possibility of being proved wrong. A classic example is that of the black swan. A scientist could make a comprehensive study of swans based on Linnaeus’s taxonomy of what constitutes a swan. After observing hundreds of swans, all of which are white, the logic of induction would conclude that all swans are white. But there are also black swans. Another classic example is the turkey which is fed every day for 364 days by a farmer and the turkey expects to be fed on the 365th day which is Thanksgiving Day. The farmer and his family enjoy eating turkey for dinner that day. 


The Hypothetico-deductive Method

The hypothetico-deductive method is opposite from the induction method in that it starts with an hypothesis instead of observation. It then proceeds to establish whether that hypothesis is correct or not by deducing a prediction which can be tested by way of experiment or observation.  The method uses evidence to test out an hypothesis as opposed to deducing an hypothesis based on the evidence. 


Albert Einstein used the hypothetico-deductive method in his 1905 development of his theory of special relativity. He based his paper primarily on two principles. The first principle referred to the uniform relative motion of two coordinate systems which basically states all motion is relative. The second principle postulated that a light ray moves with a fixed velocity independently of whether this ray of light is emitted by a body at rest or in motion. This property of light had been established experimentally by Michelson and Morley in 1887, but Einstein postulated this property of light as an axiom and proceeded by a process of deduction to develop his theory of special relativity. In a two-page addenda paper published in the same year, Einstein completed his theory of special relativity with his now famous equation E=mc2 (Stachel, 1998). Einstein's special theory of relativity applies only to systems undergoing uniform motion. In 1916, Einstein completed his general theory of relativity which included systems undergoing acceleration. Newton's theory of gravity was based on unexplained action at a distance. Newton was uncomfortable about the concept of action at a distance and when pressed for an explanation, his reply was "I form no hypotheses". Newton's theory of gravity was purely descriptive and not explanatory, whereas Einstein's theory of general relativity explained gravity as being an interaction between mass and spacetime, an explanation which does not involve action at a distance. John Wheeler (1998) provides a simple and apt explanation: "Spacetime tells matter how to move; matter tells spacetime how to curve."

 

Abductive Reasoning

Instead of working from a cause or precondition to a consequence or effect, scientists work back from a consequence to a precondition using a process of inference to find the best explanations. Mel Thompson (2011) provides an example of someone finding an unopened letter on the pavement. The finder of the letter infers that a postman dropped the letter. There are many explanations of how the letter could have turned up on the pavement, but only a few reasonably probable explanations. The owner of the letter could have dropped the letter on the pavement or someone else removed the letter from the owner’s letterbox and dropped it on the pavement. The postman might have delivered the letter, but was too hasty in shoving it in the letterbox. The wind caught it and it floated onto the pavement. Each additional piece of information about the letter would provide support for one possible inference over another - the address on the letter compared to the location where found, the date stamp and condition of the letter, and contact with the owner of the letter to confirm delivery.  


Another example of abductive reasoning is the theory of tectonic plate movement. This theory was initially rejected by the scientific community as being impossible. Subsequent additional information firmly supported the theory of tectonic plate movement over competing theories. Although we can never know with absolute certainty what happened in the past, it is possible to identify which theories are more probable than others. 


Some people expect science to provide absolute certainty. This expectation is unrealistic. As the saying goes, there are no certainties in life except taxes and death. Some people need and seek certainty in their lives. Pseudoscience panders to this need. 



Refer to the Recommended Books Sub-section: Scientific Method for the sources of this Sub-section and further reading.