Critical thinking - questioning the credibility of information - is always a wise practice. When reviewing information we should ask ourselves: "On what basis is this claim being made? What evidence does the author give to justify his or her conclusions?"

Among scientists, social scientists, and other scholarly professionals, there are established criteria for assuring the credibility of sources:

Credentials

One criterion is the expert credentials of the author, based on successful completion of a high level of education, with the highest being a Doctoral degree, typically expressed as "PhD" (Doctor of Philosophy) within the subject area. Added to this are the recognitions that such an individual attains from their peers as a result of outstanding, rigorous scholarly activity.

Peer Review

One type of peer recognition is particularly critical, and is not directed at the individual per se but at each article, chapter or book that an individual expert seeks to publish. This 'peer review' typically consists of a thorough critical assessment of each work by a panel of at least three widely recognized experts in the same field.

The author is normally not informed of who the reviewers are, and the process is managed by the publisher as an independent third party. Reviewers examine the work for its accuracy, the rigor of its methods, and the justifiability of its conclusions based on the current state of expertise in that field. Reviewers submit written reports to the publisher. The author is given an opportunity to respond to criticisms and make corrections, and a final review is then completed by the same panel.

If the work is deemed to be rigorous it will be published. If not, the publisher will reject it. The process can be quite time consuming and contentious, especially when new forms of analysis or new theories are being developed.

A less formal kind of peer review is also expected of scholars: the presentation of their research and ideas to their peers at expert conferences. In this context a scientist may be subject to criticism from tens or even hundreds of her or his colleagues in a public forum.

These mechanisms are in place to ensure that new developments, recommendations, and decisions resulting from research will be based on sound, defensible, justifiable, logical reasoning and evidence. The very concept of peer review is grounded in an expectation of transparency and expert scrutiny. The rules for its conduct are always clearly established and not subject to arbitrary manipulation.

It is human nature to form opinions based on perceptions, comments from people we know, or statements made in the media or other public contexts. If these are not based on solid evidence and a critical understanding of the issue, they can result in misunderstanding. Critical thinking and and tools like peer review alert us to cases where people may intentionally misuse evidence or misrepresent circumstances. This can be even more challenging when people speak or write with authority, and claim to be experts. When it comes to complex and highly technical issues, it is wiser to seek sources of information from fully accredited experts and publications that have undergone formal peer review.

Peer Review is Not Perfect 

While Peer Review is the best process to mitigate inaccuracies, it is not perfect. Rob Thacker's recent article in The Mark, Reviewing Peer Review, outlines some of the challenges of the peer review process.

The CRU Hacking incident at the University of East Anglia has also shone a spotlight on the peer review process as it relates to climate science. The UK's Guardian newspaper has published an investigative series on what's refered to as 'Climategate'. For full coverage, visit Climate Wars: Guardian special investigation.

Part Eleven of the Guardian's investigation examines the peer review process and the calls to make research more accessible.