The Persistence of Error

There is an embarrassing, unanswered question about theories and models in Psychology that is screaming to be answered. If the evidence in support of Psychology’s models and theories is so meagre and feeble, how have they survived for such a long time?

The scientific method is intended to be a fail-safe procedure for abandoning disconfirmed hypotheses and progressing with hypotheses that appear not to be disconfirmed. The psychologists who dream up these theories and test them claim to be scientists, so what the heck is going on?

One reason that theories and models become semi-permanent features of textbooks and degree programmes is that simple rules at the very heart of science are persistently broken. If a theory is tested and found wanting, then one of two things happens: either (1) the theory is revised and retested or (2) the theory is abandoned. The history of science suggests that (1) is far more frequent than (2). Investigators become attached to the theories and models that they are working with, not to mention their careers, and they invest significant amounts of time, energy and funds in them, and are loath to give them up, a bit like a worn-out but comfortable armchair.

We’ve all been there – seen it, done it, even have the T-shirt:

Nothing dishonest is happening in most such cases, simply an unwitting bias to confirm one’s theoretical predilections. This is the well-known confirmation bias studied by, yes, you guessed it, psychologists (e.g. Nickerson, 1988).

The process of theory or model testing is illustrated in the diagram. The diagram shows how the research process insulates theories and models against negative results, leading to the persistence of error over many decades. Continuous cycles of revisions and extensions following meagre or negative results protect the model from its ultimate abandonment until every possible amendment and extension has been tested and tried and found to be wanting.

What textbooks don’t tell you: the persistence of error – the manner in which a model or theory is ‘insulated’ against negative results

Several protective measures are available to insulate investigators from ‘negative’ results:
(1) Amend the model and test it again, a process that can be repeated indefinitely.
(2) Test and retest the model ignoring the ‘bad’ results until some positive results appear that can happen purely by chance (a type 2 error).
(3) Carry out some ‘statistical wizardry’ to concoct a more favourable-looking outcome.
(4) Do nothing, i.e. do not publish the findings, and/or:
(5) Look for another theory or model to test and start all over again!

Beside all of these issues, there is increasing evidence of lack of replication, selective publication of positive findings, and outright fraud in psychological research, all of which militate against authentic separation of fact from fantasy (Yong, 2012).

Little attention has been paid to the cultural, socio-political and economic conditions that create the context for individual health experience and behaviour (Marks, 1996). Thousands of studies have accumulated to the evidence base that is showing that socio-cognitive approach provides inadequate theories of behaviour change. Any theory that neglects the complex cognitive, emotional and behavioural conditions that influence human choices is unlikely to be fit for purpose. Furthermore, health psychology theories are disconnected from the known cultural, socio-political, and community contexts of health behaviour (Marks, 2002). Slowly but surely these issues are becoming more widely recognized across the discipline and, at some point in the future, could become mainstream.

As we have seen, critics of the socio-cognitive approach have suggested that SCMs are tautological and irrefutable (Geir Smedslund, 2000). If this is true, then no matter how many studies are carried out to investigate a social cognitive theory, there will be no genuine progress in understanding.

Weinstein (1993: 324) summarized the state of health behaviour research as follows: ‘despite a large empirical literature, there is still no consensus that certain models of health behaviour are more accurate than others, that certain variables are more influential than others, or that certain behaviours or situations are understood better than others.’ Unfortunately, there has been little improvement since then. The individual-level approach to health interventions focuses on theoretical models, piloting, testing and running randomized controlled trials to demonstrate efficacy.

It has been estimated that the time from conception to funding and completing the process of demonstrated effectiveness can take at least 17 years (Clark, 2008). Meta-analyses, reviewed here, suggest that the ‘proof of the pudding’ in the form of truly effective individual-level interventions is yet to materialize.  Alternative approaches for the creation of interventions for at-risk communities and population groups are needed. A fresh approach requires a general theory of behaviour that encompasses human intentionality, desire and purpose within an ontology of change.