Abstract
Standard methodological and statistical texts divide research methodology into two strictly separated categories: confirmatory and exploratory research. In some fields, like scientific psychology, almost all research reports are written up as if they are confirmatory, i.e., involve rigorous tests of an antecedent theory. In reality, however, typical research in psychology involves an iterative procedure in which theory is adapted in view of the data, and new data are gathered to further investigate the adequacy of these theory changes. This has led several critics to lament the exploratory aspects of psychological research, as it leads to the possibility of hypothesizing after the facts are known: HARKing. HARKing is a problem because it involves generating hypotheses post hoc, while presenting these as prior to the research project. Thus, it presents research that is exploratory as if it is confirmatory. This practice has been argued to generate an excess of false positive findings, and as such is suspected to lie at the basis of the replication crisis in psychology. Accordingly, in response to that crisis, there has been a rapid surge in the development of methodological tools designed to make the theory testing process more rigorous: from preregistration to blinded data analysis and from many-labs paradigms to reproducibility projects. I argue that this response puts the horse behind the cart, because most psychological research should not be characterized as confirmatory or exploratory, but as aimed at theory construction. This diagnosis has direct implications for the organization of psychological science and the methodological education of psychologists. First, I will argue that even though theory construction has a creative dimension, there is also much logic to the process; as such, the process can be systematized and structured in the same way that we systematize other research processes. This invites the development of techniques and tools that can be used to support theory formation; an example is our recently introduced theory construction methodology, which is a structured series of steps that can be followed to develop theory. Second, theory construction is not covered by standard research methodology and is not taught in psychology curricula. Instead, it is treated as an almost mystical process by which a researcher is supposed to conjure theories out of thin air. However, I argue that theory construction is a skill like any other, and it should be practiced and taught. Third, reports of theory construction research do not fit current reporting standards in scientific publishing, which are almost entirely structured to present either empirical discoveries or tests of scientific theories. Thus, we need new reporting formats to allow such research to be reported truthfully. I will argue that, together, these elements of theory construction define a methodological agenda that has the potential to significantly advance psychological science.
