2.9 Literature review
Literature review is defined as the collection, synthesis, and
evaluation of published studies. Published studies could be primary,
secondary, and tertiary. Primary literature review makes use of
materials encompassing empirical studies – ranging from observation,
interviews, to experimental and is meant to provide an assessment of
what has been published on a topic of interest mainly of actual
practices. The secondary literature derived from the exposition of the
primary literature sources are generally reviewed studies. These studies
could be narrative, systematic, semi-systematic and integrative. Authors
of these studies integrate primary literature studies through synthesis,
thus, further expanding on the findings of primary literature studies.
The outcomes of this type of literature review have the potential to be
generalizable. The tertiary source encompasses the distillation and
collection of materials derived from the hybrid of primary and secondary
literature sources, for example, textbooks, guidebooks, encyclopedia.
Thus, the purpose of the tertiary literature is to offer an impression
of important findings of existing studies or research. One of the
importance of a tertiary literature review is its ability to introduce
principles to practices in a discipline. By far this type of literature
review is the least used one in academia. Literature can either be
narrative or systematic. Narrative review is the conventional and the
oldest means of studying extant studies and is generally qualitative
with no particular “formal” guide for undertaking it. As a result, it
does not generally seek generalization. Below, we provide an overview of
systematic literature review.
Literature review in the medical field has largely been systematic and
meta-analysis providing a quantitative evidence for reliable conclusion
and generalizability (Tranfield et al., 2003). Narrative literature
review has been used extensively in the non-medical field to provide
basis for further exploration of multi-facet and ongoing changes of
situations and a timely response to the experiences of people/subjects
and situations. Whilst not generally generalizable, it serves a great
purpose for providing immediate overview/understanding of existing and
new phenomena using varied methodologies including socio-cultural,
natural and library and processes in the context of data collection and
conception, data analysis, and data reporting (McAlpine, 2016). It is
interesting how researchers can draw conclusion from a narrative
literature study given unstructured nature. There must be some assurance
that the study is comprehensive and has covered prominent research of
the given time. The selection of and critically analysing large set of
literature from varying sources is appearing to be relevant to produce
generalizable results.
Often, and generally, SLR studies selection criteria have been one that
is based on published empirical materials. A concern with this criterion
emanates from the fact that “there is a growing recognition that often
evidence is difficult to find because of decisions that are made about
where, how, and when to publish the results of studies based on the
findings of those studies” (Balshem et al., 2013, p.1). This makes room
for undetected biases in publish materials. One of such sources of
materials is the grey literature. Grey literature is literature from
unpublished reports from government and others such as dissertation that
is now being touted as becoming an important additional source of
literature towards evidence-based studies. The Institute of Medicine and
the Agency for Healthcare Research and Quality (AHRQ) and current
Cochrane guidance made recommendations to this effect. Generally, useful
interventions are published far more than interventions that produced
negative poor outcomes. Whilst the inclusion of grey literature is being
encouraged, the impact of their exclusion has been found to be
relatively small or negligible. Given both technical (structured) and
non-technical (abstract) nature of IS, the careful collation and
consolidation of materials from grey literature and unpublished
dissertation and theses will be useful. Like in management this will
help respond directly and timely to changing practice and policy needs.
Whilst majority of SLR studies searched for non-English and unpublished
papers, less than 5% of these studies are included for full review
(Hartling et al., 2017).
As research continue to grow and information explodes in the discipline
of IS, literature reviews have increasingly become crucial in the
definition and understanding of IS for academics, practitioners, and
policymakers alike.
Overview of systematic literature review
In the medical and public health fields, SLR has been hailed as a gold
standard methodology for evidence-based research since the establishment
of Cochrane Collaboration in 1993. In the discipline of IS, a rigorous
systematic literature review study would be critical to collate and
summarise evidence on how IS technologies, specifically digital health
technologies are improving healthcare delivery and outcomes. Currently,
digital health technologies are generally known for their “potential”.
SLR approach is already in use in the IS research. SLR requires thought,
probing of concepts, interrogation of assumptions and expectations, and
a considered appraisal of where a SLR might fit into the general scheme
of knowledge and keep expanding in the domain of research that apply it.
Can machines be trained to apply the criteria of inclusion and
exclusion? Can machines be trained on how to apply assessment of bias?
(Gough, Oliver, & Thomas, 2017). Theory-based SLRs, which summarise
evidence on what works, when and why, strive for more than greater
policy relevance. In fact, they strive for premise for decision-making
by top management in practice. For example, in the assessment of the
evaluation process of MIS products, King and Rodriguez (1978)
categorised assessments into four categories – attitudes, value
perceptions, information usage, and decision performance. Holistically,
each of these categories requires a systematic evaluation that is
exhaustive of existing literature to gather evidence on these categories
to avoid the pitfalls of common scientific basis, fallacy of affirming
the consequent, summative validity and called for basic reasoning that
accommodates the latest developments in positivist, interpretive,
action, and design research (Lee & Hubona, 2009). This ultimately leads
to technology adoption mindfulness in an era populated with mundane and
bleeding edge technologies. This is the case for digital health
technologies. These technologies still lack rigorous evaluations as with
methods used in evaluating them to provide sufficient/reliable evidence
on their functionalities and the most suitable circumstances in which
they will be more beneficial. Reviews that answer these questions adopt
a mixed methods approach and draw on a range of study types. Answering
the ‘what works’ and ‘what doesn’t’ questions mean drawing on
effectiveness studies, conducted to standards of high-quality impact
evaluation. But in formulating answers to the ‘when’ and ‘why’ questions
require a broader range of evidence from both quantitative and
qualitative research (Snilstveit, 2012) for policy relevance. Ideally,
this would normally be theory-based SLRs, which are usually mixed
method-based reviews (White, 2018). According to White, mixed methods
review is one which draws on a variety of evidence, factual and
counterfactual, qualitative and quantitative, to address different
questions along the causal chain (p.1). Drawing on different types of
studies theory-based SLRs are policy-focused (Snilstveit, 2012). Both
theory-based SLR and health technology assessment (HTA) are all
theory-focused evaluations.
Let us look at the concept of Health Technology Assessment (HTA), which
is the application of organized knowledge and skills in the form of
devices, medicines, vaccines, procedures, and systems developed to solve
a health problem and improve quality of lives” given the rampant
medical errors and system inefficiencies during the pre-digitization.
The World Bank also defined HTA as a complete policy-focused research
that evaluates short and long-term impact from the application
operationalization of technology, which include but not limited to
benefits, costs, risks (World Bank, 1995), and
accessibility/availability. With advanced health information
technologies such as electronic health/medical records and many more the
domain of health care has witnessed improvement in every facet of its
management and administration. From improved communications between
healthcare provider and consumer to improved medication safety,
tracking, and reporting; and promoting quality of care through optimized
access to and adherence to guidelines” (American College of
Obstetricians and Gynecologists, 2015, p.1). Health technologies can
range from medicine and medical devices, and pharmaceuticals as with
computer-aided information systems such as electronic healthcare records
(EHR) systems, health information exchange (HIE), personal health
records (PHR), national information networks, 3D printing, Artificial
intelligence (AI), etc are examples of computer- supported IS. Again, a
revisit to the definition of IS provides an understanding of the
social-technical aspect of IS vis-à-vis people and their roles to
accomplish task using technology. The challenge remains with the
availability of limited contribution of high-quality evaluations
resulting in ungeneralizable reviews in the context of policy and
practice implications, necessitating the need for further rigorous
studies. Whilst substantial progress has been made, however, there
challenges with the wider adoption of health information technologies
given the lack of concrete evidence to answer ‘when’ and ‘why’ questions
in the adoption of these technologies, costs and risks associated with
such technologies would need a structured-procedure/process-based
evaluative methodology with fidelity. These requirements fit well into
SLR.
There are not many studies that have provided guidance for conducting
SLR in general. Existing ones include (Okoli & Schabram, 2010;
Onwuegbuzie & Frels, 2016; Tranfield et al., 2003). The guide/steps
recommended by these authors are summarised in the Table 2 below.