3.2 Step 2: Set your inclusion and exclusion criteria.
Prior to commencing the actual search activity, the researchers need to
develop a protocol to guide the selection of would-include and -exclude
study materials. This is determined after the research question has been
formulated and is used to set the delineations – in/out-scope of the
study. Inclusion criteria are the factors that an extant study must have
to answer the focused research question set. On the contrary, exclusion
criteria are all the characteristics of extant study that do not in any
way contribute to answering the research question. These criteria could
be study objective or purpose, year study was carried out or published,
peer-reviewed, study methodology, study population and sample size, age,
language in which study was published, geographical location of study,
theory-/non-theory-based review etc.
When the study is theory-based, inclusion and exclusion criteria should
focus on IS interventions that have been evaluated and the use of theory
of change. Over the last decade or so, IS has seen increased in
implementation science, an equivalent of process evaluation to verify
the fidelity of system implementation activities. For example, there is
no doubt about the effectiveness of digital health. However, little is
known about the unintended consequences of digital health from a system
point of view, making it difficult to institutionalize eHealth into
routine care as with the challenge of reforming health policies in this
regard.
In IS, evaluation is the appraisal of the quality and value of the
characteristics/functions of systems and their impact on end-user and on
business operations in an organization. It is “the act of measuring or
exploring properties of a health information system (in planning,
development, implementation, or operation) (Ammenwerth et al., 2004,
p.480). Conventionally, evaluations in IS has been impact evaluation, an
assessment of how an IS intervention been assessed affects existing
conditions to bring about needed change. For example, when evaluating
the effectiveness of an IS, attention is focused on the impact of an
implemented system on organizational/business operations in the context
of the effectiveness of users and other outputs (Hamilton & Chervany,
1981). Again, this could have also been how the feature/functionality of
a system impacted end-user perception. Furthermore, the evaluation of
health information technologies for high-quality, safe and
cost-effectiveness to improve quality of and increase access to
healthcare. IS requires rigorous and effective evaluation with fidelity.
“Interestingly, change theory has not been much explored in IS
research.
A SLR protocol provides guides for undertaking systematic literature
review study and help to minimise risks of bias whilst optimising
standardization/uniformity resulting in increased transparency and
fidelity. It is a mini-guide or reference within the broader guide for
conducting SLR. In fact, the protocol services as a quick reference tool
for the researcher(s). Components of the study protocol include but not
limited to the research question to be answered, study selection
criteria and procedures, eligibility, checklist for study quality
assessment, search strategy, etc. (Kitchenham, 2004) and above all the
methods for performing synthesis (Ryan, 2019). Examples of tools that
help in developing protocol are PRISMA checklist, Theta Collaborative
tool, Cochrane Review handbook, AMSTAR checklist.