Scoping review + SOCI: to assess DHIS2 applicability for NCD

research methods
1

Hi there!

I am Oleg, graduated student from Belgrade University. It is my thesis work and I am looking for community assesment.

With research team from the University of Belgrade. We are conducting a scoping review on the use of DHIS2 and DHIS2-based health information systems for non-communicable diseases (NCDs).

OSF LINK [ OSF | Applicability of DHIS2 software platform for monitoring non-communicable diseases in Serbia and Southeast Europe: Scoping Review].

Now I will explain how I imagine to combine SOCI Serbia and DHIS2 scoping review. Please assess this approach:

The second is intensity: the average, across studies, of the share of all barrier (or facilitator) statements within each study that are attributed to a given domain.

To intergarte Serbian context, I decided to use SOCI 2024 maturity level. Each criterion is rated from 1 to 5, normalized to a 0-1 scale.

From this constructed two parametrs:

The Priority index combines the prevalence of barriers with the readiness deficit, that is, how much more the country has to do in this area. Both are multiplied, not added, because a field can be considered high priority only if both criteria are met: in literature it is defined as a widespread problem and the country is not yet ready for it. Otherwise, priority is automatically lowered if one of the criteria is weak.

The Opportunity index is the exact opposite: it combines both the prevalence of facilitators and readiness. If the literature highlights positive experiences and the country has a decent basis to start with, it is considered a domain with high opportunity, which means quick wins can be possible.

Finally, I calculated an overall applicability indicator. In each domain, we looked at two things: whether the evidence is mostly from facilitators or barriers (depending on intensity), and whether Serbia’s SOCI score meets the midpoint of the scale.

Domains where both are favourable were rated Met. Domains where the picture is mixed were rated Partly met. Domains where both are unfavourable would be rated Unmet. Each rating was converted to a simple score (2, 1, or 0), and the sum across all five domains, divided by the maximum possible, gave us an overall applicability percentage of 70%, indicating moderate-to-high applicability of DHIS2 for NCD surveillance in Serbia.

I have next Questions

1. Threshold selection
Assuming that 2.5 is the middle of the SOCI scale, I used it as a threshold. Is this valid?

2. Interpreting Opportunity vs. Priority
I am examining various ways to interpret the correlation between the Opportunity and Priority indices

Should I keep both parameters and interpret them independently?
Or Is it better to combine them into a single metric, like Net Balance (Opportunity - Priority), and pay attention to the difference?

In addition, would it be reasonable to interpret based on rules, for instance:
Similar values = system is balanced (no immediate intervention needed)
Opportunity > Priority = focus on scaling strengths
Priority > Opportunity = prioritize fixing gaps

  1. I would be grateful if you could evaluate my approach and the formulas I used. It is important for me to receive comprehensive and honest feedback, including any critical remarks. Thank you in advance to everyone who takes the time to share their thoughts.

**
Example of All formulas and tables are provided Below**

1. Study-level totals (per study i)

1.1 Total number of barriers in study (i) based on SOCI Domains .

\text{Bar}_{\text{total}}(i) = \text{Bar}_{\text{L~and~G}}(i) + \text{Bar}_{\text{M~and~W}}(i) + \text{Bar}_{\text{ICT}}(i) + \text{Bar}_{\text{S~and~I}}(i) + \text{Bar}_{\text{DQ~and~U}}(i)

Explanation: Bar_total(i) is the sum of all barrier statements in study i across the five SOCI domains (Leadership and Governance, Management and Workforce, ICT Infrastructure, Standards and Interoperability, Data Quality and Us e).

Example (Study #5 from raw data): Study #5 has 3 L and G barriers, 2 M and W barriers, 1 ICT barrier, 0 S and I barriers, and 8 DQ and U barrie rs.

Bar_total(5) = 3 + 2 + 1 + 0 + 8 = 14

1.2 Total number of facilitators in stu dy (i)

\text{Fac}_{\text{total}}(i) = \text{Fac}_{\text{L~and~G}}(i) + \text{Fac}_{\text{M~and~W}}(i) + \text{Fac}_{\text{ICT}}(i) + \text{Fac}_{\text{S~and~I}}(i) + \text{Fac}_{\text{DQ~and~U}}(i)

Explanation:Fac_total(i) is the sum of all facilitator statements in study (i) across the five SOCI domains .

2. Prevalence per SOCI dom ain d

2.1 Barrier preva lence(d)

Formula:Barrier Prevalence(d) = Number of studies with ≥1 barrier in domain d / Total number of stu dies (19 )

Explanation:This is the proportion of included studies that report at least one barrier in a given domain. If 15 out of 19 studies have a DQ and U barrier, the barrier prevalence for DQ and U is 15/1 9.

Example (Data Quality and Use):In the Excel file, 18 out of 19 studies have at least one DQ and U barrier.
Barrier Prevalence (DQ and U) = 18 / 19 = 0.947 (94.7%).

Formula:Facilitator Prevalence(d) = Number of studies with ≥1 facilitator in domain d / Total number of studies (19)
Explanation:This is the proportion of studies that mention at least one facilitator in a g iven domain.

2 Likes
2

Welcome, Oleg! As a new user, you probably aren’t permitted to add very many links to your first post. So I will just drop 3 here, which I have found interesting to look up:

I hadn’t heard of DHIS2 before; the Digital public goods - Wikipedia concept is appealing.

As for the formulas you present, they remind me very much of the arbitrary ‘scoring systems’ often appearing in primitive clinical assessment tools such as the Centor criteria - Wikipedia. My feeling is that your formulas don’t warrant criticism in and of themselves; rather, their arbitrary nature points to a bigger problem.

That problem is that this whole enterprise looks like piling abstraction upon abstraction, each one supporting a new layer of bureaucracy 1 further step removed from the real problems. I noticed that the HIS SOCI link above was funded by USAID, which famously has failed to achieve localization.

One hard question to ask yourself: even if you pursue this effort with all due diligence and sincerity, what would distinguish it from something generated by an LLM? My sense is that the only way to ensure such a distinction is to adopt an actively critical stance — starting with posing a sharply-defined problem. One way to do this would be to contrast DHIS2 against one or more competing technologies. Perhaps you could consider a specific non-communicable disease. Perhaps you could examine one particular aspect of adoption, such as how difficult it is to develop a new mobile app or data dashboard in DHIS2 vs competitors.

2 Likes
3

Thank you very much for your feedback. I truly appreciate you taking the time to review my post and leave the ideas!

I want to make clear the purpose of the study. Protocol is not strictly tied to the SOCI framework. The main aim of this scoping review is “to map the evidence of applicability and benefits of using DHIS2 and related programs for NCDs through a scoping review of existing literature” and identify barriers and facilitators. Then develop criteria to assess the feasibility of implementing this system for NCD in Serbia. My starting point was SOCI. I get inspired from the Ethiopian scientific articles where they evaluated their HIS with SOCI, and then with same domains DHIS2. I found it attracrive because recently here is the study of SOCI for Serbian HIS.

I am not a professional statistician, so I am not trying to make complex mathematical or statistical models.

The methodology is made to qualitatively compare textual data from the review with reality in Serbia. The goal is to reach a clear conclusion: “met,” “partially met,” or “not met.” Your suggestion to narrow the focus to a specific themes, and compare DHIS2 with alternatives is excellent. That would be a great direction for a follow-up study. Thank you fot that!

This review is serve as an exploratory first step by mapping the global experience. Your advice would be valuable since I am not married to SOCI.

If you feel that this framework is not an appropriate choice, what would you suggest for assessing country readiness based on scoping review data? How would you personally link extracted qualitative barriers to a final assessment? Your guidance would be extremely helpful!

1 Like