Measuring and evaluating sexual health in the era of digital health: challenges and opportunities

Keywords: data, digital, health services, internet, online, public health, sexual health, STIs.

Digital health has expanded rapidly over the past two decades, a phenomenon that is particularly evident within sexual health. Although implementation has differed in approach, scale, and timeline across regions, digital health has become increasingly embedded within sexual health service delivery and is an established part of the user journey. Within high-income countries, the first major impacts of information and communication technology were the introduction of electronic health records (EHRs), capture and usage of data for surveillance purposes, and provision of laboratory results. This has more recently shifted to facilitating the remote testing and management of service users, with sexual health information, services and test results all being available online. Digital health interventions (DHIs) have also been implemented, integrated and scaled up within resource-constrained settings in different ways, many of which have utilised the availability and adaptability of mobile phones and other WiFi-enabled technology.¹

The World Health Organization (WHO) acknowledges the important role of digital health in supporting selfcare and helping to achieve a high standard of health and wellbeing globally, while recognising that it is an underexploited tool.² Within sexual health, there is huge potential for digital health to improve and increase access to screening, testing and care. Sexually transmitted infections (STIs) are stigmatised, often asymptomatic, transmissible and disproportionately affect marginalised groups. By overcoming traditional barriers to care, digital health could facilitate self-identification of need to access services, early diagnosis, linkage and management of index patients and contacts, utilising self-sampling and self-testing kits and digital care pathways, whilecapturing data required to monitor and evaluate services and maintain public health surveillance. However, evidence of the impact of digital sexual health service provision on access to care, efficiency and cost-effectiveness remains scarce.^3,4 There are increasing concerns about the inequity of access to digital technologies and the internet, particularly in low- and middle-income countries (LMIC), and the unintended consequences of the shift of health care online. Digital health has the potential to widen existing health inequalities, and the way that data is being utilised by third party providers is increasingly under scrutiny, due – in part – to a shortage of ‘fit-for-purpose’ legal and regulatory frameworks.

There has been recent recognition of the effect that digital health has had on the notion of universal health coverage (UHC) and the health of the public. To ensure equitable UHC and to meet public health needs, recommendations include proposing that the ‘digital ecosystem’ be recognised as an important determinant of health, empowering service users and marginalised populations to ensure health and digital rights, and utilising data solidarity to enable data justice and equity.⁵

One mechanism to ensure the emergent ‘digital ecosystem’ delivers on the promise of efficient delivery of universal and equitable sexual health is the rigorous measurement and evaluation of sexual health outcomes. We outline the potential of measuring and evaluating sexual health in the era of digital health, describing where we are, where we have and have not delivered, and providing recommendations for policy practice. We explore the role of digital health within the prevention and diagnosis of STIs/HIV and management of STIs. Drawing on implementation science, we frame this within a theory of change (Fig. 1), which illustrates what is required to fulfil the full potential of digital health to improve sexual health and wellbeing, and provide equitable sexual health care globally.^6,7

Fig. 1.  Theory of change.

Digital health can support provision of information and access to care across the sexual health continuum. This care continuum, illustrated within Fig. 2, includes sexual health promotion, screening for STIs, treatment and contact tracing and access to biomedical prevention (including pre- (PrEP) and post-exposure prophylaxis (PEP), emergency contraception and family planning). The stepped care model, introduced in the Netherlands over 10 years ago and now being developed and adopted in Kenya, South Africa and the United States, is one example of how efficiencies in health systems can be achieved through digital health.^8,9 Moreover, digital tools within sexual health have the potential to standardise and facilitate data capture for evaluation using implementation science frameworks, whether that is at the individual level (e.g. inputting information online to request a self-test or self-sampling test kit), service level (e.g. EHRs within clinics) or population level (e.g. multisystem surveillance of STIs).^6,7 The growth of implementation science, alongside developments within digital health and sexual health, provides us with the frameworks and methods to measure and evaluate the real-world implementation of these complex digital interventions.¹⁰ This provides an unprecedented opportunity to measure the reach, coverage, cost-effectiveness and impact of existing and novel sexual health interventions and programs, at a regional, national and international level to inform generalisability and scalability of novel and of effective digital approaches to improve the sexual health and quality of patient care.¹¹

Fig. 2.  Illustrating the sexual healthcare continuum, from demand creation to uptake and management and retention.

DHIs have the potential to identify those with sexual health needs sooner and provide tailored information and care, efficient access to testing and treatment, and support along the user journey. There is some evidence that online access to sexual health promotion, biomedical interventions, testing and treatment of STIs has high acceptability in different settings.^3,4,12 For those who are digitally literate and who have access to the internet with a device and mobile data/WiFi, digital health can overcome some of the existing barriers to timely sexual health care (Fig. 3), including for those who would otherwise not engage in testing.^13–16

Fig. 3.  Examples of how digital health can overcome existing barriers to sexual health care.

Digital health enables remote service delivery, including the provision of health and service information online, the ability for a service user to book appointments and access test kits, and to provide appointment reminders, test results and other information using SMS, email and web applications. Many DHIs, for example, online postal self-sampling services in England, have been developed to provide access to care to those with the least needs remotely in order to free up capacity to see more complex cases in clinic-based service.¹⁷ Digital health can also enable an intervention to provide tailored health promotion and service delivery to meet an individual’s needs.¹⁸

At a service level, digital health facilitates the capture and sharing of data throughout the sexual health continuum of care. The inputting of clinical information by healthcare providers and electronic capture of data within electronic health records allows providers to standardise and ensure completion of critical data fields within a clinical consultation. Developments in digital technology and diagnostics has enabled the capture of data and outcomes outside of traditional healthcare settings. For instance, smartphone technology has been used to interpret and capture the results of HIV self-tests,¹³ and healthcare workers or members of the public can use internet-enabled devices (e.g. a tablet or smartphone) to input clinical data and engage with digital health interventions,¹⁴ within community settings.

Digital technology is increasingly used to train and support healthcare providers. For example, it has been used to provide training on pre-exposure prophylaxis (PrEP) for HIV¹⁹ and for provision of training and support when delivering interventions to increase uptake of HIV testing.²⁰ The ability to deliver online training and support to providers has been particularly critical during the coronavirus disease 2019 (COVID-19) pandemic.²¹

The importance of interoperability of electronic health systems was first acknowledged at an international level 35 years ago with the introduction of the Health Level Seven (HLS7) international standards,¹⁵ which provide a framework for systems to be able share, retrieve and receive data with/from other services (e.g. laboratories and national surveillance systems). This framework allows for aggregation of data, facilitating analyses that allow us to understand who is accessing services, monitor and evaluate clinical outcomes, and perform audits that inform service delivery and support evaluations.

Digital systems can also allow services to comprehensively measure and evaluate the resource use and outcome data associated with different service pathways and components.¹⁶ DHIs often incorporate measures of resource use as part of their design, reducing the need to rely on patient recall or analysis of clinic records. A range of theoretical reductions in costs have been proposed for digital health technologies; for example, the potential to reduce reliance on physical spaces and staff in fixed locations (particularly for low-risk groups).²² This could allow services to increase provision to meet growing patient need and demand, in the context of limited funding. Such DHIs can also give patients more autonomy and increase self-monitoring and self-care, which can reduce the clinical time needed for tasks such as updating patient records and providing information.²³

Some digital services and interventions have been shown to be cost-saving, cost-effective or affordable compared to face-to-face services.²⁴ In LMIC settings, digital health, particularly m-health, has been seen as crucial to the expansion of sexual healthcare provision due to the challenges associated with limited staff resources and point-of-care services, particularly for communities in remote areas.²⁵

The use of digital technology at the user and service level allows the aggregation of captured data to be used for surveillance purposes at a population level. This in turn facilitates the early identification of infectious disease outbreaks,^3,26 enables the monitoring and evaluation of prevention interventions on the target population and health of the public, and provides evidence to inform future service delivery and policy. The WHO consolidated HIV strategic information guidelines recognise the utility of electronic data systems for capturing longitudinal data to monitor short-, medium- and long-term outcomes.¹¹ The aggregation of user and service-level data also allows the analysis of the costs associated with different conditions, assessment of the determinants of such costs, and investigations of the relationships between healthcare costs and explanatory variables.²⁷

Despite advances in digital health, many challenges remain in the measurement and evaluation of sexual health. Reaching underserved populations, ensuring that both the intervention and the outcomes being measured are appropriate, and consistent collection of data (across settings and over time) are all potential obstacles to a full realisation of the opportunities outlined above.

A key challenge, when thinking about digital health, is a concept known as the ‘digital divide’. Although technology has increased accessibility to a range of products, services, and experiences, increased digitisation has created its own form of inequity, due to the differential access to technology, digital literacy and comfort with digital health, and the cost of data that is experienced by certain groups. The digital divide creates barriers for many populations that have been historically marginalised; including racially minoritised groups,²⁸ women and girls,²⁹ older people, and low-income groups³⁰ and geographies.³¹ Although access to mobile technology and the internet now appears to be universal, disparities persist in both access and usage at a global and regional level. Within some LMIC, cell phone penetration is high; however, there are intraregional and gender differences in access and usage, with those living in urban areas 37% less likely to use mobile internet compared to individuals living in rural areas, and women 20% less likely to use mobile internet than men.³¹

Given the ubiquity of DHIs within sexual health, from health promotion, to online testing, to follow up by text message,³ it is likely that there are groups of people who are being excluded from sexual health services due to the increase in digitisation.²⁶ When considering the measurement and impact evaluation of sexual health, this gap is likely to have an impact on the information that we are able to collect. Reduced service access for underserved groups may lead to a poor understanding of the sexual health challenges that are unique to these populations, and may also give us a skewed perspective on the utility of digital interventions.

In the 21st century, health service users often experience a combination of clinic-based and online care; something that has become even more prevalent in the wake of the COVID-19 pandemic. Fig. 4 shows an example user journey for an individual wanting to access a repeat course of aciclovir for herpes simplex virus (HSV) suppression. In this scenario, there is duplication of data input, and multiple interactions with different healthcare providers through different media (phone, online and face-to-face), creating inefficiencies for individuals and services. In addition, current EHRs and other data capture systems are often not set up to be able to accurately capture this type of hybrid user journey, which makes capturing clinical, public health and economic outcomes challenging.

Fig. 4.  Example service user journey with data capture points (ACV, aciclovir; EHR, electronic health record; HCP, healthcare professional; HSV, herpes simplex virus; OPSS, online postal self-sampling; IM, intramuscular).

Traditionally, EHRs have been completed by a healthcare professional during a clinical consultation. In the wider medical literature, this has been shown to increase the quality of data capture and improve efficiency.³²

Self-completed online triage forms, surveys and consultations, whether within a clinical or research setting, have been shown to be a suitable substitute for face-to-face interviews in sexual health^33,34 and to facilitate the disclosure of sensitive information.^31,32 This can potentially reduce the requirement for people to repeatedly provide this information at subsequent points of engagement with the service. It can also aid service users who are diagnosed with an STI in the notification of sexual partners.³⁵

However, online data capture is reliant on the service user understanding and interpreting a question in the way that it is intended and being willing and able to disclose information online accurately. Much of the existing evidence on this is from questionnaires and assessments that have been cognitively tested and/or that use computer-assisted structured interviews in a research or clinic-based setting.³⁶ Digital health services that have been deployed in clinical practice may not have gone through the rigorous cognitive testing that has occurred in a research context. In addition, the data entry requirements need to be adjusted and tailored to reflect diverse levels of digital and health literacy. For example, a study in South Africa found that information entry needs to be minimal and that digital health interventions need to be user-friendly.¹⁴ However, the cost of data, lack of free-WiFi, and no or limited access to a cell- or smartphone remain a huge barrier for some people globally and prohibits initial engagement with a clinical or research DHIs.²¹

Trust and privacy are key factors that need to be taken into consideration when developing, monitoring and evaluating digital sexual health interventions to ensure uptake and engagement with an intervention, and willingness for data to be captured and shared for clinical and research purposes. Although digital health can overcome some barriers to accessing care, it can also introduce new ones (e.g. privacy concerns about STI test kits being delivered to homes).³⁷

The use of digital sexual health within traditional clinical settings also poses certain challenges. Within England for example, each clinic-based service has its own stand-alone EHR system. Patient identification numbers and EHRs are only shared within a commissioned service, and there is a lack of interoperability. EHRs have often been developed with billing or data collection in mind and are bought ‘off the shelf’, rather than being developed and tailored for a service’s requirements. Online services have been implemented and offered differently in different areas, with some serving as a stand-alone system for a large number of commissioned areas (e.g. Sexual Health London), whereas others are integrated to an existing clinic-based service (e.g. Umbrella service). This is in contrast to Scotland, where the same EHR and patient ID is used nationally and where there is ongoing work towards a national online postal self-sampling (OPSS) service.³⁸ The resultant variability within and between regions in streamlining of the user journey and data flow, means that these are often fragmented and can involve repetitive data capture (as illustrated in Fig. 4).

Decision-makers all around the world continually need to make difficult decisions around which sexual health services and interventions should be funded, and this can be particularly challenging in resource-constrained settings such as LMICs.³⁹ For complex interventions and programs, it can be difficult to measure the costs and outcomes comprehensively and in ways that are meaningful for stakeholders.⁴⁰ Very few economic evaluations of digital sexual health interventions and services have been undertaken, and those that are published are often limited in their scope.⁴¹ In particular, existing evidence has focused on a limited range of outcomes, short follow-up periods, and adopted a narrow perspective, which means a comprehensive understanding of sustainability over the longer term is not possible.¹⁷ In addition, outcomes for partners and transmission of infection are often not fully considered.

Data capture is often driven by a focus on costs, rather than considering costs alongside clinical and public health outcomes,⁴² and fails to capture the complexity of hybrid systems. In some cases, assumptions around cost savings are made without an in-depth understanding of how an intervention or service integrates with the wider sexual health economy, and hidden or spill-over costs may be missed. In addition, standard economic analyses often do not adequately capture the longer-term impacts associated with prevention of transmission and sequelae, particularly those that fall on other parts of the healthcare sector or other parts of society. Other factors that need to be more fully considered include the costs associated with staff training, investment in digital literacy for providers and users, new technologies, maintenance and updating, and ensuring stable internet provision for reliable digital services (particularly in LMICs).⁴³

The collection of data electronically means that data can be aggregated within mega-datasets that allow for large-scale understanding of trends within sexual health and facilitates a range of analyses.⁴⁴ There are, however, certain obstacles – particularly with regards to data collection – that compromise the utility of these surveillance systems in some cases. One barrier is the siloing of the data surrounding complex outcomes. There is excellent data linkage between some areas of the health service (e.g. the UK Health Security Agency Chlamydia Testing Activity Dataset links testing data from community, primary care and secondary care sources⁴⁵); however, tracing and mapping of the long-term sequelae of certain infections – such as pelvic inflammatory disease – is challenging, as this outcome data are collected predominantly within hospital datasets. In addition, data on certain reproductive health outcomes, such as termination of pregnancy, are also collected and held entirely separately from the data on sexual health outcomes. However, there are examples of where different datasets have been collated to understand the causal agents of complications of STIs (e.g. the proportion of pelvic inflammatory disease in Aboriginal women living in remote Australia that is attributable to chlamydia and/or gonorrhoea⁴⁶). The ability to link individuals across episodes of care, to capture short-, medium- and long-term outcomes of an infection or an intervention, would be greatly facilitated by individual-level data being linked by unique identifiers.¹¹ In order to maintain privacy and confidentiality, the ethical, regulatory and legal frameworks need to be in place to ensure the protection, and appropriate use, of personal and sensitive data, as well as the provision of safe and high-quality clinical care.⁴⁷ These frameworks will need to keep up with the rapid developments within this field so that they do not hinder innovation and development.⁴⁷

There is also the question of whether the large digital datasets held within the UK, for example, are able to capture all outcomes that are required to understand sexual health across the population. Most of the data collected are provided by sexual health services, who report the characteristics of each sexual health consultation. As these consultations are recorded electronically using proforma, it is likely that there are certain aspects of the interactions between healthcare providers and sexual health attendees that is not being captured. This is exacerbated by the absence of patient-reported outcomes for sexual health, and in the context of online self-completed consultations that are linear in nature, have only limited response options, and where no direct interaction between HCP and service user occurs. Sexual health remains stigmatised and disproportionately affects marginalised populations who may be reluctant to provide identifiable and sensitive information online – particularly in settings where, for example, sex work or same-sex sexual relationships are illegal.

More recently, ‘big data’, using either a single large dataset or combined datasets from different sources, have been interrogated using deep-learning algorithms to identify individuals who will benefit from pre-exposure prophylaxis for HIV (PrEP).^48,49 This use of data to target tailored interventions to individuals at risk of poor sexual health outcomes has huge potential to improve sexual health and wellbeing at both an individual and population level, as well as service-level efficiency and effectiveness. However, there are ethical implications that need to be considered with the use of artificial intelligence in this way.⁵⁰ This includes ‘missing data’, with those people who are not accessing care or who do have access to mobile technology and/or the internet being excluded from these datasets. In addition, the datasets used may not have been designed to capture the outcomes of interest or will not be interpreted within the context that they were collected.⁵⁰

Many of these challenges serve to exacerbate a part of health that is particularly difficult to measure – unmet need.⁵¹ In Fig. 2, individuals who do not get beyond ‘demand creation’, and therefore do not access care, are not captured as having a need and included within datasets that are then used to evaluate access to care. The barriers that certain groups face when seeking services due to increased digitisation, combined with the difficulty in using digital methods to capture the needs of the most complex patients, is likely to result in the exclusion of those who are most vulnerable to threats to their sexual health. When considering sexual health in the era of digital health, it is therefore vital to ensure that the increased inclusion that digital health enables does not result in other forms of exclusion.

There has been a growing recognition of the weaknesses of standard approaches of economic evaluation for public health interventions, which have aims beyond health alone. In particular, impacts on equity are not often taken into account, often a key consideration for stakeholders in this area.⁵² This is particularly important in a digital health context due to issues relating to unmet need and differential access to digital technologies across different social groups.

Turning to the future, unless we address the current inequity and other challenges described within this piece through a social justice lens, then we will not fulfil the true potential of digital health and could potentially widen existing health inequalities. Using our theory of change model, the following need to happen for DHIs to improve sexual health and wellbeing, and reduce morbidity at a global level:

• 1. Ensure the necessary infrastructure is in place to provide access for those with a sexual health need

The unmet healthcare, laboratory, digital and surveillance infrastructure needs to be mapped. Closing the gap should be key to the evaluation and included in the ongoing measurements. Where gaps exist, the missing infrastructure needs to be built with the flexibility to adapt service to the context as it changes. The same needs to happen in terms of education, to ensure that people have sufficient health and digital literacy to be empowered to initiate and maintain access to services.

• 2. Patient and community engagement

A critical factor to ensure equity is for interventions to be developed with and for the end users, and by empowering people and communities to take control of their own health.⁵³ Target end user and community engagement from the initial conception of an intervention, through co-development and co-creation of the intervention. End users are also key to establishing outcomes of importance and methods of evaluation that capture unmet need.

• 3. Tailoring and layering of interventions

In order to provide cost-effective equitable access, digital technology has the potential to allow us to tailor interventions and services to meet individual user’s needs. This differentiated care will allow those with the least needs to receive the most basic level of care/intervention, whereas those with the most needs are offered a complex package of care (whether that is digital or clinic-based services).

• 4. Integrating the digital ecosystem with the existing healthcare and external ecosystem

Effective implementation and deployment of DHIs to improve access to primary and secondary STI and HIV prevention requires integration with existing successful sexual health interventions and service delivery. For example, within South Africa, the roll-out of PrEP was informed by experience of what had worked successfully with contraception,⁵⁴ and integrating screening for gender-based violence with HIV testing has been found to be acceptable for young women.⁵⁵

As well as integration occurring at the healthcare service delivery level, it also needs to occur at the level of data capture and measurement, so that the whole individual user journey and clinical, public health and economic outcomes are captured accurately, for each episode of care and over time. To do this, the necessary legal and regulatory frameworks need to be in place that protect individual personal and sensitive data, and engender trust in the system.

• 5. Measure and evaluate unmet need, equity and quality of the experience

By empowering the end user, providing the necessary infrastructure and ensuring accurate data capture, within an ethical legal and regulatory framework and informed by implementation science, it should be possible to accurately map, monitor, and evaluate the unmet need and quality of the end user experience using quantitative and qualitative methods, and iteratively refine the intervention to fill the gaps. These data can also be used to identify the effectiveness of an intervention and identify unintended consequences. Given the complexity of sexual DHIs and programs, a realist evaluation approach is required to understand what works for whom, in what context and in which settings.⁵⁶

• 6. Measure and evaluate the economic and distributional impacts associated with DHIs and services in sexual health

Methods for incorporating equity considerations within economic evaluations are being refined and a range of approaches have been proposed.⁵⁷ As such methods have not previously been used in a sexual health context, we must continue to be informed by emerging practice in multiple fields.⁵⁸

Data sharing is not applicable as no new data were generated or analysed during this study.

The authors declare no conflicts of interest.

This report is independent research supported by the NIHR ARC North Thames. JG, FB and LJ report receiving NIHR funding to research digital sexual health (NIHR129157 (FB, JG, LJ) and NIHR200856 (JG)). DS holds a clinical fellowship funded by the Wellcome Trust (102186/B/13/Z). SM receives funding from Unitaid (2017-21-Wits-PrEP); FHI360 (639391218); USAID (72067419CA00001); the Bill and Melinda Gates Foundation (INV-022667). MS is an NIHR Global Research Professor (NIHR 301634). MS also receives funding from US National Institutes of Health (NIH) R01 (5R01MH114560-03); Wellcome Trust (082384/Z/07/Z) and the Bill and Melinda Gates Foundation (INV-033650). This study is funded by the NIHR Health Services and Delivery Research Programme (NIHR129157). For the purpose of open access, the authors have applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission. The views expressed are those of the authors and not necessarily those of the NIHR, the Department of Health and Social Care, NIH, the Bill and Melinda Gates Foundation or the Wellcome Trust.